https://arxiv.org/api/rYr4iJ0olrsutMj5zg9uCiNiFX4 2026-06-13T13:46:11Z 6065 390 15 http://arxiv.org/abs/2601.16988v1 2026-01-06T04:36:51Z

The rapid expansion of scholarly publications across diverse disciplines has made it increasingly difficult to systematically evaluate how research contributes to the United Nations Sustainable Development Goals (SDGs). Domain classification of research articles done manually through research experts is extremely impractical because of the number of publications, expensive in time and may not be consistent when done by human beings. This paper proposes an automated and rule-based computational model of classifying research papers based on SDGs with expert curated Boolean query mappings to overcome these challenges. The proposed system has a web-based interface to input data and display results, a backend application programming interface to do high throughput processing, and a Python-based classification engine which uses structured Boolean expressions to process bibliographic metadata (titles, abstracts, and keywords). The framework can be used to support single-paper-based classification and batch-based classification as well as offer clear and understandable outputs that clearly show what query parts motivated each SDG assignment. The experimental testing on massive bibliographic data sets has shown that the system can process thousands of research records in an hour with reproducible and consistent results. The proposed approach provides a viable solution to institutions, researchers and policymakers who are interested in analysis of research alignment with the goal of sustainability in a systematic fashion that would not involve the use of machine learning models whose inputs and outputs are not easily understandable.

2026-01-06T04:36:51Z 16 pages, 8 figures and 1 table Sahil Dewani Kiran Sharma http://arxiv.org/abs/2508.15834v2 2026-01-05T19:28:57Z

Research profiles highlight scientists' research focus, enabling talent discovery and collaborations, but are often outdated. Automated, scalable methods are urgently needed to keep profiles current. We design and evaluate two Large Language Models (LLMs)-based methods to generate scientific interest profiles--one summarizing PubMed abstracts and the other using Medical Subject Headings (MeSH) terms--comparing them with researchers' self-summarized interests. We collected titles, MeSH terms, and abstracts of PubMed publications for 595 faculty at Columbia University Irving Medical Center, obtaining human-written profiles for 167. GPT-4o-mini was prompted to summarize each researcher's interests. Manual and automated evaluations characterized similarities between machine-generated and self-written profiles. The similarity study showed low ROUGE-L, BLEU, and METEOR scores, reflecting little terminological overlap. BERTScore analysis revealed moderate semantic similarity (F1: 0.542 for MeSH-based, 0.555 for abstract-based), despite low lexical overlap. In validation, paraphrased summaries achieved a higher F1 of 0.851. Comparing original and manually paraphrased summaries indicated limitations of such metrics. Kullback-Leibler (KL) Divergence of TF-IDF values (8.56 for MeSH-based, 8.58 for abstract-based) suggests machine summaries employ different keywords than human-written ones. Manual reviews showed 77.78% rated MeSH-based profiling "good" or "excellent," with readability rated favorably in 93.44% of cases, though granularity and accuracy varied. Panel reviews favored 67.86% of MeSH-derived profiles over abstract-derived ones. LLMs promise to automate scientific interest profiling at scale. MeSH-derived profiles have better readability than abstract-derived ones. Machine-generated summaries differ from human-written ones in concept choice, with the latter initiating more novel ideas.

2025-08-19T03:45:39Z Journal of Biomedical Informatics 172, 104949 (2025) Yilun Liang Gongbo Zhang Edward Sun Betina Idnay Yilu Fang Fangyi Chen Casey Ta Yifan Peng Chunhua Weng 10.1016/j.jbi.2025.104949. http://arxiv.org/abs/2504.20600v2 2026-01-05T10:46:32Z

We propose a citation index $ν$ (``nu'') and show that it lies between the classical $h$-index and $g$-index. This idea is then generalized to a monotone parametric family $(ν_α)$ ($α\ge 0$), whereby $h=ν_0$ and $ν=ν_1$, while the limiting value $ν_\infty$ is expressed in terms of the maximum citation.

2025-04-29T10:01:38Z 15 pages, 1 table, 2 figures Ruheyan Nuermaimaiti Leonid V. Bogachev Jochen Voss http://arxiv.org/abs/2601.01764v1 2026-01-05T03:44:00Z

Overton, a global policy index, provides new opportunities to study the interactions between science and policy. This study aims to characterize the presence of scholarly and policy references in Overton-indexed policy documents and examine their distribution across key bibliographic dimensions, thereby assessing Overton's potential as a data source for policy metrics. We analyze a dataset of approximately 17.5 million policy documents from Overton, incorporating metadata such as publication year, policy source, country, language, subject area, and policy topic. Descriptive statistics are employed to assess the presence and distribution of reference data across these dimensions. Overton indexes a substantial volume of policy documents and identifies considerable reference data within them: 7.7% of documents contain scholarly references and 10.6% contain policy references. However, the presence of references varies significantly across publications years, source types, countries, languages, subject areas, and policy topics, indicating coverage biases that may affect interpretations of policy impact. The analysis is based on the Overton database as of June 2025. As Overton is regularly updated, the distribution patterns of indexed documents and references may evolve over time. The findings offer insights into the opportunities and constraints of using Overton for investigating evidence-based policymaking and for assessing the policy uptake of research outputs in the context of research evaluation. This is the first large-scale study to systematically examine the distribution of reference data in Overton. It contributes a foundational understanding of this emerging source for policy metrics, highlighting both its potential applications and limitations, and underlining the importance of addressing current coverage imbalances.

2026-01-05T03:44:00Z Journal of Data and Information Science, 2026, 11(1), 63-87 Biegzat Murat Zhichao Fang Ed Noyons Rodrigo Costas 10.2478/jdis-2025-0054 http://arxiv.org/abs/2601.01716v1 2026-01-05T01:39:28Z

In this study, we systematically elucidate the background and functionality of the Scilit database and evaluate the feasibility and advantages of the comprehensive impact metrics I3 and I3/N, introduced within the Scilit framework. Using a matched dataset of 17,816 journals, we conduct a comparative analysis of Scilit I3/N, Journal Impact Factor, and CiteScore for 2023 and 2024, covering descriptive statistics and distributional characteristics from both disciplinary and publisher perspectives. The comparison reveals that the Scilit I3 and I3/N framework significantly outperforms traditional mean-based metrics in terms of coverage, methodological robustness, and disciplinary fairness. It provides a more accurate, diagnosable, and responsible solution for interdisciplinary journal impact assessment. Our research serves as a "getting started guide" for Scilit, offering scholars, librarians, and academic publishers in the fields of bibliometrics or scientometrics a valuable perspective for exploring I3 and I3/N within an inclusive database. This enables a more accurate and comprehensive understanding of disciplinary development and scientific progress. We advocate for piloting and validating this method in broader evaluation contexts to foster a more precise and diverse representation of scientific progress.

2026-01-05T01:39:28Z Haochen Dong Sun Qiao Yanping Mu Lu Liao Diogo Rodrigues Frank Sauerburger Yi Bu Robin Haunschild http://arxiv.org/abs/2509.19833v3 2026-01-04T23:19:53Z

The United Nations' Sustainable Development Goals (SDGs) provide a globally recognised framework for addressing critical societal, environmental, and economic challenges. Recent developments in natural language processing (NLP) and large language models (LLMs) have facilitated the automatic classification of textual data according to their relevance to specific SDGs. Nevertheless, in many applications, it is equally important to determine the directionality of this relevance; that is, to assess whether the described impact is positive, neutral, or negative. To tackle this challenge, we propose the novel task of SDG polarity detection, which assesses whether a text segment indicates progress toward a specific SDG or conveys an intention to achieve such progress. To support research in this area, we introduce SDG-POD, a benchmark dataset designed specifically for this task, combining original and synthetically generated data. We perform a comprehensive evaluation using six state-of-the-art large LLMs, considering both zero-shot and fine-tuned configurations. Our results suggest that the task remains challenging for the current generation of LLMs. Nevertheless, some fine-tuned models, particularly QWQ-32B, achieve good performance, especially on specific Sustainable Development Goals such as SDG-9 (Industry, Innovation and Infrastructure), SDG-12 (Responsible Consumption and Production), and SDG-15 (Life on Land). Furthermore, we demonstrate that augmenting the fine-tuning dataset with synthetically generated examples yields improved model performance on this task. This result highlights the effectiveness of data enrichment techniques in addressing the challenges of this resource-constrained domain. This work advances the methodological toolkit for sustainability monitoring and provides actionable insights into the development of efficient, high-performing polarity detection systems.

2025-09-24T07:23:44Z Updated as one author was mispelled Andrea Cadeddu Alessandro Chessa Vincenzo De Leo Gianni Fenu Francesco Osborne Diego Reforgiato Recupero Angelo Salatino Luca Secchi http://arxiv.org/abs/2601.01118v1 2026-01-03T08:42:53Z

The rapid growth of AI for Science (AI4S) has underscored the significance of scientific datasets, leading to the establishment of numerous national scientific data centers and sharing platforms. Despite this progress, efficiently promoting dataset sharing and utilization for scientific research remains challenging. Scientific datasets contain intricate domain-specific knowledge and contexts, rendering traditional collaborative filtering-based recommenders inadequate. Recent advances in Large Language Models (LLMs) offer unprecedented opportunities to build conversational agents capable of deep semantic understanding and personalized recommendations. In response, we present ScienceDB AI, a novel LLM-driven agentic recommender system developed on Science Data Bank (ScienceDB), one of the largest global scientific data-sharing platforms. ScienceDB AI leverages natural language conversations and deep reasoning to accurately recommend datasets aligned with researchers' scientific intents and evolving requirements. The system introduces several innovations: a Scientific Intention Perceptor to extract structured experimental elements from complicated queries, a Structured Memory Compressor to manage multi-turn dialogues effectively, and a Trustworthy Retrieval-Augmented Generation (Trustworthy RAG) framework. The Trustworthy RAG employs a two-stage retrieval mechanism and provides citable dataset references via Citable Scientific Task Record (CSTR) identifiers, enhancing recommendation trustworthiness and reproducibility. Through extensive offline and online experiments using over 10 million real-world datasets, ScienceDB AI has demonstrated significant effectiveness. To our knowledge, ScienceDB AI is the first LLM-driven conversational recommender tailored explicitly for large-scale scientific dataset sharing services. The platform is publicly accessible at: https://ai.scidb.cn/en.

2026-01-03T08:42:53Z 12 pages, 9 figures Qingqing Long Haotian Chen Chenyang Zhao Xiaolei Du Xuezhi Wang Pengyao Wang Chengzan Li Yuanchun Zhou Hengshu Zhu http://arxiv.org/abs/2601.00871v1 2025-12-30T15:29:39Z

This study offers a new perspective on the depth-versus-breadth debate in innovation strategy, by modeling inventive search within dynamic collective knowledge systems, and underscoring the importance of timing for technological impact. Using frontier machine learning to project patent citation networks in hyperbolic space, we analyze 4.9 million U.S. patents to examine how search strategies give rise to distinct temporal patterns in impact accumulation. We find that inventions based on deep search, which relies on a specialized understanding of complex recombination structures, drive higher short-term impact through early adoption within specialized communities, but face diminishing returns as innovations become "locked-in" with limited diffusion potential. Conversely, when inventions are grounded in broad search that spans disparate domains, they encounter initial resistance but achieve wider diffusion and greater long-term impact by reaching cognitively diverse audiences. Individual inventions require both depth and breadth for stable impact. Organizations can strategically balance approaches across multiple inventions: using depth to build reliable technological infrastructure while pursuing breadth to expand applications. We advance innovation theory by demonstrating how deep and broad search strategies distinctly shape the timing and trajectory of technological impact, and how individual inventors and organizations can leverage these mechanisms to balance exploitation and exploration.

2025-12-30T15:29:39Z 47 pages, 8 figures, 3 tables Likun Cao James Evans http://arxiv.org/abs/2203.17259v4 2025-12-30T13:47:19Z

Double-blind conferences have engaged in debates over whether to allow authors to post their papers online on arXiv or elsewhere during the review process. Independently, some authors of research papers face the dilemma of whether to put their papers on arXiv due to its pros and cons. We conduct a study to substantiate this debate and dilemma via quantitative measurements. Specifically, we conducted surveys of reviewers in two top-tier double-blind computer science conferences -- ICML 2021 (5361 submissions and 4699 reviewers) and EC 2021 (498 submissions and 190 reviewers). Our three main findings are as follows. First, more than a third of the reviewers self-report searching online for a paper they are assigned to review. Second, conference policies restricting authors from publicising their work on social media or posting preprints before the review process may have only limited effectiveness in maintaining anonymity. Third, outside the review process, we find that preprints from better-ranked institutions experience a very small increase in visibility compared to preprints from other institutions.

2022-03-31T17:56:24Z 18 pages, 3 figures Charvi Rastogi Ivan Stelmakh Xinwei Shen Marina Meila Federico Echenique Shuchi Chawla Nihar B. Shah http://arxiv.org/abs/2512.23882v1 2025-12-29T21:58:32Z

Multiple organisational affiliations are an increasingly common feature of research systems, yet their implications for organisational performance had received limited systematic attention. We developed a scalable, network-based analytical framework that represents simultaneous researcher affiliations as relational links between organisations and applied it to bibliometric data from Austria. Using harmonised publication and affiliation metadata, we constructed two complementary co-affiliation networks: a complete network capturing all simultaneous affiliations and a temporally filtered network retaining only organisational pairs that recurred over time. Network regression analyses showed that geographical proximity remained an important determinant of co-affiliation formation, with spatial distance consistently reducing shared appointments. Clear sectoral differences emerged beyond geography. Universities formed a dense and persistent core of co-affiliations, whereas ties involving medical institutions, government, non-profit and private-sector organisations were often short-lived and attenuated under temporal filtering. Among crosssector links, co-affiliations between universities and research institutes were notably resilient, indicating a more structurally embedded form of organisational integration. We assessed the effect of concurrent affiliations on organisational citation impact across organisational types using field- and year-normalised indicators. Research institutes and universities consistently exhibited higher citation impact than organisations from other sectors, and persistent co-affiliations were associated with greater and more stable scientific visibility.

2025-12-29T21:58:32Z Quantitative Science Studies 2026 Christoph Schlager Lutz Bornmann Gerald Schweiger 10.1162/QSS.a.486 http://arxiv.org/abs/2512.23429v1 2025-12-29T12:49:21Z

The influence of gender diversity on the success of scientific teams is of great interest to academia. However, prior findings remain inconsistent, and most studies operationalize diversity in aggregate terms, overlooking internal role differentiation. This limitation obscures a more nuanced understanding of how gender diversity shapes team impact. In particular, the effect of gender diversity across different team roles remains poorly understood. To this end, we define a scientific team as all coauthors of a paper and measure team impact through five-year citation counts. Using author contribution statements, we classified members into leadership and support roles. Drawing on more than 130,000 papers from PLOS journals, most of which are in biomedical-related disciplines, we employed multivariable regression to examine the association between gender diversity in these roles and team impact. Furthermore, we apply a threshold regression model to investigate how team size moderates this relationship. The results show that (1) the relationship between gender diversity and team impact follows an inverted U-shape for both leadership and support groups; (2) teams with an all-female leadership group and an all-male support group achieve higher impact than other team types. Interestingly, (3) the effect of leadership-group gender diversity is significantly negative for small teams but becomes positive and statistically insignificant in large teams. In contrast, the estimates for support-group gender diversity remain significant and positive, regardless of team size.

2025-12-29T12:49:21Z Journal of Informetrics, 2026 Yi Zhao Yongjun Zhu Donghun Kim Yuzhuo Wang Heng Zhang Chao Lu Chengzhi Zhang 10.1016/j.joi.2025.101766 http://arxiv.org/abs/2512.22524v1 2025-12-27T08:58:23Z

Subject classification schemes are foundational to the organization, evaluation, and navigation of scientific knowledge. While expert-curated systems like Scopus provide widely used taxonomies, they often suffer from coarse granularity, subjectivity, and limited adaptability to emerging interdisciplinary fields. Data-driven alternatives based on citation networks show promise but lack rigorous, external validation against the semantic content of scientific literature. Here, we propose a novel quantitative framework that leverages classification tasks to evaluate the effectiveness of journal classification schemes. Using over 23 million paper abstracts, we demonstrate that labels derived from k-means clustering on Periodical2Vec (P2V)--a periodical embedding learned from paper-level citations--yield significantly higher classification performance than both Scopus and other data-driven baselines (e.g., citation, co-citation, and Node2Vec variants). By comparing journal partitions across classification schemes, two structural patterns emerge on the map of science: (1) the reorganization of disciplinary boundaries--splitting overly broad categories (e.g., "Medicine" into "Oncology", "Cardiology", and other specialties) while merging artificially fragmented ones (e.g., "Chemistry" and "Chemical Engineering"); and (2) the identification of coherent interdisciplinary clusters--such as "Biomedical Engineering", "Medical Ethics", and "Information Management"--that are dispersed across multiple categories but unified in citation space. These findings underscore that citation-derived periodical embeddings not only outperform traditional taxonomies in predictive validity but also offer a dynamic, fine-grained map of science that better reflects both the specialization and interdisciplinarity inherent in contemporary research.

2025-12-27T08:58:23Z Zhuoqi Lyu Qing Ke http://arxiv.org/abs/2512.21832v1 2025-12-26T02:24:17Z

While scholarly citations are pivotal for assessing academic impact, they often reflect systemic biases beyond research quality. This study examines a critical yet underexplored driver of citation disparities: authors' structural positions within scientific collaboration networks. Through a large-scale analysis of 17,942 papers from three top-tier machine learning conferences (NeurIPS, ICML, ICLR) published between 2005 and 2024, we quantify the influence of author centrality on citations. Methodologically, we advance the field by employing beta regression to model citation percentiles, which appropriately accounts for the bounded nature of citation data. We also propose a novel centrality metric, Harmonic Closeness with Temporal and Collaboration Count Decay (HCTCD), which incorporates temporal decay and collaboration intensity. Our results robustly demonstrate that long-term centrality exerts a significantly stronger effect on citation percentiles than short-term metrics, with closeness centrality and HCTCD emerging as the most potent predictors. Importantly, team-level centrality aggregation, particularly through exponentially weighted summation, explains citation variance more effectively than conventional rank-based approaches, underscoring the primacy of collective network connectivity over individual prominence. Integrating centrality features into machine learning models yields a 2.4% to 4.8% reduction in prediction error (MSE), confirming their value beyond content-based benchmarks. These findings challenge entrenched evaluation paradigms and advocate for network-aware assessment frameworks to mitigate structural inequities in scientific recognition.

2025-12-26T02:24:17Z Renlong Jie Longfeng Zhao Chen Chu Danyang Jia Zhen Wang http://arxiv.org/abs/2512.04448v2 2025-12-24T10:20:31Z

The recent surge of language models (LMs) has rapidly expanded NLP/AI research, driving an exponential rise in submissions and acceptances at major conferences. Yet this growth has been shadowed by escalating concerns over conference quality, such as plagiarism, reviewer inexperience, and collusive bidding. However, existing studies rely largely on qualitative accounts, for example expert interviews and social media discussions, lacking longitudinal empirical evidence. To fill this gap, we conduct a ten-year empirical study (2014-2024) spanning seven leading conferences. We build a four-dimensional bibliometric framework covering conference scale, core citation statistics, impact dispersion, and cross-venue and journal influence. Notably, we further propose a metric called Quality-Quantity Elasticity (QQE), which measures the elasticity of citation growth relative to acceptance growth. We highlight two key findings. First, conference expansion does not lead to proportional growth in scholarly impact, as QQE consistently declines over time across all venues. Second, ACL has not lost its crown, continuing to outperform other NLP conferences in median citations, milestone contributions, and citation coverage. This study provides the first decade-long, cross-venue empirical evidence on the evolution of major NLP/AI conferences. Our code is available at https://anonymous.4open.science/r/acl-crown-analysis-38D5.

2025-12-04T04:39:40Z Jianglin Ma Ben Yao Xiang Li Yazhou Zhang http://arxiv.org/abs/2509.22013v2 2025-12-23T07:35:53Z

This study explores funding, authorship patterns, and citation impact of articles funded by the Ministry of Education and Science of Ukraine (MESU), the National Academy of Sciences of Ukraine (NASU), and the National Research Foundation of Ukraine (NRFU). The analysis focuses on articles published in Scopus-indexed journals between 2020 and 2023. The findings show that the share of articles funded by these agencies increased from 8.6% in 2020-2021 to 11.9% in 2022-2023. Foreign co-funding as well as international co-authorship and co-affiliations are consistently associated with higher citation impact. In particular, foreign co-affiliations are associated with higher field-normalised citation impact (FNCI) for MESU-funded articles in 2022-2023, exceeding that of articles jointly funded by MESU and foreign agencies. NASU funding is associated with only modest differences in citation impact relative to unfunded articles. These effects are small and not consistently significant across authorship patterns and become less pronounced in 2022-2023, as the citation impact of unfunded articles partially converges with that of funded articles. While the results should be interpreted as average group-level tendencies rather than deterministic effects, they raise important questions about the effectiveness of current funding allocation mechanisms and evaluation criteria, highlighting the need for evidence-based reform of Ukraine's research funding system.

2025-09-26T07:47:59Z Myroslava Hladchenko Rodrigo Costas