SciImpact: Scientometric Impact Metrics

• SciImpact is an umbrella term for scientometric frameworks that measure research impact through citation analyses, semantic shifts, and predictive models.
• It integrates normalized citation metrics like fractional counting and the Integrated Impact Indicator to correct for field differences and skewed citation distributions.
• It extends to pre-publication and qualitative techniques, leveraging manuscript semantics and knowledge graphs to forecast scholarly influence.

Searching arXiv for papers related to “SciImpact” and scientometric impact indicators. SciImpact can be understood as an umbrella term for scientometric frameworks that measure, normalize, or predict the impact of scientific work. In its strict citation-based sense, research impact means “the contribution of research output to further scientific and technical advancement,” with publications as the basic unit of analysis and citations as the principal certificate of use in subsequent research (Abramo, 2018). In newer formulations, SciImpact also includes semantic and predictive systems in which impact is inferred from the evolution of scientific concepts in knowledge graphs, from manuscript semantics before publication, or from the qualitative function of citations rather than their raw counts alone (Belikov, 19 Feb 2025).

1. Conceptual foundations and scope

In a narrow scientometric sense, impact is scholarly rather than societal. Abramo defines research impact as “the contribution of research output to further scientific and technical advancement,” and explicitly distinguishes this from social, economic, educational, or policy effects, which citation-based scientometrics does not directly measure (Abramo, 2018). This definition makes publication-level use in later research the central object of analysis: no use implies no scholarly impact, and accumulated citations after a sufficiently long citation life-cycle are treated as the benchmark approximation to total impact.

This view is conceptually narrower than the broader policy discourse on “impact.” Bornmann describes a major shift from measuring impact on science to measuring impact on society, but also emphasizes that societal impact is harder to standardize because science has one main internal audience whereas societal impact involves many audiences, including policy makers, industry, clinicians, and the public (Bornmann, 2014). This suggests that SciImpact, if used without qualification, risks conflating scientifically internal influence with broader public value.

A recurrent implication across the literature is that impact measurement is inseparable from assumptions about what counts as evidence of influence. Citation-based systems assume that citations register use in later work; semantic systems assume that impact is expressed in changes in the “semantic fabric of science”; manuscript-level predictors assume that future impact can be inferred from textual and bibliometric priors before citation data exist (Belikov, 19 Feb 2025). These are distinct operationalizations rather than interchangeable definitions.

2. Citation normalization and integrated indicators

A central problem in classical scientometrics is that citation practices vary strongly across fields. In the comparison between SNIP and fractional counting, SNIP is defined as a journal’s Raw Impact per Paper divided by its Relative Database Citation Potential, with the latter based on the mean number of one- to three-year-old cited references per paper in citing journals relative to the database median (Leydesdorff et al., 2010). The critique is methodological: SNIP normalizes at the journal level after aggregation, whereas fractional counting normalizes each citation first. For a citation from paper $p$ with $R_p$ references to paper $q$, the fractional contribution is

$w_{p \to q} = \frac{1}{R_p},$

and a journal’s weighted impact per paper is

$$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$

This ordering makes it possible to obtain citation-weight distributions and test differences between journals statistically (Leydesdorff et al., 2010).

Another influential response to skewed citation distributions is the Integrated Impact Indicator, $I3$, which replaces mean citation rates with percentile-based aggregation: $I3 = \sum_i x_i\,n(x_i).$ Here $x_i$ is the weight assigned to percentile rank $i$ or to a percentile-rank class, and $n(x_i)$ is the number of papers in that class (Wagner et al., 2012). Because citation counts are highly skewed, $R_p$0 uses non-parametric percentiles rather than averages; impact then “adds up” across papers instead of averaging out. The same article-level construction can be re-aggregated for authors, groups, journals, or institutions.

Scilit’s recent implementation extends this logic at journal level through $R_p$1 and a size-normalized version $R_p$2. In that framework,

$R_p$3

with four percentile classes weighted $R_p$4, $R_p$5, $R_p$6, and $R_p$7 for the top $R_p$8, top $R_p$9 excluding top $q$0, top $q$1 excluding top $q$2, and bottom $q$3, respectively. The normalized version is

$q$4

where $q$5 is the number of publications of journal $q$6 in the window (Dong et al., 5 Jan 2026). Scilit also uses citing-side fractional counting,

$q$7

to reduce field differences in citation density before percentile assignment (Dong et al., 5 Jan 2026).

At article level, Ferrara and Romero propose the discounted h-index,

$q$8

where $q$9 is total citations and $w_{p \to q} = \frac{1}{R_p},$0 total self-citations (Ferrara et al., 2012). The construction is designed to discount self-citation bias without requiring the full distribution of self-citations over papers. A different article-level proposal, the Individual Impact Index, defines

$w_{p \to q} = \frac{1}{R_p},$1

with $w_{p \to q} = \frac{1}{R_p},$2 combining the impact factor of the original journal and citations weighted by the impact factors of the citing journals, and $w_{p \to q} = \frac{1}{R_p},$3 depending on the size $w_{p \to q} = \frac{1}{R_p},$4 of the journal’s JCR category (Balayla, 2017). These models share a common aim: preserving article-level specificity while correcting citation counts for context.

3. Fair comparison, universality, and quality versus quantity

A major theme in SciImpact research is that the same raw metric may reflect different things in different fields or at different career stages. Radicchi, Fortunato, and Castellano formalize universality as the requirement that if one selects the top $w_{p \to q} = \frac{1}{R_p},$5 of scholars globally by a metric, then each discipline should contribute approximately its top $w_{p \to q} = \frac{1}{R_p},$6 (Kaur et al., 2013). They propose a simple normalized metric,

$w_{p \to q} = \frac{1}{R_p},$7

where $w_{p \to q} = \frac{1}{R_p},$8 is the average h-index in discipline $w_{p \to q} = \frac{1}{R_p},$9, and show that $$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$0 behaves as a universal metric under their distributional test (Kaur et al., 2013).

A more individualized approach is the cloning-based framework for separating quality from quantity. For a researcher with publication profile $$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$1, clones preserve the number of papers, publication years, and subject categories, while resampling citation counts from papers in the same year and category. The observed metric $$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$2 is then compared with the clone distribution $$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$3 through a quantile score

$$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$4

and a z-score

$$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$5

This makes it possible to identify high-quality careers even when raw h-index is modest; Nobel laureates are used as validation cases, and the method is applied to 996,288 author records and 6,129 journals (Kaur et al., 2014).

Another strand treats impact as an author-specific threshold-crossing problem. In “Will This Paper Increase Your h-index?”, the prediction target is whether a paper will obtain enough citations within five years to reach the current h-index of its primary author (Dong et al., 2014). The paper reports “greater than 87.5% potential predictability” and finds that “the researcher’s authority on the publication topic and the venue in which the paper is published are crucial factors,” whereas “the topic popularity and the co-authors’ h-indices are of surprisingly little relevance” (Dong et al., 2014). This shifts SciImpact from retrospective scoring to personalized impact forecasting.

4. Semantic and pre-publication predictive systems

A recent redefinition of SciImpact treats impact as semantic change rather than social recognition. The XSI framework constructs a biomedical knowledge graph from 324K publications from bioRxiv, medRxiv, and arXiv quantitative biology submissions, using TriEL for relation extraction and entity linking with BERN2 and entity-fishing (Belikov, 19 Feb 2025). For a publication $$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$6, raw Semantic Impact is defined as

$$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$7

and the final score is

$$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$8

A paper has high semantic impact when the edges in its semantic subgraph become more heavily used in later work. Using only semantic graph features at publication time, histogram-based gradient boosting regression reaches $$\text{Weighted Impact}_J = \frac{\sum_{(p \to q) \in C(J)} \frac{1}{R_p}}{N_J}.$$9 at a 36-month horizon, and the metric shows positive but modest correlations with citation counts, typically in the range $I3$0 (Belikov, 19 Feb 2025).

The same framework introduces a portfolio-optimization layer. With predicted semantic impact $I3$1, predicted relative prediction error $I3$2, selection variables $I3$3, budget $I3$4, and risk-aversion parameter $I3$5, portfolio choice is formulated as

$I3$6

subject to

$I3$7

This is presented as a 0–1 knapsack-like integer linear program solved with Google OR-Tools CP-SAT (Belikov, 19 Feb 2025).

A different pre-publication direction is represented by IMAC, a manuscript assessment classifier trained on 69,707 articles from 99 journals spanning multiple disciplines and publication years 2015–2019 (Sun et al., 26 Mar 2025). IMAC predicts two binary targets from information available before publication: whether the hosting journal has $I3$8, and whether the article reaches a composite Article Impact Factor threshold $I3$9. The model combines SciBERT embeddings of title and abstract, an attention-based text fusion layer with AFF and MS-CAM, metadata embedding, and a joint cross-entropy plus supervised contrastive loss. Reported performance reaches accuracy $I3 = \sum_i x_i\,n(x_i).$0 and F1 $I3 = \sum_i x_i\,n(x_i).$1 for journal-impact classification, and accuracy $I3 = \sum_i x_i\,n(x_i).$2 and F1 $I3 = \sum_i x_i\,n(x_i).$3 for article-impact classification (Sun et al., 26 Mar 2025). This suggests a version of SciImpact in which semantic features, reference features, and author statistics act as early proxies for later citation-based impact.

5. Qualitative citation analysis and journal-level semantic analytics

SciImpact is not limited to counts or predictions of counts. ImpactCite approaches citation impact analysis at the level of citation function, classifying each citation instance by sentiment and intent (Mercier et al., 2020). Citation intent is modeled with the SciCite categories Result, Method, and Background; citation sentiment is modeled as positive, negative, or neutral. The proposed XLNet-based system, ImpactCite, is trained separately for the two tasks and is reported to improve state of the art by $I3 = \sum_i x_i\,n(x_i).$4 in F1-score for citation intent classification and by $I3 = \sum_i x_i\,n(x_i).$5 in F1-score for citation sentiment classification (Mercier et al., 2020). A plausible implication is that SciImpact can treat not all citations as equivalent: a positive Method citation and a neutral Background citation need not contribute equally to impact, even if both count as one citation in conventional bibliometrics.

BioMedJImpact extends journal-level scientometrics by integrating bibliometric variables, collaboration features, and LLM-derived semantic indicators of AI engagement across 1,740,112 PMC articles in 2,744 journals from 2016 to 2023 (Wang et al., 16 Nov 2025). AI engagement is derived for all abstracts by a three-stage pipeline built on Gemma-3-12B-IT served through vLLM, and the corpus-wide AI engagement rate is $I3 = \sum_i x_i\,n(x_i).$6. At journal-year level, AI engagement is defined as

$I3 = \sum_i x_i\,n(x_i).$7

and mixed-effects models are used to relate lagged bibliometric, collaboration, and AI-engagement features to Impact Factor, Quartile, and Total Cites (3Y) (Wang et al., 16 Nov 2025). The reported empirical pattern is that journals with higher collaboration intensity, especially larger and more diverse author teams, tend to achieve greater citation impact, and AI engagement has become an increasingly strong correlate of journal prestige, especially in quartile rankings (Wang et al., 16 Nov 2025).

These developments move SciImpact toward hybrid systems in which textual content, structural properties of collaboration, and semantic traces of methodological adoption become explicit covariates in impact analysis. The resulting models remain journal-level or citation-level, but they no longer assume that raw citation volume is the only usable observational signal.

6. Limitations, controversies, and evaluative implications

The literature repeatedly emphasizes that impact indicators are theory-laden and statistically fragile. The comparison between SNIP and fractional counting argues that SNIP, as a ratio of aggregated quantities involving means and medians over skewed distributions, makes significance testing impractical, whereas citation-level fractional counting yields distributions on which Kruskal–Wallis and Bonferroni-corrected post-hoc tests can be applied (Leydesdorff et al., 2010). This is not merely a technical complaint; it concerns whether ranking differences can be interpreted as statistically meaningful.

Bornmann’s broader critique is that impact measurement is distorted by “inequality, random chance, anomalies, the right to make mistakes, unpredictability, and a high significance of extreme events” (Bornmann, 2014). In highly skewed systems, a single anomalous paper can dominate institutional averages; long-latency “Sleeping Beauties” can be undervalued by short windows; and metric-centered evaluation can induce “mimicry in science,” salami slicing, or strategic journal targeting. Abramo adds a conceptual warning: journal metrics and altmetrics may be useful as very early predictors, but they should not be used as direct proxies for article or researcher impact, and scientometrics should not claim to measure social impact when it is measuring only scholarly impact (Abramo, 2018).

The new semantic and pre-publication systems carry their own limitations. XSI is currently restricted to abstract-only processing of biomedical preprints, is affected by entity-linking errors and first-mover bias, and measures semantic propagation rather than epistemic correctness (Belikov, 19 Feb 2025). IMAC relies on a dataset from 99 Scopus journals, uses binary thresholds such as $I3 = \sum_i x_i\,n(x_i).$8 and $I3 = \sum_i x_i\,n(x_i).$9, and may invite optimization to the model rather than to scientific substance (Sun et al., 26 Mar 2025). BioMedJImpact depends on PMC-OA coverage, JCR matching, and LLM-derived AI labels that show substantial but not perfect human agreement (Wang et al., 16 Nov 2025).

Taken together, these strands suggest that SciImpact is best treated not as a single metric but as a family of measurement regimes. Citation counts remain the canonical benchmark for scholarly impact; percentile-based and field-normalized indicators correct skewness and disciplinary bias; clone baselines decouple quality from quantity; qualitative citation models distinguish kinds of influence; and semantic knowledge-graph systems estimate conceptual propagation before citations mature. A plausible conclusion is that robust SciImpact requires plural instrumentation rather than metric monism.