- The paper critically examines full vs. fractional counting methods for field-normalized citation impact indicators, advocating for fractional counting.
- Full counting creates a 'full counting bonus' bias, inflating citation impact in highly collaborative fields and violating field normalization principles.
- Empirical evidence confirms this overestimation, leading to the recommendation that fractional counting be used, particularly for macro-level analyses.
Essay on "Field-normalized citation impact indicators and the choice of an appropriate counting method"
The paper by Ludo Waltman and Nees Jan van Eck critically examines the role of counting methods in bibliometric studies where field-normalized citation impact indicators are employed. Central to their analysis is the comparison between full counting and fractional counting as methodologies for handling publications with multiple co-authors. Their robust theoretical and empirical investigations culminate in a strong endorsement of fractional counting when field normalization is requisite.
Full Counting vs. Fractional Counting
Bibliometrics inherently involves the assessment of scientific productivity and impact through publications and citations. When publications involve multiple contributors, determining the most equitable method for credit allocation is essential. Full counting attributes full credit to each contributing entity, regardless of the number of contributors, which can inflate the apparent impact in areas with prevalent co-authorship. Conversely, fractional counting allocates a fraction of the credit proportional to the number of contributors, thus potentially offering a more balanced representation.
Field Normalization and its Theoretical Justification
Field-normalized citation metrics adjust for disparities in citation practices across different disciplines to allow for fairer comparisons. Waltman and van Eck argue that full counting, by assigning full weight to each contributor regardless of shared authorship, violates the principle of field normalization, leading to biases towards fields with high collaboration rates. This bias, referred to as the 'full counting bonus', artificially inflates citation impact indicators such as the Mean Normalized Citation Score (MNCS) for certain fields and countries engaged in notable international collaborations.
Two illustrative examples in the paper show how using full counting leads to anomalies where the perceived average performance exceeds the actual average due to multiple allocations of credit for the same work. Fractional counting, in contrast, ensures both weak and strong field normalization by assigning credits in a proportionate manner. This method upholds the integrity of bibliometric analyses, enabling unbiased inter-field comparisons.
Empirical Evidence and Implications
Empirically, the authors depict the systematic overestimation of citation impact using full counting. Their analysis, encompassing vast datasets primarily from the Web of Science, evaluates variations across disciplines and organizations at varying citation windows. The empirical analysis corroborates the theoretical misgivings about full counting, demonstrating that fields and countries with extensive collaboration exhibit a higher full counting bonus, thus skewing their perceived citation influence.
Practically, the paper advises the use of fractional counting, especially at macro levels such as countries or institutions, where field normalization is pivotal for drawing meaningful insights from citation data. Given the paper's implications, reliance on fractional counting should mitigate the bias introduced by counting methods in bibliometric evaluations, lending greater validity and reliability to such metrics in assessments and policy framings.
Potential Future Research
The exploration of different fractional counting variants—such as author-level and address-level—is briefly touched upon, indicating potential areas for further refinement. Additionally, alternate approaches like multiplicative counting, which aim to combine the benefits of full assignment of credit with proper field normalization, offer intriguing prospects for future bibliometric research.
In conclusion, this paper builds an evidentiary and conceptual scaffold that challenges prevailing tendencies in bibliometrics, steering the discourse towards methodologies that provide equitable and representative insights across diverse scientific landscapes.