The h-index is no longer an effective correlate of scientific reputation (2102.03234v1)

Abstract: The impact of individual scientists is commonly quantified using citation-based measures. The most common such measure is the h-index. A scientist's h-index affects hiring, promotion, and funding decisions, and thus shapes the progress of science. Here we report a large-scale study of scientometric measures, analyzing millions of articles and hundreds of millions of citations across four scientific fields and two data platforms. We find that the correlation of the h-index with awards that indicate recognition by the scientific community has substantially declined. These trends are associated with changing authorship patterns. We show that these declines can be mitigated by fractional allocation of citations among authors, which has been discussed in the literature but not implemented at scale. We find that a fractional analogue of the h-index outperforms other measures as a correlate and predictor of scientific awards. Our results suggest that the use of the h-index in ranking scientists should be reconsidered, and that fractional allocation measures such as h-frac provide more robust alternatives. An interactive visualization of our work can be found at https://h-frac.org

• The paper finds a significant decline in the correlation between the h-index and scientific awards, particularly in fields like physics, attributing this decline to the rise of hyperauthorship.
• It proposes using fractional allocation of citations to calculate h-frac, a fractional analogue of the h-index that shows superior correlation with scientific awards across diverse fields.
• Fractional measures like h-frac are more robust and predictive of future recognition than the traditional h-index, suggesting a need to re-evaluate current scientist evaluation processes.

The h-index and Scientific Reputation: An Analysis of Efficacy and Alternatives

The paper "The h-index is no longer an effective correlate of scientific reputation" by Vladlen Koltun and David Hafner presents a critical evaluation of the h-index, a widely used citation-based metric for assessing the impact of a scientist's work. By leveraging a large-scale dataset comprising millions of articles and hundreds of millions of citations across various scientific fields, the authors provide a comprehensive analysis of the correlation between the h-index and scientific awards. Their findings underscore a significant decline in the efficacy of the h-index as an indicator of scientific reputation, which they attribute to evolving authorship patterns, particularly the rise of hyperauthorship.

Key Findings

1. Decline in the h-index Correlation: The paper observes a substantial decrease in the correlation between the h-index and scientific awards, especially in fields like physics where the correlation has dropped to negligible levels. This trend is closely associated with a shift towards larger collaborative authorship, often termed hyperauthorship, where publications often list thousands of authors.
2. Emergence of Fractional Allocation: To address the limitations of the h-index, the authors propose the use of fractional allocation of citations among authors. This approach, which has been discussed but not widely implemented, redistributes credit proportionally based on the number of coauthors. The paper identifies a fractional analogue of the h-index, termed h-frac, which demonstrates superior correlation with scientific awards compared to traditional metrics.
3. Effectiveness Across Fields: The paper conducts parallel analyses in biology, computer science, economics, and physics. It finds that the performance of fractional measures like h-frac is consistent and superior across these diverse fields and across different data platforms (Google Scholar and Scopus).
4. Robustness and Predictive Power: Fractional measures not only correlate well with scientific awards but also exhibit stability over time and better predictive power regarding future recognition compared to non-fractional indices. The findings highlight that these measures retain their reliability despite shifts in publication and citation patterns.

Implications and Future Considerations

The implications of this research are significant for academia and policy makers involved in scientist evaluation processes. The h-index has long been a staple in hiring, promotion, and funding decisions, but the presented evidence suggests a need for reevaluation of its usage. The adoption of fractional measures like h-frac could lead to more accurate assessments of individual contributions, discourage unwarranted authorship practices, and ultimately support a more equitable allocation of scientific credit and resources.

Moving forward, the paper opens avenues for further exploration into citation metrics that account for variables such as self-citations and author order, aspects not fully addressed within this work. Furthermore, the integration of these refined metrics in existing scientometric tools could enhance empirical analyses of scientific impact across various disciplines, fostering a more nuanced understanding of scientific contributions.

In conclusion, while the h-index has served as a straightforward measure of scientific output, its declining correlation with reputable scientific achievements necessitates reconsideration. The proposed h-frac index offers a promising alternative, aligning more closely with scientifically esteemed recognition and robustly adapting to contemporary collaborative research practices.