Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations' COCI: a multidisciplinary comparison of coverage via citations (2004.14329v3)

Abstract: New sources of citation data have recently become available, such as Microsoft Academic, Dimensions, and the OpenCitations Index of CrossRef open DOI-to-DOI citations (COCI). Although these have been compared to the Web of Science (WoS), Scopus, or Google Scholar, there is no systematic evidence of their differences across subject categories. In response, this paper investigates 3,073,351 citations found by these six data sources to 2,515 English-language highly-cited documents published in 2006 from 252 subject categories, expanding and updating the largest previous study. Google Scholar found 88% of all citations, many of which were not found by the other sources, and nearly all citations found by the remaining sources (89%-94%). A similar pattern held within most subject categories. Microsoft Academic is the second largest overall (60% of all citations), including 82% of Scopus citations and 86% of Web of Science citations. In most categories, Microsoft Academic found more citations than Scopus and WoS (182 and 223 subject categories, respectively), but had coverage gaps in some areas, such as Physics and some Humanities categories. After Scopus, Dimensions is fourth largest (54% of all citations), including 84% of Scopus citations and 88% of WoS citations. It found more citations than Scopus in 36 categories, more than WoS in 185, and displays some coverage gaps, especially in the Humanities. Following WoS, COCI is the smallest, with 28% of all citations. Google Scholar is still the most comprehensive source. In many subject categories Microsoft Academic and Dimensions are good alternatives to Scopus and WoS in terms of coverage.

Overview and Implications of Citation Index Coverage Study

This paper, authored by Martín-Martín et al., presents a comprehensive comparison of multiple citation data sources, including Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science (WoS), and OpenCitations’ COCI. The paper involves an extensive analysis of approximately 3,073,351 citations across 2,515 highly-cited documents from 252 subject categories, originated in 2006, thereby expanding the scope of previous coverage studies.

Methodology

The researchers employed a methodical approach due to access restrictions on full bibliographic databases. They focused on a seed sample from Google Scholar’s Classic Papers, ensuring these documents were highly cited across various disciplines. They matched citations across the six data sources, taking care to avoid false positives by using conservative matching criteria.

Key Results

• Google Scholar’s Dominance: Google Scholar displayed the most extensive coverage, capturing 88% of all possible citations from the seed documents, and found a significant proportion of citations detected by the other sources (89%-94%).
• Microsoft Academic and Scopus: Microsoft Academic followed Google Scholar, with 60% coverage, finding more citations than Scopus and WoS in numerous categories, particularly excelling in Humanities and Social Sciences albeit with gaps in Physics.
• Dimensions and COCI: Dimensions provided similar coverage to Scopus and exceeded WoS in several areas, though it, too, had gaps in some Humanities categories. COCI remained the least comprehensive, encompassing only 28% of the total citations.

Implications

Practical Applications

• Citation Analysis: Google Scholar remains a leading choice for comprehensive citation analysis, beneficial for obtaining broad citation counts, albeit without providing exhaustive lists of citing sources. Microsoft Academic serves as a robust alternative for more detailed citation studies and is an open-access option.
• Research and Accessibility: The general openness of data, as seen in Microsoft Academic and COCI, reflects a shift towards more accessible research infrastructures, which could inspire broader data sharing practices and reduce dependency on proprietary databases.

Theoretical Considerations

• Coverage Divergence: This paper underscores existing divergences in database coverage, which vary significantly across disciplines. This variability in data source comprehensiveness should inform bibliometric discussions and the selection of databases based on specific research needs.
• Future Directions: New forms of citation indexing, particularly those incorporating open access data like COCI, might catalyze advancements in bibliometric methodologies. The potential integration of citation links from free and open sources could pave the way for more equitable academic research ecosystems.

Conclusion

The research confirms Google Scholar's robust coverage across disciplines, highlighting its persistent place at the forefront of citation databases, although challenges such as fluctuating coverage remain. Microsoft Academic and Dimensions present viable alternatives to traditional resources like Scopus and WoS, with notable diversity in their subject-specific capabilities. Overall, this comparative paper provides critical insights for researchers into the nuanced landscape of citation metrics, encouraging informed choices for database selection tailored to specific scholarly inquiries.

Future bibliometric analyses would benefit from continued exploration into the evolving landscapes of these databases, particularly as more data sources commit to openness and interoperability, influencing both theoretical foundations and practical applications within computational and bibliometric research spheres.