Visualizing a Field of Research: A Methodology of Systematic Scientometric Reviews (1906.04800v1)

Abstract: Systematic scientometric reviews, empowered by scientometric and visual analytic techniques, offer opportunities to improve the timeliness, accessibility, and reproducibility of conventional systematic reviews. While increasingly accessible science mapping tools enable end users to visualize the structure and dynamics of a research field, a common bottleneck in the current practice is the construction of a collection of scholarly publications as the input of the subsequent scientometric analysis and visualization. End users often have to face a dilemma in the preparation process: the more they know about a knowledge domain, the easier it is for them to find the relevant data to meet their needs adequately; the little they know, the harder the problem is. What can we do to avoid missing something valuable but beyond our initial description? In this article, we introduce a flexible and generic methodology, cascading citation expansion, to increase the quality of constructing a bibliographic dataset for systematic reviews. Furthermore, the methodology simplifies the conceptualization of globalism and localism in science mapping and unifies them on a consistent and continuous spectrum. We demonstrate an application of the methodology to the research of literature-based discovery and compare five datasets constructed based on three use scenarios, namely a conventional keyword-based search (one dataset), an expansion process starting with a groundbreaking article of the knowledge domain (two datasets), and an expansion process starting with a recently published review article by a prominent expert in the domain (two datasets). The unique coverage of each of the datasets is inspected through network visualization overlays with reference to other datasets in a broad and integrated context.

• The paper introduces cascading citation expansion to automate dataset construction for comprehensive scientometric reviews.
• It employs iterative forward and backward citation mapping to bridge local maps with global research landscapes.
• Visual analytics validate the method’s ability to uncover emerging topics and improve the structural clarity of literature datasets.

An Analysis of "Visualizing a Field of Research: A Methodology of Systematic Scientometric Reviews"

The paper "Visualizing a Field of Research: A Methodology of Systematic Scientometric Reviews" by Chaomei Chen and Min Song introduces an innovative approach to enhance the quality and scope of systematic reviews through scientometric and visual analytic techniques. The central premise of this work is the introduction of cascading citation expansion, a flexible methodology aimed at improving the construction of bibliographic datasets for comprehensive systematic reviews. This approach addresses critical issues related to the timeliness, accessibility, and reproducibility of conventional methods, particularly when domain knowledge is limited.

Key Methodological Contributions

The paper primarily contributes a computational technique that automates the iterative process of dataset expansion. This methodology allows researchers to:

1. Start with Pioneering or Recent Articles: The process can be initiated with a groundbreaking article or a recent review, thereby enabling the collection of related publications both retrospectively and prospectively.
2. Utilize Cascading Citation Expansion: This methodology leverages citation links to expand the dataset incrementally, either through forward or backward citation analysis, which improves coverage and precision.
3. Bridge Global and Local Mapping: The approach offers a transition between local maps (focused domains) and global maps (comprehensive coverage) of scientific knowledge, unifying them on a continuous spectrum.

Dataset Construction and Comparison

The paper demonstrates the efficacy of cascading citation expansion by constructing five different datasets focusing on literature-based discovery (LBD). These include:

• A conventional keyword-based search dataset.
• Three and five-generation forward expansions starting from a notable article by Don Swanson.
• Two datasets from forward and backward expansions starting with a recent review by Neil Smalheiser.

The comparison of these datasets highlights distinct thematic coverage and unique articles captured by the cascading citation expansion method, revealing gaps that conventional keyword-based queries may miss.

Visualization and Analysis

The visual analytic methods employed in the paper provide insightful overlays of the different datasets, showcasing how various search strategies cover the research landscape. For instance, the visual comparison between the keyword-based search and the forward expanded dataset (S5) clearly shows the latter’s superior coverage, particularly in emerging domains like "deep learning" and "big data."

The paper also discusses the role of structural stability and thematic granularity, using modularity and silhouette scores to assess the quality of expanded networks. Networks derived from cascading citation expansions generally exhibit higher structural clarity and homogeneity, making them more reliable for systematic reviews.

Practical and Theoretical Implications

The implications of this research are multifaceted:

1. Practical Utility: By offering a method to systematically capture a broad and representative body of literature, researchers can reduce the risk of overlooking significant emerging topics, which is crucial for constructing thorough and up-to-date systematic reviews.
2. Methodological Improvements: The cascading citation expansion methodology addresses the core challenge of dataset completeness and relevance, enhancing the robustness of bibliometric analyses.
3. Future Research Directions: The paper opens avenues for further exploration in optimizing the iterations of citation expansion, understanding the stability of expanded datasets, and refining the thresholds for forward and backward citations.

Conclusion

The introduction of cascading citation expansion marks a significant methodological advancement in the field of scientometric reviews. It provides a scalable and systematic approach to dataset construction that bridges the gap between localized and globalized science mapping. Researchers in scientometrics and related fields may find this methodology particularly useful for developing comprehensive, high-quality systematic reviews. Future studies could further explore the nuances of this expansion technique and its applicability across various disciplines, potentially enhancing the overall landscape of scholarly communication and bibliometric analysis.