Cited But Not Archived: Analyzing the Status of Code References in Scholarly Articles
Abstract: One in five arXiv articles published in 2021 contained a URI to a Git Hosting Platform (GHP), which demonstrates the growing prevalence of GHP URIs in scholarly publications. However, GHP URIs are vulnerable to the same reference rot that plagues the Web at large. The disappearance of software hosting platforms, like Gitorious and Google Code, and the source code they contain threatens research reproducibility. Archiving the source code and development history available in GHPs enables the long-term reproducibility of research. Software Heritage and Web archives contain archives of GHP URI resources. However, are the GHP URIs referenced by scholarly publications contained within the Software Heritage and Web archive collections? We analyzed a dataset of GHP URIs extracted from scholarly publications to determine (1) is the URI still publicly available on the live Web?, (2) has the URI been archived by Software Heritage?, and (3) has the URI been archived by Web archives? Of all GHP URIs, we found that 93.98% were still publicly available on the live Web, 68.39% had been archived by Software Heritage, and 81.43% had been archived by Web archives.
- Bibliotheca Alexandrina: Internet Archive. https://www.bibalex.org/en/project/details?documentid=283 (2023)
- Di Cosmo, R.: Archiving and referencing source code with Software Heritage. In: Proceedings of the International Congress on Mathematical Software (ICMS). pp. 362–373 (2020). https://doi.org/10.1007/978-3-030-52200-1_36
- Software Heritage: Getting started with the Software Heritage API. https://docs.softwareheritage.org/devel/getting-started/api.html (2023)
- Software Heritage: Mission. https://www.softwareheritage.org/mission/ (2023)
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.