---
title: 'Open-Sci-Ref: Open Scholarly Citation Infrastructure'
url: https://www.emergentmind.com/topics/open-sci-ref
type: topic
---

# Open-Sci-Ref: Open Scholarly Citation Infrastructure

Open-Sci-Ref denotes an open, machine-actionable reference and citation layer for scholarly communication. In the most explicit formulation, OpenCitations characterizes itself as an “Open‑Sci‑Ref” infrastructure that publishes open bibliographic and citation data using Semantic Web technologies and aims “to make global scholarly citation data available at zero cost and without restriction for third party analysis and reuse, in both human‑ and machine‑readable formats” [2206.07476]. Related work extends the same logic to living literature synthesis, executable publication environments, open peer review corpora, claim verification, and local manuscript linting, so the term also functions as a broader label for systems that expose references, citations, reviews, and evidence chains as reusable digital objects rather than opaque strings in static documents [2109.08633].

## 1. Conceptual scope and system boundaries

Open-Sci-Ref is most concrete when understood as an infrastructure for open scientific referencing: citation links, bibliographic metadata, citation contexts, review artifacts, and verification outputs are all represented in machine-readable forms, assigned persistent identifiers, and made reusable under open licenses. Taken together, the literature suggests a layered architecture rather than a single software artifact. At the base is an open reference graph; above it sit tools for acquisition, screening, synthesis, review, verification, and benchmarking.

| Layer | Representative systems | Function |
|---|---|---|
| Reference graph backbone | OpenCitations, OpenCitations Meta, CROCI | Open citation and bibliographic data |
| Living synthesis | Manubot COVID-19 Review Consortium | Continuously updated review manuscripts |
| Executable dissemination | LiveDocs, reference environments | Remotely executable or self-contained reproduction |
| Review and evidence corpora | ORB, AR-Annotator, SciFact-Open | Open review data and claim-evidence links |
| Verification | sciwrite-lint | Local integrity and evidence-chain checking |
| Baselines | open-sci-ref-0.01 | Open reference baselines for model comparison |

This breadth matters because the underlying problem is not only citation access. The recurring target is reusability: references should be structured, separable, open, queryable, and auditable; review comments should be anchored to document structure; literature syntheses should be versioned and reproducible; and claims about cited work should be testable against the cited text itself [1812.01027]. In that sense, Open-Sci-Ref sits at the intersection of open scholarship infrastructure, reproducible publishing, and reference-centered knowledge graph construction.

## 2. Data representation: citations as first-class entities

The technical center of Open-Sci-Ref is the treatment of citations as first-class entities rather than untyped binary edges. OpenCitations models scholarly works as bibliographic entities and each individual citation as an instance of `cito:Citation` with its own identifier, metadata, provenance, and optional characterization such as self-citation type or citation timespan. In the formalization given for the OpenCitations data model, if \(V\) is the set of publications and \(E\) is the set of citation entities, the citation graph is represented as
\[
G = (V, E, s, t)
\]
where \(s(e)\) and \(t(e)\) map a citation entity \(e \in E\) to its citing and cited publications via `cito:hasCitingEntity` and `cito:hasCitedEntity` [2206.07476].

This representation is implemented in RDF and OWL through the SPAR ontologies and related vocabularies. CiTO provides citation semantics; FaBiO provides FRBR-aligned bibliographic classes such as `fabio:Expression` and `fabio:Manifestation`; BiRO represents bibliographic references; C4O represents in-text reference pointers and citation contexts; PRO models roles in time; FOAF models agents; DataCite models identifiers and identifier schemes; PRISM, DCTERMS, OA, and DEO supply additional publication, annotation, and discourse structure. The result is a richly typed RDF graph in which publications, references, annotations, identifiers, and agents are all URI-addressable.

Identifier normalization is critical because the same resource may appear under multiple schemes. Open Citation Identifiers (OCIs) provide persistent identifiers for individual citations, while OpenCitations Meta assigns OpenCitations Meta Identifiers (OMIDs) to bibliographic resources, identifier instances, agent roles, responsible agents, and embodiments. OMIDs have the form
\[
[\text{entity\_type\_abbreviation}]/[\text{supplier\_prefix}][\text{sequential\_number}],
\]
for example `br/0601` for a bibliographic resource. This enables a single internal representation even when the same publication appears as a DOI-bearing Crossref record, a PMID-bearing PubMed record, or a work with no external PID at all [2306.16191].

Scale is a defining property of this layer. As of March 2022, COCI contained “more than \(1.29\) billion citation links” from “more than \(72\) million bibliographic resources” [2206.07476]. OpenCitations Meta, in its initial release, contained 98,243,101 bibliographic resources, 309,881,223 author roles, and 3,749,729,755 triples excluding provenance graphs, with data drawn from Crossref, DataCite, and the NIH Open Citation Collection [2306.16191]. Provenance is tracked snapshot by snapshot through PROV-O and `oco:hasUpdateQuery`, so entity histories and merges are reconstructible rather than merely overwritten.

## 3. Acquisition, screening, living synthesis, and executable publication

Once references are open and structured, Open-Sci-Ref workflows shift from manual bibliographic handling to programmable acquisition and synthesis. A clear example is the BaySREn framework for systematic reviews: it combines an automatic citation search engine and manager, a Bayesian active machine-learning citation screener based on BART and an uncertainty zone, and a semi-automatic query generator. On the example topic reported in the paper, the first session collected 17,755 unique records; only 766 required human evaluation; theoretical efficiency was 95.6% \([95.3\%, 95.7\%]\) with sensitivity 100% \([93.5\%, 100\%]\). A second generated query collected 82,579 additional records; only 567 required human review; overall expected sensitivity became 97.3% \([73.8\%, 100\%]\) while efficiency increased to 98.6% \([98.2\%, 98.7\%]\) [2202.10033].

The Manubot-based COVID-19 Review Consortium demonstrates the same principle at the document level. Its GitHub-centered workflow treats manuscripts as version-controlled source, citations as resolvable identifiers embedded in text, and figures/statistics as products of continuous integration. Contributors cited by DOI, PubMed ID, arXiv ID, Compact URI, and clinical trial identifier; nightly workflows refreshed external data and regenerated figures; and synchronized HTML, PDF, DOCX, and LaTeX outputs were published through GitHub Pages. By September 2021, the consortium involved 52 authors, evaluated at least 1,676 references, opened 285 structured paper issues, and produced seven interlinked literature reviews [2109.08633].

Executable dissemination extends the same reference-centered logic to computational reproducibility. LiveDocs define a LiveDoc as “a remotely executable encapsulation of development environments containing all the source code and datasets needed to reproduce scientific findings with ease,” deployable through BinderHub, Docker, JupyterLite, or static HTML, and explicitly designed for both reproduction and interactive reuse [2402.09475]. Reference environments similarly package “a self-contained set of the minimal software needed to reproduce a published result,” with functionally equivalent outputs as VirtualBox VMs, Docker containers, Amazon EC2 images, and bootable ISO images [1810.03766]. In both cases, references are no longer confined to citation strings; they become executable linkages among code, data, figures, and narrative.

A plausible implication is that Open-Sci-Ref workflows reorganize review and synthesis around stable identifiers, machine-readable metadata, and executable state. This lowers the cost of updating living reviews, rerunning analyses, and reusing search strategies, but it also shifts burden toward environment specification, CI/CD maintenance, and explicit provenance capture.

## 4. Review, claim verification, and integrity assessment

Open-Sci-Ref extends naturally from references to reviews and evidence claims. AR-Annotator addresses transparent peer review by converting articles and their reviews into semantically enriched HTML+RDFa using DEO, schema.org, SWRC, and the Web Annotation Vocabulary. Review comments become explicit annotation entities anchored to sections or text spans rather than free text blocks. In the user study reported in the paper, participants working with fine-grained comments completed the task in an average of 9 minutes, compared with 20 minutes for separate block reviews [1812.01027]. The technical point is not merely interface design; it is that review statements become queryable scholarly objects.

The ORB dataset generalizes this into a reusable review corpus. Its first public release aggregates 36,949 unique submissions, 2,081 submissions with multiple revisions, 27,510 submissions with at least one review, and 92,879 reviews from OpenReview.net and SciPost.org. ORB couples this dataset with `OrbRaw*` interfaces, `Orb*` dataclasses, an ETL pipeline, and an OWL-based `orb.rdf` ontology, thereby turning open reviews into a structured and extensible reference dataset for analysis and modeling [2312.04576].

SciFact-Open moves from review text to claim-evidence verification. It evaluates open-domain scientific claim verification on a corpus of 500,000 abstracts, using 279 claims and 460 evidentiary claim–abstract pairs after pooling and annotation. Systems that performed strongly on the smaller original SciFact corpus dropped by at least 15 F1 in this open-domain setting. The corpus also reveals phenomena central to any Open-Sci-Ref verifier: some claims have many evidentiary abstracts while others have none; 44% of 206 specifically analyzed evidentiary pairs involve a specificity mismatch; and 16 of 81 claims with at least two evidentiary abstracts have conflicting evidence [2210.13777]. An open reference layer therefore cannot be reduced to existence checks alone; it must confront partial support, generalization, contradiction, and evidence heterogeneity.

sciwrite-lint operationalizes this as manuscript linting. It runs entirely on the researcher’s machine with open-weight models, free public databases, and a single consumer GPU; checks whether references exist; verifies retraction status; compares metadata against canonical records; downloads and parses cited papers; verifies whether cited papers support the claims made about them; and follows one level deeper into cited papers’ own bibliographies. In evaluation on 30 unseen papers with injected errors, `dangling-cite` achieved 38/38 detections, `dangling-ref` 29/30, and total recall was 98.5%; 27 of 30 full-pipeline runs completed successfully [2604.08501]. This introduces a distinct but compatible Open-Sci-Ref layer: reference integrity as a local, reproducible, pre-publication verification process.

## 5. Reference corpora, baselines, and large-scale analytics

Open-Sci-Ref also appears in a second, narrower sense as an open reference baseline for model training. “Open-sci-ref-0.01” is not a citation graph; it is a family of dense transformer language-model baselines intended to provide a reference frame for comparing models, procedures, and datasets under matched compute. The suite spans four model sizes—0.13B, 0.40B, 1.31B, and 1.71B parameters—trained on eight open reference datasets at 50B, 300B, and, for the top three datasets, 1T tokens, with compute aligned by
\[
C \approx 6ND.
\]
Across model sizes at 300B tokens, the ranking is stable: Nemotron-CC-HQ is best, DCLM-baseline is second, and FineWeb-Edu is third; at 1.7B/1T, Nemotron-CC-HQ reaches an average score of 0.66 and matches SmolLM2-1.7B despite SmolLM2 using approximately 11 times more tokens and compute [2509.09009]. This is a different usage of the name, but it preserves the same epistemic function: an openly released, reproducible reference point against which later claims can be calibrated.

At the bibliometric end of the spectrum, Alexandria3k provides a local publication-research stack over bulk open metadata. Using the April 2022 Crossref Public Data File, it exposes 134,048,223 works, 1,748,421,617 references, 359,556,891 author records, and a total schema of roughly 2.6 billion records over 45 tables in SQLite. The system supports direct SQL over raw containers or populated local slices, and the paper uses it to visualize post-war publication trends, map field dependencies, analyze COVID-19 research, and reproduce journal impact factor, h5-index, and disruption-style measures [2301.13312]. In Open-Sci-Ref terms, this is the analytical warehouse layer: reproducible publication studies executed over openly archived snapshots rather than live proprietary indexes.

Open reference data also propagate beyond scholarship into innovation analysis. Using 3.2 million in-text non-patent references in 497,098 USPTO patents, mapped to approximately 1 million distinct scientific publications, the patent study defines a Reliance on Science Index and a Normalized Open Access Index. It finds that formal OA publications are overrepresented in patent NPRs, with overall \( \text{NOAI} = 1.38 \), and that when Sci-Hub availability is included the index rises to \( \text{NOAI} = 3.41 \); 85% of NPRs then point to accessible literature [2508.00829]. This suggests that Open-Sci-Ref is not only a scholarly infrastructure question: the openness and machine-actionability of references materially shape downstream technological use.

## 6. Governance, openness, policy, and unresolved issues

Open-Sci-Ref depends on governance as much as on data models. OpenCitations explicitly aligns itself with the UNESCO Recommendation on Open Science, the I4OC recommendations that citation data be structured, separable, and open, and the FAIR principles, especially reusability. All OpenCitations data are published under a CC0 Public Domain Waiver; provenance is machine-readable; and the software stack is open source, so the infrastructure can in principle be recreated elsewhere if needed [2206.07476]. OpenCitations Meta extends this stance by storing bibliographic metadata, provenance snapshots, and change queries in-house, reducing dependence on third-party APIs while keeping the entire graph reusable under CC0 [2306.16191].

Coverage, however, remains uneven. The COCI analysis based on the 2018-11-12 release reported 449,840,503 DOI-to-DOI citation links among 46,534,705 distinct bibliographic entities, but also a persistent gap between open and unavailable references. Elsevier deposited references for 11,020,314 publications and kept them entirely closed; IEEE deposited 3,331,913 publications with closed references, 15,189 with limited references, and none open; ACS likewise showed no open reference deposition in the reported snapshot. CROCI was proposed precisely to address such gaps by allowing scholars, editors, and publishers to upload citation data directly in a four-column CSV or in Scholix, publish the file in Figshare or Zenodo with ORCID provenance, and request ingestion through a GitHub issue [1902.02534]. An open reference graph therefore remains partly a political accomplishment: it depends on publisher behavior, community pressure, and crowdsourced supplementation.

Repository policy creates a parallel tension. The arXiv/REF study showed that, although arXiv held over 1,377,000 e-prints as of April 2018 and is “essential” in several disciplines, it did not meet the formal HEFCE/REF technical requirements because it lacked an acceptance-date field and an explicit AAM/version tag. At the same time, the empirical evidence suggested that arXiv deposits generally satisfied the policy’s substantive aim of timely open access [1804.06648]. Open-Sci-Ref repeatedly encounters this distinction between formal compliance and functional openness: a system may deliver broad access and community utility while still falling short of bureaucratic audit fields.

The literature also converges on a common set of unresolved issues. Living workflows remain hard to sustain at scale; GitHub-centric contribution models can be intimidating; section-level staleness in long documents is difficult to display; JupyterLite and browser execution impose WebAssembly constraints; abstract-only or partially judged corpora leave residual evaluation ambiguity; and compute-aligned model baselines are still limited in parameter scale and multilingual scope [2109.08633]. A plausible implication is that the future of Open-Sci-Ref will depend less on inventing wholly new abstractions than on tightening the interfaces among existing ones: open citation graphs, executable documents, structured reviews, verification tools, and reference baselines need to become routine parts of scholarly communication rather than exceptional overlays.

Source: https://www.emergentmind.com/topics/open-sci-ref