Data Referencing Errors (DREs)
- Data Referencing Errors (DREs) are failures in correctly linking, citing, or retrieving data, compromising referential integrity across scholarly and digital systems.
- They manifest as incomplete metadata, misaligned table citations, and hard-coded spreadsheet formulas, adversely affecting reproducibility and verification.
- Mitigation strategies include DOI-centric lookups, standardized citation styles, and critic-based model filtering to enhance reference accuracy and operational reliability.
Data Referencing Errors (DREs) denote failures in the way systems identify, copy, link, retrieve, or update referenced data. In the tabular-LLM literature, a DRE is an incorrect or omitted citation of table values despite apparent understanding of table structure (Yang et al., 30 Jun 2026). In bibliographic studies, closely related phenomena appear as “errors in citing and referencing” or “errors in bibliographic citations”: incorrect, incomplete, or ambiguous descriptive elements; misalignment between in-text reference pointers and reference-list entries; and mis-formatting or mis-structuring that impedes identification and retrieval (Santos et al., 2020). Large scientific databases expose analogous failures through manual transcription, inconsistent multi-format outputs, broken associations between data and sources, ambiguities, duplications, and outdated metadata (Skinner et al., 2020). Spreadsheet engineering treats hard coding of input data values into formulas as a design defect that breaks proper data referencing and creates latent future errors (0803.0169). LLM-assisted literature retrieval exhibits a related form through erroneous or fabricated references, incorrect DOI or PubMed ID fields, invalid Google Scholar links, and “complete miss” references failing all applicable validation metrics (Gao et al., 21 Mar 2026). Taken together, these works suggest that DREs are not a narrow anomaly but a recurring failure mode wherever referenced data must remain uniquely identifiable, structurally linked, and operationally retrievable.
1. Conceptual foundations
A bibliographic reference has been defined as a facet of descriptive representation whose purpose is to encode a “minimal set of descriptive bibliographic metadata which enables the identification of a publication, a speech, a piece of information, or anything else that may be citable, to locate and retrieve it” (Santos et al., 2020). From that perspective, a DRE is any failure of that encoding that undermines identification, location, or retrieval. The cross-disciplinary study of citing practices extends the same logic to bibliographic references, mentions, quotations, and in-text reference pointers treated as a unified system; errors occur not only in each component but also in the relations between them, especially when they fail to provide “easy access” to the cited work or passage (Santos et al., 2022).
The FRBR framework sharpens this view by distinguishing Work, Expression, Manifestation, and Item. Mentions and quotations primarily refer to the Expression of a Work; in-text reference pointers with page numbers are manifestation-oriented; bibliographic references are generally formulated at the Manifestation level and sometimes approach the Item level (Santos et al., 2020). A DRE in this setting can therefore be a misalignment between the FRBR levels implied by the in-text citation and those described in the reference list, or a failure to supply enough data to identify the relevant Work, Expression, or Manifestation.
Outside bibliographic scholarship, the same structural issue appears in other data systems. Spreadsheet modeling frames correct design through separation of inputs, processing, and outputs; typing a parameter directly into a formula instead of referencing the authoritative input cell merges data and logic, violating that separation (0803.0169). Data accounting generalizes the idea further: a data space is formalized as , and complete summarization requires that data not silently disappear under transformation but remain accounted for through partitions and error traces (Gajda, 2023). This suggests a broad interpretation of DREs as failures of referential integrity across descriptive, computational, and analytical representations.
2. Principal forms of DREs
Across the literature, DREs recur in a small number of structurally stable forms, even though the surface manifestations differ by domain.
| Domain | Representative DREs | Source |
|---|---|---|
| Scholarly references | Incorrect or incomplete metadata; pointer-reference mismatches; ambiguous journal abbreviations; missing page numbers | (Santos et al., 2020, Santos et al., 2022) |
| Scientific databases | Manual transcription mistakes; wrong source-to-data associations; format divergence; outdated metadata | (Skinner et al., 2020) |
| Tables and LLM reasoning | Incorrect citation of a cell; row mix-up; column mix-up; omitted row in a purportedly complete list | (Yang et al., 30 Jun 2026) |
| Medical literature retrieval | Wrong DOI, PubMed ID, or Google Scholar link; irrelevant reference; complete miss | (Gao et al., 21 Mar 2026) |
| Spreadsheets | Hard-coded constants replacing cell references | (0803.0169) |
In bibliographic corpora, the observed forms include incorrect, incomplete, or ambiguous descriptive elements in reference-list entries; misalignment or ambiguity between in-text pointers and reference-list entries; reference-list ordering errors in numeric and author-date systems; ambiguous same-author same-year citations without disambiguators such as 2000a/2000b; URLs pointing to bibliographic database records rather than the cited work; omission of DOI URLs for freely available works; under-specified journal title abbreviations; missing source attribution for non-textual content; and inadequate or missing pagination for quotations, particularly for HTML or other non-paginated formats (Santos et al., 2020). The broader all-discipline analysis adds orphan references, mentions or quotations without corresponding reference-list entries, incorrect alphabetical or numerical arrangement, mixed citation systems inside one article, and styles that do not specify how to cite born-digital resources such as datasets (Santos et al., 2022).
In tabular reasoning, the literature isolates two main types. “Incorrect Citation” is value-level: the model cites a wrong cell value, confuses rows or columns, or fabricates table-based content not present in the table. “Omitted Information” is subset-level: the model attempts to enumerate all relevant rows or items but leaves some out (Yang et al., 30 Jun 2026). Synthetic error construction further operationalizes row mix-ups, column mix-ups, row removal from the table, and removal of a listed row from the model response.
In database administration, DREs arise at metadata entry and synchronization stages. The manual HITRAN and AMBDAS workflow required transcribing reference metadata, cutting and pasting from websites and PDF files, correlating bibliography information to the main article or resource, adding DOI information, checking spelling, and maintaining separate plain text, , and HTML markups (Skinner et al., 2020). Each step created opportunities for metadata inaccuracy, incompleteness, format inconsistency, association errors, ambiguity, duplication, and stale records.
In spreadsheet systems, hard coding is a distinct DRE pattern because the formula ceases to reference the authoritative data cell at all. A formula such as =NetPrice * 0.075 embeds the VAT rate in logic rather than referencing a dedicated VAT-rate cell; when the assumption changes, the spreadsheet contains stale and inconsistent references by design (0803.0169).
Medical literature retrieval introduces an identifier-centric formulation. The error classes are operationalized through DOI validity, PubMed ID validity, Google Scholar link validity, and relevance to the index article. A “complete miss” occurs when the retrieved reference fails all applicable metrics (Gao et al., 21 Mar 2026).
3. Empirical prevalence and measurement
The empirical record shows that DREs are persistent, frequent, and distributed across domains rather than confined to isolated workflows.
In a 2019 sample of Medicine and Social Sciences journals, 8.66% of Medicine articles using citation-sequence systems had bibliographic references whose numeric ordering did not correspond to the order in which they were first cited; in author-date systems, misuse of alphabetical ordering appeared in 40% of Medicine articles adopting alphabetical arrangement and 3% of Social Sciences articles (Santos et al., 2020). Among references to freely available works, 50.1% of Medicine references and 61% of Social Sciences references lacked a DOI URL. Medicine also showed a substantial access-metadata problem: 42% of the non-DOI URLs in bibliographic references often referred to records within a bibliographic database rather than directly to the cited work. Non-textual content was pervasive, yet 84.8% of Medicine articles containing such content did not provide a source for it.
The broader survey across 729 articles in 147 journals and 27 subject areas found that bibliographic errors had been perpetuated for decades and that their possible causes had increased despite reference managers and digital tools (Santos et al., 2022). On average, 10.48% of articles using a citation-sequence system had incorrect numerical ordering, and 7.93% of articles using an author-date system had incorrect alphabetical ordering. Among articles containing quotations, about 70% did not provide “easy access” to the cited content. Reference-style quality was also poor: 62.49% of reference styles analyzed were classified as not providing clear guidelines.
The tabular-LLM study provides the most explicit DRE metrics. It defines
with companion measures for the overlap between DREs and final-answer correctness (Yang et al., 30 Jun 2026). DREs occurred across all tested models from 1.7B to 20B parameters. On WikiTableQuestions, Qwen3-1.7B had a DRE Rate of 35.52%, Qwen3-8B 14.04%, Distill-Llama-8B 37.96%, Llama4-Scout-17B 46.48%, and gpt-oss-20b 5.71%. Across tasks, Qwen3-8B showed DRE rates of 10.55% on TableBench, 33.57% on FinQA, 14.06% on SciTab, and 18.45% on ToTTo. The study also reports that a lightweight 4B critic model achieved an average F1 score of 78.16% in DRE detection across model-dataset pairs, and that critic-based methods improved answer accuracy by up to 12.0%.
In LLM-assisted retrieval of medical literature, the multimetric score ratio is computed by dividing an adjusted total score over DOI, PubMed ID, Google Scholar link, and relevance by a score cap reflecting the applicable metrics (Gao et al., 21 Mar 2026). Across 2,000 retrieved references from five platforms, the average score ratio was 0.29, the range was 0–1.25, and the complete miss rate was 47.8%. Grok achieved the highest score ratio at 0.57 and the lowest complete miss rate at 11.2%, whereas Gemini had the lowest score ratio at 0.11 and a complete miss rate of 78.5%.
Database and spreadsheet studies report prevalence differently but still document nontrivial scale. In HITRAN and AMBDAS, about fourteen hundred references were corrected with up-to-date information and tested and double checked before updating the database (Skinner et al., 2020). In one student spreadsheet workbook, 481 formulas were examined, 88 instances of hard coding were identified, and 50 numeric values were entered directly into cells; a practitioner model contained over 50,000 formulas and repeated unexplained constants such as 236, 259, and 279 (0803.0169).
4. Mechanisms that generate DREs
The literature identifies DREs as products of both local slips and systemic conditions. Sweetland’s classic causes remain central: lack of standardization in citation formats, diffusion of responsibility in the publishing process, lack of training in citation norms, and failure to examine the cited document (Santos et al., 2020). The later all-discipline survey concludes that these causes still hold and that additional causes have emerged, especially proliferation and customization of styles, unclear or broken journal instructions, negligence in editorial and peer-review processes, and uncritical dependence on reference managers and export tools (Santos et al., 2022).
Style proliferation is a recurrent generator of DREs. In 46 journals across Medicine and Social Sciences, 20 different reference styles were observed, and 33% of Medicine journals and 23.5% of Social Sciences journals provided styles that were “not clear, comprehensive, and exhaustive” for accurate metadata description (Santos et al., 2020). Across all disciplines, over half of journals did not provide clear referencing guidelines (Santos et al., 2022). Where style guides omit instructions for secondary citations, DOI formatting, non-textual content, born-digital resources, or the mapping between in-text pointers and reference-list entries, authors improvise, and those improvisations become sources of omission, ambiguity, and non-standard structure.
Manual workflow amplifies this problem. Before automation in HITRAN and AMBDAS, administrators processed references “painstakingly by hand,” including transcription, copy-paste from websites and PDFs, DOI entry, spelling checks, and upkeep of multiple markup formats (Skinner et al., 2020). Each repeated re-entry of the same record created opportunities for divergence across plain text, HTML, and BibTeX representations. The paper explicitly characterizes the prior workflow as “tedious, time-consuming and error-prone.”
Automation does not remove DREs if the model of reference resolution is weak. In tabular LLMs, self-reflection and extended reasoning do not prevent early mis-citations from being repeated later in the chain of thought (Yang et al., 30 Jun 2026). A direct prompting intervention—“Use only the table. Do not omit, miscite, or fabricate information. Ensure all cited values exactly match the table.”—reduced Qwen3-8B’s DRE rate only slightly, from 14.04% to 12.50%, and did not materially change final-answer accuracy, from 77.14 to 77.51. In medical retrieval, even an explicit follow-up prompt asking the platform to confirm DOI, PubMed ID, and Google Scholar link did not eliminate fabricated or inaccurate bibliographic data (Gao et al., 21 Mar 2026).
Spreadsheet hard coding reveals another mechanism: the replacement of explicit data references with literals. Blayney treats this as a qualitative design defect because it may not immediately change bottom-line values, yet it creates the conditions under which later assumption changes produce quantitative errors (0803.0169). A formula with embedded 200, 0.075, or /12 is not merely stylistically inelegant; it is a frozen reference that fails as soon as the authoritative input changes.
5. Detection, validation, and mitigation
The principal mitigation strategy in bibliographic systems is to reduce free-form entry and increase identifier-centered structure. The HITRAN/AMBDAS system uses the DOI as the canonical key: the user enters only the DOI, the system retrieves bibliographic information from ADS or DOI-based services, parses it into a relational database, and produces HTML, JSON, and BibTeX outputs programmatically (Skinner et al., 2020). The DOI-centric mapping can be written as
The architecture adds global integer IDs, cross-references per-molecule IDs, supports notes and nested references, and uses fallbacks when ADS lacks coverage. The design goal is explicit minimization of “errors, ambiguities and duplications.”
Bibliographic scholarship recommends parallel interventions: simplification and clarification of reference styles, explicit rules for linking in-text pointers to reference-list entries, standardized representation of DOIs and URLs, DOI URL inclusion whenever available, direct linking to the full text or landing page for the exact Manifestation rather than to catalog records, inclusion of disambiguating metadata such as ISSN, and training for authors and editorial staff (Santos et al., 2020). The cross-disciplinary study complements these recommendations by calling for fewer proprietary styles, clear style names and editions, and better coordination between journals and reference-manager vendors (Santos et al., 2022).
For table-grounded LLMs, the field has converged on process-level criticism. The judge framework asks whether a response segment fails a “Copied Values Consistency Check” or an “Omission Check,” and Claude 3.7 Sonnet with reference answers achieved 92.67% accuracy in a manual evaluation of 100 random judge outputs (Yang et al., 30 Jun 2026). Critic-based filtering ranks multiple samples by the number or severity of DREs, while segment-level rejection sampling regenerates only those chain-of-thought segments judged to contain DREs. With Sonnet-3.7+gt as critic, rejection sampling improved full-set accuracy, for example from 77.14 to 78.94 for Qwen3-8B on WTQ and from 54.77 to 66.73 for Distill-Qwen-7B on TableBench. A smaller Critic-4B model, trained by supervised fine-tuning and RLVR, further reduced DRE rates while improving accuracy.
The medical retrieval study adopts a different validation strategy: per-reference scoring of DOI validity, PubMed ID validity, Google Scholar link validity, and relevance, plus a binary complete-miss flag (Gao et al., 21 Mar 2026). This makes DREs measurable even when a reference is partially correct. The approach directly supports cross-platform comparison and multivariable regression on platform and journal effects.
Spreadsheet detection relies on structural auditing. The Automated Hard Coding Identification Method scans each formula string character by character; if an operator is followed by a numeric character, the formula is flagged as containing a potential hard-coded constant (0803.0169). The method intentionally favors recall over precision, flagging even benign uses of 0, 1, or 12 so that a human reviewer can distinguish genuine parameters from acceptable constants.
Data accounting offers a more general formal mitigation model. It replaces destructive selection with partition, plain union with tagged union, and inner joins with outer-join-like left/inner/right partitions; error-trace relations ensure that records are not silently dropped (Gajda, 2023). This architecture supports transparent assertions about the impact of individual records and enables localization of DRE-like failures such as missing joins, misspellings, outdated lookup keys, and other referential mismatches.
6. Consequences and open directions
DREs affect more than formatting. In bibliographic communication, they distort citation indexing and metrics, impede linking to digital objects, reduce discoverability and retrieval, frustrate verification and reproducibility, and complicate automated parsing and mining (Santos et al., 2020). The all-discipline study adds that they violate the “easy access” function of citation and make current citing practices poorly suited to a global, multidisciplinary readership (Santos et al., 2022). Where quotations lack page numbers or references lack minimally identifying metadata, readers must perform complementary searches or inspect long excerpts to locate the cited passage.
In scientific databases, DREs directly affect accreditation and traceability. HITRAN emphasizes that contributors are accredited through the bibliography produced alongside returned data and that every data set returned by a search is accompanied by a bibliography (Skinner et al., 2020). Wrong references, missing authors, or broken links therefore undermine both credit assignment and the ability of users to examine how parameters were determined.
In LLM systems, DREs undermine the correctness and reliability of intermediate reasoning steps even when the final answer remains correct (Yang et al., 30 Jun 2026). This matters especially in dense tables and verification settings, where a chain of thought may misquote numbers yet still arrive at a correct label. The medical retrieval study reaches an analogous conclusion for bibliographic assistants: LLMs should be treated as assistive, not authoritative, because bibliographic data often remain plausible-looking but wrong (Gao et al., 21 Mar 2026).
Several future directions recur. Bibliographic researchers call for broader disciplinary coverage, longitudinal studies, and deeper integration with computational methods capable of automatically detecting DREs and mapping references to FRBR entities (Santos et al., 2020). The tabular-LLM study points toward improved table grounding, joint training of generators and critics, and extension of DRE detection beyond tables to charts, spreadsheets, and other structured modalities (Yang et al., 30 Jun 2026). The medical retrieval study recommends cross-platform checking and continued manual verification (Gao et al., 21 Mar 2026). Data accounting suggests that complete summarization, explicit error paths, and per-record impact assertions could supply a general infrastructure for debugging referential failures in large analytical pipelines (Gajda, 2023).
The resulting picture is consistent across domains. DREs are not merely typographic nuisances or isolated hallucinations. They are failures of reference data as operational infrastructure: failures that obstruct identification, weaken linkage, degrade retrieval, and complicate both human verification and machine interpretation.