CiteGuard: Retrieval-Augmented Citation Validation
- CiteGuard is a retrieval-augmented validation framework that reframes citation evaluation as citation attribution alignment, checking if citations match what a competent human author would select.
- It employs a multi-stage workflow that retrieves relevant literature and compares candidate citations against alternative valid sources to substantiate scientific claims.
- Empirical results on the CiteME benchmark show a 12.3% improvement and up to 65.4% accuracy, narrowing the gap toward human-level citation validation.
CiteGuard is a retrieval-augmented validation framework for checking whether citations attached to LLM-generated scientific text are actually appropriate. Its defining contribution is to reframe citation evaluation as citation attribution alignment: a generated citation is faithful if it aligns with what a competent human author would reasonably cite for the same text, given the available literature, rather than merely passing a binary support judgment on the cited paper in isolation (Choi et al., 15 Oct 2025).
1. Problem setting and motivation
CiteGuard addresses a practical failure mode in LLM-based scientific writing assistants: models can produce fluent prose with citations, but the citations are often unfaithful. In the formulation associated with CiteGuard, unfaithfulness includes fabricated references, attaching a real paper that does not actually support the claim being made, and citing a paper that is weaker or less appropriate than what a human author would choose (Choi et al., 15 Oct 2025).
The framework is motivated by limitations in citation-checking approaches based on LLM-as-a-Judge. The cited framing argues that a judge model asked whether a paper supports a claim may rely too heavily on priors and surface plausibility rather than direct evidence, may lack access to relevant external literature beyond the cited paper, may treat validation as a binary support check even though scientific writing often admits multiple valid citations, and may inherit calibration, bias, and grounding problems already documented for judge models (Choi et al., 15 Oct 2025).
This problem formulation matters because the surface form of a citation can look plausible even when the underlying attribution is wrong. CiteGuard therefore treats citation validation as a literature-grounded attribution problem rather than as isolated semantic relatedness.
2. Citation attribution alignment
The central concept in CiteGuard is citation attribution alignment. The task is: given an LLM-generated text span and a citation attached to it, determine whether that citation aligns with what a human author would cite for the same text (Choi et al., 15 Oct 2025).
In the paper’s framing, alignment has two parts. First, support/evidence validity: the cited paper should actually substantiate the claim or statement in the generated text. Second, attribution appropriateness relative to available alternatives: the citation should be one that a human author could reasonably choose, rather than an arbitrary related paper (Choi et al., 15 Oct 2025).
This reframing is stricter and more realistic than asking only whether the cited paper contains overlapping keywords or broadly related content. A citation can be “about the topic” and still be a poor attribution choice. Conversely, a citation may differ from the benchmark answer and still be valid because there are alternative valid citations that support the same statement. CiteGuard is therefore designed to identify both whether a given citation is faithful and whether other valid citations also align with the text (Choi et al., 15 Oct 2025).
A plausible implication is that CiteGuard treats benchmark mismatch as a modeling problem rather than as a labeling nuisance. Scientific writing rarely has a single uniquely correct citation, and the framework is organized around that premise.
3. Retrieval-augmented validation workflow
CiteGuard is described as “Faithful Citation Attribution for LLMs via Retrieval-Augmented Validation.” It is retrieval-augmented, agentic, and validation-focused. Rather than asking an LLM to judge a citation only from the sentence and the cited paper, CiteGuard first retrieves relevant literature and then uses that broader context during validation (Choi et al., 15 Oct 2025).
At a high level, the workflow proceeds in stages. The system takes an LLM-generated sentence or passage together with its attached citation. It retrieves papers relevant to the textual claim, including the cited paper itself when accessible, semantically relevant literature, and likely competing or alternative citations. It then gathers evidence from retrieved documents and runs an agentic validation process that reasons over the claim and the evidence. A distinctive stage is alternative valid citation identification: if the benchmark citation differs from the generated one, the framework asks whether the generated citation is still an acceptable attribution according to the retrieved evidence, and if the generated citation is weak or unsupported, it can identify stronger alternatives (Choi et al., 15 Oct 2025).
Retrieval is the key architectural differentiator. Without retrieval, a judge model sees only a sentence and a candidate citation, so it cannot answer comparative questions such as whether the citation is merely tangential, whether better-aligned sources exist, or whether a benchmark mismatch is acceptable because another valid citation was chosen. CiteGuard’s validation criteria are therefore grounded in direct support evidence, comparison against retrieved alternatives, and alignment with human citation practice (Choi et al., 15 Oct 2025).
4. Benchmarking and empirical results
CiteGuard is evaluated on CiteME, a benchmark used to study citation faithfulness and alternative valid citations. The headline empirical claims are that CiteGuard improves the prior baseline by 12.3%, achieves up to 65.4% accuracy on CiteME, and approaches human performance of 69.7% (Choi et al., 15 Oct 2025).
| Measure | Reported value |
|---|---|
| Improvement over prior baseline | 12.3% |
| Best accuracy on CiteME | 65.4% |
| Human performance | 69.7% |
These results support the paper’s thesis that retrieval-aware, attribution-aware validation is more effective than judge-only validation. The metric explicitly reported is accuracy, and the comparison is framed as closing part of the gap between automatic citation validation and human judgment rather than fully solving the task (Choi et al., 15 Oct 2025).
The empirical emphasis on alternative valid citations is also methodologically significant. Exact-match evaluation can wrongly penalize acceptable citations when multiple papers support the same statement. CiteGuard’s reported ability to identify alternative but valid citations is therefore part of both its validation behavior and its evaluation philosophy (Choi et al., 15 Oct 2025).
5. Limitations and practical significance
CiteGuard does not claim human-level reliability. Even at up to 65.4% accuracy, there remains a gap to the reported 69.7% human score, and the absolute accuracy remains modest for a high-stakes scientific-writing task (Choi et al., 15 Oct 2025).
The paper’s framing suggests several persistent difficulty classes. Retrieval bottlenecks remain fundamental: if the retriever fails to surface the most relevant or canonical papers, the validator loses the literature context needed for correct comparison. Subtle support mismatches are also difficult: papers may be topically related while differing in population, task definition, modality, experimental setting, or strength of evidence. Annotation ambiguity remains intrinsic because citation appropriateness can be subjective, especially when multiple reasonable sources exist. Finally, retrieval does not remove dependence on LLM reasoning; even with external evidence, errors can arise from misreading, overgeneralization, or weak comparative judgment (Choi et al., 15 Oct 2025).
The practical implication is that trustworthy scientific writing assistants need validation mechanisms, not just generation mechanisms. In this view, generating a plausible citation string is insufficient. A deployed assistant should verify whether a citation truly supports the generated statement, recognize when another citation would be more appropriate, and tolerate the fact that multiple references may be valid. CiteGuard’s significance lies in turning citation checking into a retrieval-grounded, attribution-aware validation problem (Choi et al., 15 Oct 2025).
6. Relation to adjacent citation-verification research
CiteGuard sits within a broader research program on citation integrity, but its emphasis is specific: it validates attribution choices in generated scientific text rather than only checking existence, only discovering references, or only scoring report-level source quality.
Several adjacent systems illustrate this division of labor. CiteAudit focuses on hallucinated scientific references and presents a multi-agent verification pipeline centered on extraction, retrieval, and strict metadata-based judgment, with strong emphasis on reference authenticity and field-level mismatch reports (Yuan et al., 26 Feb 2026). citecheck is a TypeScript system and MCP server for bibliographic verification and repair in paper-like project folders; it validates entries against PubMed, Crossref, arXiv, and Semantic Scholar and returns structured correction proposals together with replacement-safety diagnostics (Lee, 18 Mar 2026). CiteLLM addresses trustworthy reference discovery rather than post hoc validation, retrieving candidates exclusively from trusted repositories and using LLMs for query generation, ranking, and paragraph-level support explanation (Hong et al., 26 Feb 2026). DeepResearchGuard is broader still: a multi-stage source-safety and report-safety framework in which the reference guard scores sources on helpfulness, authority, and timeliness and the output stage includes “Credibility & Citation Quality” as a report dimension (Huang et al., 13 Oct 2025).
From the generation side, C-Cite treats citation markers as contextual semantic pointers and reports an average 5.8% gain in citation quality and 17.4% in response correctness on ALCE, but its main target is attributed generation rather than retrieval-aware citation validation (Yu et al., 19 Nov 2025). From the recommendation side, CiteRAG benchmarks citation prediction with separate coarse-grained and fine-grained tasks and a retrieval-augmented generation setup, addressing what should be cited rather than whether a generated citation is faithful to a specific claim (Zheng et al., 21 Jan 2026).
Within that landscape, CiteGuard is distinguished by its combination of retrieval-augmented validation, citation attribution alignment, and explicit handling of alternative valid citations. Its core object is not merely the existence of a reference, the retrieval of a candidate paper, or the formatting of a bibliography entry, but the faithfulness of attribution in LLM-generated scientific writing (Choi et al., 15 Oct 2025).