Explicit Evidence Grounding via Structured Inline Citation Generation

Abstract: As AI systems become more widely adopted, the demand for factual and faithful generation grows. Properly attributing information through citations becomes, therefore, crucial. This work introduces FullCite, a framework that, in contrast to most previous works, generates structured inline citations linking each claim to both its source document and supporting evidence. FullCite proposes three strategies to inline citation generation: prompt-based generation, constrained decoding over a citation grammar, and posthoc span alignment. Using three question answering benchmarks, namely, ASQA, BioASQ, and ExpertQA, we assess citation quality and faithfulness along three dimensions: document-level correctness, evidence span identification, and claim-citation faithfulness. Our evaluation shows that while LLMs are generally effective at identifying relevant documents, they struggle to identify the precise supporting spans within them. This gap suggests that achieving faithful attributed QA will require research to place greater emphasis on precise evidence span identification.

• The paper introduces FullCite, which generates structured inline citations to achieve fine-grained evidence grounding for LLM claims.
• It utilizes prompt-based generation, constrained decoding, and posthoc alignment to enhance citation precision and support verification.
• Experimental results reveal significant gains in snippet-level citation accuracy, though challenges remain in comprehensive document coverage and bias mitigation.

Explicit Evidence Grounding via Structured Inline Citation Generation

Motivation and Problem Definition

Direct answer generation by LLMs is rapidly displacing conventional search engines for information access. This transition foregrounds the issues of output verifiability and faithfulness, particularly in high-stakes domains where grounding every claim in trustworthy evidence is essential. Prior approaches to attributions in generative QA primarily focus on document-level citations, offering only coarse-grained support. Such citations frequently fail to unambiguously substantiate individual claims since portions of the cited document may be irrelevant or even contradictory. Thus, explicit fine-grained attribution—where citations link each claim to precise supporting evidence at the span level—is a central open challenge.

The FullCite Framework

FullCite introduces a framework for generating structured inline citations that simultaneously address document-level and span-level attribution. Each generated claim is directly followed by a citation structured as {doc_id: <id>, snippet: <verbatim evidence>}. This fine-grained approach is operationalized via three distinct strategies:

• Prompt-based generation: Models are instructed to insert verbatim citations during auto-regressive decoding.
• Constrained decoding: Decoding is restricted by a finite-state automaton enforcing both structure and verbatim grounding.
• Posthoc alignment: Generated snippets are mapped to the most similar evidence spans from retrieved documents using word-token Jaccard similarity, correcting for near-verbatim outputs.

  Figure 1: An overview of FullCite, structured inline generation.

Experimental Design

FullCite is evaluated on three QA datasets spanning general (ASQA), biomedical (BioASQ), and multi-domain (ExpertQA) tasks. ASQA and ExpertQA were extended with additional evidence-level annotations using a semi-automatic process followed by manual validation. The evaluation benchmarks citation quality along three orthogonal axes:

• Document-level correctness (Doc-F1)
• Evidence span localization (Snippet-F1)
• Claim-citation faithfulness (using semantic similarity, LLM and human evaluation)

Models include Qwen3-8B and Gemma-3-12B-it, representing modern open-weight architectures. The three FullCite strategies, along with posthoc generate-then-retrieve and ReClaim constrained baselines, create a comprehensive comparison.

Results

Document vs. Span Attribution

Across all datasets and model configurations, document-level citing is consistently more reliable than evidence span localization. For example, generate-then-retrieve (posthoc baseline) achieves Doc-F1 as high as 94% on ExpertQA and 80% on ASQA, yet Snippet-F1 scores remain substantially lower (see main-text). This indicates LLMs can identify relevant documents but lack the granularity for constant fine-grained evidence extraction.

FullCite's posthoc strategy delivers decisive gains in Snippet-F1: On ASQA, Qwen3-8B improves from 12.80 to 61.87 and Gemma-3-12B-it from 12.42 to 41.80. While Doc-F1 and Snippet-F1 improvements are more moderate for BioASQ and ExpertQA, the approach demonstrates its robustness by trading off minor declines in document coverage for substantial increases in snippet precision.

Figure 2: Overall quality comparison of citations across settings and datasets.

Biases and Error Modes

A systematic primacy bias is observed: 81.8% of BioASQ citations, regardless of method, target the first two of five provided documents—a clear instantiation of the lost-in-the-middle effect. Rather than comprehensive attribution, models default to citing documents with positional primacy, limiting evidence coverage.

Figure 3: Position of the cited document in BioASQ, evidencing strong primacy bias.

Yes/no questions display the lowest citation rates—models frequently omit citations altogether for such items. The baseline prompt-based setting reflects this in inflated downstream QA metrics, which diminish when attribution is enforced.

Figure 4: Citation rate for each question type under each setting, showing suppression for yes/no queries.

Claim-Citation Faithfulness

Evaluation with both LLMs (GPT-5.4-as-Judge) and human annotators exposes a trade-off: posthoc span alignment increases snippet correctness but may slightly degrade semantic faithfulness between claim and citation (relative to baseline joint decoding strategies). Nevertheless, the average support and relevance scores remain high, with agreement between human and LLM evaluation showing that automated judging is robust for this use case.

Figure 5: Average scores for Citation Support (Q1) and Citation Relevance (Q2) by GPT-5.4.

Figure 6: Agreement and correlation between human annotators and GPT-5.4 for support and relevance judgments.

Coverage and Uniqueness

Analysis demonstrates that, even with FullCite, models often fail to comprehensively cite all relevant gold documents, with limited unique document coverage. The posthoc and constrained decoding settings improve on prompt-based strategies in overall citation rate, unique cited documents, and error reduction, but complete coverage remains out of reach.

Figure 7: Ratio of unique cited documents, quantifying citation exhaustiveness.

Figure 8: Overall citation rate per method; structure-enforcing strategies yield more consistent attribution.

Posthoc citation repair also exhibits sensitivity to the similarity threshold; tuning this hyperparameter is essential for optimal evidence localization without excessive mismatches.

Figure 9: Quality of chosen citation under different similarity thresholds for posthoc citation generation.

Downstream QA Task Performance

For factoid, list, and summary questions, enforcing fine-grained attribution generally improves downstream QA answer correctness compared to ungrounded generation, except in yes/no cases where omission of citations in baseline artificially inflates F1. FullCite's gains are most pronounced for tasks requiring unambiguous, verifiable responses.

Implications and Future Directions

FullCite establishes that document-level retrieval and coarse attribution are insufficient for robust, faithful attributed text generation with LLMs. Precise evidence span grounding remains unsolved—further progress will necessitate innovations in model architecture, data annotation, and decoding or training strategies that prioritize fine-grained verifiability.

Persistent biases (e.g., primacy, context-length limitations) and task-specific behaviors (e.g., omitted citations for binary responses) highlight limitations of current models for critical application domains. Methods that jointly optimize document and span-level attribution—potentially via instruction tuning or preference optimization informed by frameworks such as SelfCite [Chueng et al., 2025]—are likely to be fruitful.

Enhanced metrics and scalable human-LM consensus evaluation protocols are warranted to distinguish surface-level similarity from genuine factual grounding. Additionally, extending evaluation to more complex, adversarial, or fine-grained attributions (such as subsentence-level support [Cao & Wang, 2024]) will be essential for comprehensive model assessment and deployment.

Conclusion

FullCite advances the state of attributed question answering by integrating document and verbatim span attribution into a unified generation and evaluation framework. Despite strong document-retrieval capacity, LLMs under all current prompting and decoding strategies exhibit substantive challenges in precise support localization. Enforcing explicit fine-grained citation is both empirically superior and more transparent for stakeholders seeking reliable, audited evidence chains from autoregressive models. These findings reinforce the theoretical and practical requirement for verifiable grounding in LLM output—an essential precondition for safe and trustworthy AI deployment in high-stakes settings.

Please refer to (2606.07130) and related citations for full bibliographic details.