Document-Grounded QA

• Document-grounded QA is a system that extracts precise answers using information strictly present in provided documents, ensuring answer fidelity.
• It employs multi-stage pipelines that combine retrieval, spatial localization, and agentic reasoning to manage noisy, multi-page, and multimodal data.
• Emerging approaches integrate compositional reasoning with explicit spatial grounding to enhance interpretability and reduce hallucination in high-stakes applications.

Document-Grounded Question Answering

Document-grounded question answering (QA) is a research area in natural language understanding centered on generating answers to user queries based strictly on the content of one or more grounding documents. This field integrates techniques from information retrieval, multi-modal modeling, dialogue, and interpretable machine learning, with increasing emphasis on spatial localization, compositional reasoning, and faithfulness of answer derivation.

1. Definition, Scope, and Motivation

Document-grounded QA refers to systems that answer user questions by leveraging the explicit content of input documents—textual, visual, or both—rather than world knowledge or parametric memory alone. Unlike open-domain QA, which may synthesize or hallucinate based on large-scale LLM pretraining, document-grounded QA constrains model outputs to information verifiably present in the input document(s) (McDonald et al., 2022, Onami et al., 2024). This distinction is reflected in system architectures, dataset construction, evaluation methodologies, and application scenarios, particularly in high-stakes domains such as enterprise knowledge management, financial document analysis, legal compliance, and historical archive querying (Shi et al., 20 Jun 2025, Mudet et al., 14 Dec 2025).

The core challenge is to reliably extract, locate, synthesize, and justify the answer using content explicitly present in the provided document(s), which may be noisy, multi-page, multimodal, or semi-structured.

2. Methodological Taxonomy: Architectures and Pipelines

Document-grounded QA systems typically follow a multi-stage pipeline, often described as Detect–Retrieve–Comprehend (McDonald et al., 2022), with recent advances integrating compositional reasoning, spatial localization, and agentic tool use. Major paradigm distinctions include:

Paradigm	Main Workflow Steps	Notable Examples
Retrieval-Augmented Generation (RAG)	Retrieve evidence passages/chunks → condition a generator	eSapiens (Shi et al., 20 Jun 2025), DRC (McDonald et al., 2022), SimpleDoc (Jain et al., 16 Jun 2025)
Multi-Stage Extraction-Generation	Span labeling/extraction → answer composition	UniGDD (Gao et al., 2022), DocPrompt (Wu et al., 2023)
Agentic Tool-Driven Loop	Iterative search/read/answer actions	DocDancer (Zhang et al., 8 Jan 2026), Re3G (Zhang et al., 2023)
Spatial-Localization Enhanced	Answer string + bounding box/region grounding	DocExplainerV0 (Chen et al., 12 Sep 2025), BBox DocVQA (Yu et al., 19 Nov 2025)

Retrieval-Based Pipelines

Most practical systems operationalize document-grounded QA using retrieval-augmented pipelines, where documents are chunked (by page, paragraph, sentence), embedded in dense and/or sparse vector spaces, filtered or re-ranked, and passed as context to generative QA models (McDonald et al., 2022, Jain et al., 16 Jun 2025, Shi et al., 20 Jun 2025). Hybrid retrieval—dense (e.g., multilingual-e5-large) plus sparse (BM25)—is increasingly standard, with reciprocal rank fusion or LLM-driven re-ranking improving robustness to document noise, OCR errors, and orthographic variation (Mudet et al., 14 Dec 2025, Jain et al., 16 Jun 2025).

Visual and Multimodal Pipelines

Documents as images, especially PDFs, necessitate multi-modal pipelines: initial page or region segmentation (e.g., using SAM or DiT) is followed by OCR and/or visual embedding extraction (Yu et al., 19 Nov 2025, McDonald et al., 2022). Vision–LLMs (VLMs) generate answer strings, and may also output bounding boxes or region masks, either directly or via auxiliary modules such as DocExplainerV0 (Chen et al., 12 Sep 2025). Multi-modal models can process spatial layouts, figures, and tables—crucial for enterprise, scientific, and government document analysis.

Coarse-to-Fine and Agentic Architectures

Recent research integrates coarse-grained retrieval with fine-grained extraction/generation. Re3G, for instance, deploys a bi-encoder retriever, passage reranker, and prompt-based T5 span extractor in an early-fusion setup (Zhang et al., 2023). DocDancer formalizes the QA task as an agentic loop over actions (Search, Read, Answer) operating over structured outlines and multimodal content, with all steps trainable end-to-end using synthetic Exploration-then-Synthesis data (Zhang et al., 8 Jan 2026).

3. Spatial Grounding, Interpretability, and Faithfulness

Localization of answer provenance is central for both interpretability and deployment in real-world settings. Traditional VLMs, when prompted to return both answer and bounding box, exhibit a pronounced gap—high textual accuracy (Average Normalized Levenshtein Similarity, ANLS >0.7) yet very low mean Intersection-over-Union (IoU <0.04) for bounding-box localization (Chen et al., 12 Sep 2025). Plug-and-play modules like DocExplainerV0 decouple answer string generation from spatial localization, yielding substantial improvements in MeanIoU (up to ~0.19), but still well below OCR-based upper bounds (~0.49) (Chen et al., 12 Sep 2025, Yu et al., 19 Nov 2025).

BBox DocVQA extends this paradigm with a large-scale, multi-page, multi-region benchmark, supporting fine-grained quantitative evaluation of region-based QA (Yu et al., 19 Nov 2025). Multi-stage pipelines that first enumerate evidence bounding boxes, followed by answer generation, are shown to improve both accuracy and alignment—yet multi-page and multi-region queries remain challenging.

Masking-based region grounding, as in EaGERS, restricts VLM answer generation to spatially selected subregions via embedding similarity voting, further enhancing transparency without model retraining (Lagos et al., 15 Jul 2025). Across methodologies, correct spatial grounding is critical for interpretability (answer location reference) and faithfulness (requiring that model outputs be supported by retrieved/contextualized content) (Mudet et al., 14 Dec 2025, Shi et al., 20 Jun 2025).

4. Reasoning, Generalization, and Structured Content

Complex question answering over documents requires compositional and systematic reasoning. The GLT (Grounded Latent Trees) framework employs explicit latent-tree induction (using a CKY-style chart) over question spans, with span-level representations and denotations grounded to document elements. This inductive bias yields strong out-of-distribution generalization on both arithmetic and visual QA datasets, outperforming standard transformers on length and operator-split tasks (Bogin et al., 2020).

Structured document elements—lists, tables, and hierarchical relationships—demand specialized treatment. LIST2QA and the ISL (Intermediate Steps for Lists) pipeline parse document lists, classify their logical relations (e.g., AND, OR), align user context to list items, and inject explicit intermediate reasoning steps into model inputs, achieving measurable gains in answer faithfulness and completeness (Sung et al., 2024). Extractive summarization and sequential block selection approaches (e.g., MemSum-DQA) efficiently address long-document QA and relationship understanding by prefixing question information at the block level and iteratively selecting relevant spans (Gu et al., 2023).

5. Datasets, Evaluation Metrics, and Empirical Findings

A substantial body of benchmarks exists for document-grounded QA, differing by domain, modality, scale, and annotation richness.

• BoundingDocs v2.0: 48,151 documents, 249,016 QA pairs, annotated bounding boxes, 8 languages (Chen et al., 12 Sep 2025).
• BBox DocVQA: 3,671 documents, ~32,000 QA pairs, fine-grained manual bounding box annotations (Yu et al., 19 Nov 2025).
• JDocQA: 5,504 Japanese PDFs, 11,600 QA pairs with bounding boxes and unanswerable questions (Onami et al., 2024).
• LIST2QA: 2,498 QA triples focused on structured list reasoning, covering multiple domains (Sung et al., 2024).
• QASPER: Scholarly PDFs with 5,049 questions, supports extractive, abstractive, Boolean QA (McDonald et al., 2022).
• DocCVQA: Joint document collection QA and evidence retrieval across 14,362 form images (Tito et al., 2021).

Metrics include ANLS (string similarity), MeanIoU (bounding box overlap), Exact Match, F1 for span extraction, BLEU and ROUGE-L for generation, and faithfulness/completeness as assessed by LLMs or human judges. Datasets encompassing unanswerable questions promote abstention and reduce hallucination (Onami et al., 2024, Mudet et al., 14 Dec 2025).

Empirical studies consistently demonstrate:

• Large pretrained VLMs and generative LLMs can achieve high answer string accuracy, but spatial grounding and evidence alignment remain significant weaknesses without explicit localization modules or fine-tuning on grounding-annotated data (Chen et al., 12 Sep 2025, Yu et al., 19 Nov 2025).
• Multi-stage or agentic pipelines (SimpleDoc, DocDancer, Re3G) yield state-of-the-art results with efficient retrieval and iterative evidence integration (Jain et al., 16 Jun 2025, Zhang et al., 8 Jan 2026, Zhang et al., 2023).
• Explicit modeling of document structure, region annotations, or list semantics leads to substantial gains over baseline end-to-end LLMs (Sung et al., 2024, Gu et al., 2023).

6. Limitations, Challenges, and Future Directions

Despite rapid progress, several open challenges persist:

• Spatial Grounding: Even advanced multimodal models have a large gap between answer string correctness and spatial localization fidelity; MeanIoU remains far below possible upper bounds, especially on multi-page and multi-element queries (Chen et al., 12 Sep 2025, Yu et al., 19 Nov 2025).
• Scalability and Efficiency: Handling extremely long documents (>256k tokens) or collections requires linear-time or efficient single-pass models (e.g., state-space models), as chunked retrieval pipelines lose context (Cao et al., 4 Apr 2025).
• Structure, Layout, and Multimodality: Tables, lists, figures, and complex visual layouts still pose detection and reasoning difficulties—new training objectives and benchmarks are needed for grounding and multi-hop reasoning across structured elements (Sung et al., 2024, Yu et al., 19 Nov 2025).
• Hallucination and Abstention: Encouraging systems to abstain on unanswerable queries is critical. Inclusion of unanswerable examples in training and evaluation demonstrably reduces unsupported hallucinations (Onami et al., 2024, Mudet et al., 14 Dec 2025).
• End-to-End Training: Most current systems operate in staged or modular fashion; future work may focus on joint optimization of retrieval, reasoning, and grounding using unified end-to-end objectives (Chen et al., 12 Sep 2025, Shi et al., 20 Jun 2025).

Recommendations include explicit spatial-objective integration, extensions to multi-region/multi-hop answers, end-to-end multi-modal pipelines including OCR and layout, and the development of standardized evaluation for multimodal, grounded, and abstaining QA systems (Chen et al., 12 Sep 2025, Yu et al., 19 Nov 2025, Shi et al., 20 Jun 2025, Mudet et al., 14 Dec 2025).

7. Applications and Impact

Document-grounded QA underpins applications in scientific literature mining, financial document analysis, enterprise knowledge access, legal and regulatory compliance, government information services, and historical archive exploration. Faithful, interpretable, and spatially grounded answers facilitate transparency, support auditability, and enable human-in-the-loop validation—key requirements in settings involving contracts, invoices, medical records, and archival research (Shi et al., 20 Jun 2025, Mudet et al., 14 Dec 2025, Onami et al., 2024).

The integration of spatial grounding modules, agentic exploration, and structured reasoning establishes a robust foundation for next-generation document understanding systems capable of accurate, explainable, and trustworthy information extraction from complex, real-world documents (Chen et al., 12 Sep 2025, Jain et al., 16 Jun 2025, Zhang et al., 8 Jan 2026, Sung et al., 2024).