Document-Level Retrieval Mismatch (DRM)

• DRM is characterized by retrieval systems selecting high-similarity text chunks from incorrect documents, with top-k mismatch rates exceeding 95% in legal benchmarks.
• Summary-Augmented Chunking (SAC) injects concise LLM-generated summaries into each chunk to restore global document context and reduces DRM by about 50%.
• The technique boosts retrieval precision and recall while ensuring provenance fidelity in scalable, RAG-based legal document systems.

Document-Level Retrieval Mismatch (DRM) refers to a critical failure mode in information retrieval systems—especially those built on Retrieval-Augmented Generation (RAG) paradigms applied to large, highly structured datasets such as legal corpora—where the retrieval engine selects information from entirely incorrect source documents. DRM arises when the retriever, despite achieving high chunk-level or word-level similarity, fails at the document level by returning chunks not belonging to the document holding the ground-truth content. This phenomenon is quantified by measuring the proportion of top-k retrieved chunks not part of the answer-containing document. The structural homogeneity and repeated boilerplate language typical of legal datasets exacerbate DRM, making global document context and provenance a fundamental requirement for reliable retrieval.

1. Definition, Symptoms, and Measurement of DRM

In the legal domain, DRM is the situation in which a retriever selects text chunks based on local similarity metrics but, crucially, the chunks originate from documents other than the ground-truth answer’s source. The breakdown can be subtle; for example, boilerplate clauses or standardized legal phrasing may yield high lexical overlap but lack provenance correlation. Empirically, DRM manifests as very high mismatch rates—document-level mismatch percentages exceeding 95% in certain ContractNLI benchmarks highlight the severity of the issue. DRM is evaluated as the fraction of retrieved chunks in the top-k that do not belong to the correct document.

Table: DRM Measurement in a Legal RAG Benchmark

Dataset	Top-k DRM (%)	Error Type
ContractNLI	>95	Chunks from incorrect document
PrivacyQA	High	Structural similarity confusion

These metrics are derived from direct observation of retrieval outputs: even when semantic similarity is preserved at the chunk level, the lack of document-level alignment leads to severe errors.

2. Structural Challenges in Large Legal Datasets

Legal corpora (contracts, NDAs, privacy policies) are built on standardized templates and hierarchies, resulting in near-identical structures across documents. When traditional chunking methods split documents into small, context-independent pieces (often using character or paragraph boundaries), the retriever’s embedding model loses the full hierarchical and contextual signals necessary to disambiguate document provenance. Boilerplate headings and repeated legal terminology increase the chance of retrieving superficially similar but incorrect chunks. Without a document-level “fingerprint,” retrieval is easily misled by these similarities.

This structural redundancy necessitates a retrieval strategy that preserves global document context during chunk-level scoring and ranking, otherwise DRM is inevitable.

3. Summary-Augmented Chunking (SAC): Technique and Process

Summary-Augmented Chunking (SAC) is a pragmatic and scalable intervention designed to mitigate DRM by injecting document-level context into every chunk. The SAC process consists of the following steps:

1. Document Summarization: For each document, an LLM generates a concise synthetic summary (typically ~150 characters) that captures the primary entities, purpose, and legal topics.
2. Chunking: The document is partitioned into smaller segments using a recursive character splitting strategy that respects natural linguistic boundaries.
3. Summary Augmentation: The generated summary is prepended to every chunk derived from the corresponding document.
4. Indexing: These summary-augmented chunks are then embedded and indexed within a vector database.

This process ensures that the retriever operates on enriched chunks, each carrying both local textual content and a high-salience global summary as provenance. The operational pseudocode for SAC is:

For each document D in corpus: Generate summary S = LLM_summarize(D) # S ≈ 150 chars Split D into chunks C1, C2, ..., Cn # Recursive strategy For each chunk Ci: Augmented chunk Ci' = S + " " + Ci Index Ci' via embedding model

The summarization prompt is formulated to request a brief, informative summary focusing on entities and principal legal topics—not dense legalese—thus optimizing generalization across retrieval queries.

4. Experimental Evidence and Comparative Findings

Extensive experiments conducted on the LegalBench-RAG suite—spanning tasks like contract review (CUAD), merger agreements (MAUD), ContractNLI, and privacy policy QA—validate the efficacy of SAC:

• DRM Reduction: SAC reduces the rate of document-level retrieval mismatch by approximately 50% relative to standard chunking pipelines across diverse top‑k retrieval settings.
• Precision and Recall Gains: The injection of global context via SAC yields not only lower DRM but also higher text-level precision and recall, indicating that retrieved snippets are both provenance-faithful and locally relevant.
• Generic vs. Expert-Guided Summaries: When comparing generic LLM-generated summaries with legal expert prompts, generic summaries outperform on DRM reduction and overall retrieval quality. Expert summaries, while rich and detailed, induce overfitting to narrow cues and exhibit lower generalizability.

5. Integration and Scalability in RAG Systems

SAC is engineered as a low-overhead, modular augmentation to existing retrieval pipelines. It requires only one LLM summarization per document and an adjustment in the chunking process—no fundamental change to backbone retrieval, embedding, or ranking mechanisms. This design enables rapid integration, robust scalability for evolving document corpora, and minimal computational overhead. SAC’s performance does not degrade under dynamic updates or expansions of the document database, making it suitable for real-world, large-scale legal information retrieval systems.

A plausible implication is that such an approach should also generalize to other high-structure contexts (e.g., business contracts, regulatory documents) where document-level context is a critical signal.

6. Implications for System Reliability and Future Directions

By directly attacking the main cause of hallucinations—retrieval of text from incorrect documents—SAC significantly enhances the reliability and faithfulness of RAG systems in high-stakes domains. The improved provenance tracking via summary augmentation justifies both increased precision and trustworthiness required in sensitive legal applications.

Further practical avenues include:

• Extending SAC to richer multi-modal contexts (e.g., documents with tables, images, diagrams).
• Automating the summarization procedure for dynamic and streaming corpora.
• Investigating optimal balance between summary length, chunk granularity, and retrieval accuracy for specific domains.

This suggests that future retrieval systems could universally benefit from integrating document-level context augmentation, particularly in any domain typified by template-driven structure and repeated linguistic patterns.

Summary

Document-Level Retrieval Mismatch (DRM) is a failure mode endemic to RAG-based legal information retrieval attributable to the loss of document-level context during chunk-based ranking. Summary-Augmented Chunking (SAC) mitigates DRM by prepending concise, LLM-generated synthetic summaries to every chunk, thus restoring global provenance signals and dramatically increasing the reliability of retrieval. Experimental data establish that SAC reduces DRM, raises precision and recall, and is more effective when using generic rather than expert-guided summarization. SAC’s low integration overhead and scalability make it suitable for deployment in mission-critical legal AI systems, establishing a clear path toward robust, provenance-faithful retrieval even in homogeneously structured document collections (Reuter et al., 8 Oct 2025).