Replace, Don't Expand: Mitigating Context Dilution in Multi-Hop RAG via Fixed-Budget Evidence Assembly (2512.10787v1)

Abstract: Retrieval-Augmented Generation (RAG) systems often fail on multi-hop queries when the initial retrieval misses a bridge fact. Prior corrective approaches, such as Self-RAG, CRAG, and Adaptive-$k$, typically address this by \textit{adding} more context or pruning existing lists. However, simply expanding the context window often leads to \textbf{context dilution}, where distractors crowd out relevant information. We propose \textbf{SEAL-RAG}, a training-free controller that adopts a \textbf{``replace, don't expand''} strategy to fight context dilution under a fixed retrieval depth $k$. SEAL executes a (\textbf{S}earch $\rightarrow$ \textbf{E}xtract $\rightarrow$ \textbf{A}ssess $\rightarrow$ \textbf{L}oop) cycle: it performs on-the-fly, entity-anchored extraction to build a live \textit{gap specification} (missing entities/relations), triggers targeted micro-queries, and uses \textit{entity-first ranking} to actively swap out distractors for gap-closing evidence. We evaluate SEAL-RAG against faithful re-implementations of Basic RAG, CRAG, Self-RAG, and Adaptive-$k$ in a shared environment on \textbf{HotpotQA} and \textbf{2WikiMultiHopQA}. On HotpotQA ($k=3$), SEAL improves answer correctness by \textbf{+3--13 pp} and evidence precision by \textbf{+12--18 pp} over Self-RAG. On 2WikiMultiHopQA ($k=5$), it outperforms Adaptive-$k$ by \textbf{+8.0 pp} in accuracy and maintains \textbf{96\%} evidence precision compared to 22\% for CRAG. These gains are statistically significant ($p<0.001$). By enforcing fixed-$k$ replacement, SEAL yields a predictable cost profile while ensuring the top-$k$ slots are optimized for precision rather than mere breadth. We release our code and data at https://github.com/mosherino/SEAL-RAG.

• The paper introduces a novel SEAL-RAG system that replaces evidence addition with gap-aware, fixed-budget replacement to enhance multi-hop QA accuracy.
• The paper details an entity-centric, iterative controller that diagnoses evidence gaps and uses targeted micro-queries for precise retrieval.
• Experimental results on HotpotQA and 2WikiMultiHopQA demonstrate significant gains in evidence precision and answer accuracy under strict context budgets.

Replace, Don’t Expand: Fixed-Budget Evidence Assembly for Multi-Hop RAG

Introduction

This work re-examines the persistent context dilution failure mode in multi-hop retrieval-augmented generation (RAG) systems, specifically under strict inference budget constraints. Context dilution—where an expanded retrieval set decreases the relative density of relevant evidence—remains a critical bottleneck for multi-hop question answering tasks. Existing iterative RAG controllers, including Self-RAG and CRAG, address missing bridge facts by expanding the context window via breadth-first addition or pruning, but typically operate under the assumption that increased context capacity yields higher recall and answer coverage. However, both empirical and theoretical analysis (see [liu2023lostinthemiddle]) demonstrates that augmentation without selectivity is counterproductive due to the generator's limited ability to filter distractors, particularly as token lengths grow.

The paper introduces SEAL-RAG, a training-free controller distinguished by a "replace, don't expand" paradigm for optimally assembling the fixed-size context set via iterative gap-aware evidence repair. This new strategy enforces a hard constraint on context size ($k$), treating the evidence window as a scarce resource to be intensely curated, rather than a buffer to be accumulated.

Figure 1: SEAL-RAG pipeline (Search $\rightarrow$ Extract $\rightarrow$ Assess $\rightarrow$ Loop)—from initial retrieval through targeted entity-driven repair, orchestrating fixed-budget evidence assembly.

The SEAL-RAG Controller

Architectural Overview

SEAL-RAG reformulates multi-hop RAG as a fixed-budget evidence optimization problem. Rather than appending new passages or passively pruning from an initial pool, SEAL-RAG leverages an explicit entity-centric, iterative controller that: (1) diagnoses missing entities/relations by projecting the unstructured context into a structured ledger; (2) issues atomic, gap-driven micro-queries to the retriever; and (3) replaces lowest-utility items in the evidence set with higher-value candidates, strictly maintaining $|E| = k$ throughout.

Figure 2: Execution graph for SEAL-RAG—showing the flow of evidence ledger updating, sufficiency gating, repair triggering, and candidate replacement.

The system state at each loop consists of the current evidence buffer ($E_t$), a structured entity ledger ($U_t$), and a blocklist ($B_t$) for unproductive queries, ensuring loop efficiency and avoidance of cyclical stagnation.

Core Mechanisms

• Gap-Aware Sufficiency Assessment: The controller invokes an LLM-based scoring function integrating coverage, corroboration, contradiction, and answerability metrics to determine whether the current $E_t$ supports a sufficient answer given the question schema.
• Explicit Gap Specification: By mapping from question attribute requirements to the entity ledger, the controller identifies missing entities, relations, or qualifiers. These precise gaps inform the construction of targeted, non-generic micro-queries that minimize retrieval drift and maximize semantic coverage.
• Fixed-Capacity Replacement Policy: An entity-first utility ranking, functionally reminiscent of MMR (but avoiding relevance-dominated redundancy), scores each candidate based on gap coverage, corroborative strength, novelty, and penalizes lexical redundancy. Candidates only replace current evidence if they improve the utility margin by at least a hysteresis threshold, with additional dwell-time protection to prevent thrashing.

Complexity Control

By enforcing a fixed $k$ and capping the loop budget $L$, SEAL-RAG offers strictly predictable inference cost, both in terms of retriever invocations and generator context length. Only a single, fixed-width context is ever input to the generator—a substantial efficiency and cost control improvement relative to baseline policies that allow unbounded context expansion.

Empirical Evaluation

Experimental Design

A stringent, controller-isolated environment ensures that all methods—including Basic RAG, Self-RAG, CRAG, and Adaptive-$k$—share identical retrievers, retrieval indices, document segmentation, and generation backbones (multiple variants of GPT-4). Only high-level controller logic is varied, ensuring any performance deltas derive from the replacement vs. addition paradigm rather than hardware, retrieval, or model factors.

Evaluation is conducted on HotpotQA and 2WikiMultiHopQA, using both single-hop ($k=1$) and multi-hop ($k=3,5$) settings. Answer correctness is externally judged by GPT-4o, blinded to model and parametric context. Evidence precision and recall are calibrated via rigorous title normalization.

Main Results

Across all settings, SEAL-RAG delivers consistent, significant improvements in answer accuracy (Judge-EM) and evidence precision, particularly as $k$ increases. Notably:

• On HotpotQA under $k=3$, SEAL-RAG leads the strongest baseline by +3 to +13 percentage points in accuracy and +12 to +18 pp in evidence precision.
• On 2WikiMultiHopQA at $k=5$, SEAL-RAG attains 96% evidence precision (vs. 22% for CRAG) and outperforms Adaptive-$k$ by +8.0 pp in accuracy.
• All performance gains are statistically significant under paired McNemar and t-test analyses ($p < 0.001$).

Critically, results demonstrate that context expansion by addition (CRAG, Self-RAG) drives recall but at the cost of catastrophic precision collapse due to distractors, while selection-based methods (Adaptive-$k$) are strictly bounded by the recall ceiling of the initial candidate pool. Only SEAL-RAG, by dynamically expanding the retrieval frontier and enforcing replacement over addition, maintains high-precision evidence selection under a hard budget.

Ablations and Error Analysis

Loop budget ablation establishes that almost all accuracy gain is accrued within the first or second repair, supporting the claim that gap-driven repair, when accurately targeted, is highly efficient. Qualitative case studies detail typical successful and failure pathways; failures center upon alias resolution errors and extraction noise, both addressable by more robust entity-linking and verification modules at some cost to complexity and latency.

Theoretical and Practical Implications

The SEAL-RAG controller reframes the notion of context in RAG, formally demonstrating that, under known LLM attention allocation and context overload phenomena ([liu2023lostinthemiddle]), strict replacement outperforms expansion for compositional, multi-hop queries. Practically, this architecture affords hard inference budget guarantees even in online, real-time RAG settings—a property necessary for production-scale deployment, especially as LLM context windows continue to increase.

The fixed-budget repair framework generalizes naturally to non-LLM settings (e.g., retrieval-based or hybrid architectures) and provides new baselines for robust evidence set optimization under constrained resources, potentially informing model design in long-context or retrieval-heavy agentic pipelines.

Future Developments

Future directions include integrating more sophisticated zero-shot entity linking, dynamic ledger composition over streaming contexts, and extending replacement policies to hierarchical or tree-structured evidence accumulation. Further, joint optimization over retrieval and generation could yield additional precision/recall synergy. Exploring learned (rather than rule-based) replacement scoring functions and integrating chain-of-verification paradigms is another promising avenue.

Conclusion

This work establishes that context curation via evidence replacement, not expansion, is key to effective multi-hop RAG under hard budget constraints. SEAL-RAG’s gap-driven controller, by combining explicit sufficiency checking, targeted micro-queries, and fixed-capacity replacement, provides repeatable gains in both accuracy and evidence precision relative to both additive and static selection baselines. These results substantiate Fixed-Budget Evidence Assembly as a necessary design element for scalable, robust, and cost-contained retrieval-augmented generation (2512.10787).