Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generation

Abstract: Retrieval-augmented generation (RAG) increasingly requires models to answer questions from multiple retrieved documents, where only some sources are relevant and the retrieved bundle may contain stale, noisy, or conflicting evidence. Existing contrastive decoding methods primarily focus on resolving conflicts between the model's internal memory and the retrieved context. In contrast, we study the complementary problem of intra-context conflict in multi-document RAG. To evaluate this setting, we introduce DRQA, a factual-conflict question answering benchmark derived from enterprise deep-research scenarios, where answers are grounded in synthetic enterprise-specific facts that are designed not to be recoverable from the model's internal memory. We further propose Dual-Confidence Contrastive Decoding (DCCD), a training-free decoding method that combines document-level confidence, which estimates whether a document appears sufficient for answering the question, with token-level confidence, which estimates whether that document supports a confident next-token prediction. DCCD selects positive and negative document-conditioned streams using these dual-confidence signals and scales a document-level contrast by their confidence margin. Across DRQA and standard multi-document QA benchmarks, DCCD achieves the best average performance among full-context and contrastive decoding baselines, with the largest gains on DRQA. These results highlight the importance of source-aware, confidence-gated decoding when retrieved evidence is internally conflicting.

• The paper introduces DCCD, a decoding algorithm that uses dual confidence scores at the document and token levels to resolve retrieval conflicts.
• It demonstrates significant improvements over baselines, particularly on the DRQA benchmark with accuracy gains exceeding 4 points.
• DCCD’s training-free design and fine-grained confidence calibration offer scalable solutions for noisy, multi-document retrieval settings.

Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generation

Introduction

The proliferation of Retrieval-Augmented Generation (RAG) paradigms for LLMs has raised the complexity of inference, especially in scenarios where retrieved bundles are rife with intra-contextual conflicts, including misinformation, temporal inconsistencies, and noise. While contrastive decoding techniques effectively arbitrate between model-internal and context-based evidence, prior work largely ignores fine-grained evidence selection among retrieved sources during generation. This paper introduces Dual-Confidence Contrastive Decoding (DCCD), a decoding-time algorithm that resolves such conflicts by leveraging combined document-level and token-level confidence estimates to guide evidence selection at every generation step. The work also contributes DRQA, a factual-conflict QA benchmark in enterprise deep-research settings, engineered for scenarios in which the LLM's parametric memory is uninformative and precise retrieval-grounded reasoning is necessary (2607.00570).

Dual-Confidence Contrastive Decoding: Formulation and Implementation

DCCD operates in a multi-document setting: given a query and a pool of retrieved documents, it computes a document-level answerability confidence and a dynamic, prefix-sensitive token-level confidence for each document. Document-level confidence is obtained as the model's predicted probability that a document suffices to answer the query—a quantity derived from a single, batched support-probe evaluation for each document. Token-level confidence is calculated via a Dirichlet uncertainty framework over the top-$k$ logits at each generation step, capturing the local sharpness of the next-token distribution under document conditioning.

At each token step, the aggregate dual-confidence score (sum of normalized document- and token-level confidences) is used to select the most plausible evidence source (positive stream) and the least plausible (negative stream). The difference (margin) between these scores modulates the strength of DCCD's document-level contrast—the positive stream's logits are amplified while negatives are suppressed. The final decoding distribution is a sum of the full-context logits and a confidence-adaptive contrastive adjustment. This design introduces source-aware, confidence-adaptive contrast at the document level, enabling robust reasoning under conflict-laden retrieval (Figure 1).

Figure 1: Overview of Dual-Confidence Contrastive Decoding (DCCD). Given a query $x$ and retrieved documents $\{d_1,\ldots,d_n\}$, $z^i$ are document-conditioned next-token logits; dual confidence guides dynamic source selection and the contrast margin $\epsilon_t$.

The DRQA Benchmark: Isolating Retrieval-Grounded Conflict Resolution

DRQA is derived from DRBench and features 568 synthetic, private enterprise facts, each mapped to a structured QA task. Each example provides a retrieved bundle including one correct document, plausible but incorrect misinformation, temporally stale but contextually valid documents, and unrelated noise. The design ensures that the correct answer is unrecoverable from the model’s parametric knowledge and must be directly retrieved and adjudicated among distractors. Construction incorporates fine-grained controls over document plausibility and style, with document and conflict-type distinctions rigorously validated via LLM-based rubric checks. This dataset enables evaluation specific to the challenge of retrieval-grounded intra-context conflict resolution, a setting distinct from public benchmarks where parametric knowledge can act as a prior or backstop.

Experimental Results

DCCD was evaluated on DRQA as well as standard QA datasets (Natural Questions, TriviaQA, RetrievalQA, PopQA) using the Qwen3.5-2B, Qwen3.5-9B, and Phi-3-medium models, comparing against strong full-context and contrastive decoding baselines: CAD, AdaCAD, DVD, and CoCoA. Across all settings, DCCD achieved the best average performance, with the most pronounced relative improvement on DRQA.

Notable claims supported by experimental results include:

• Zero-shot performance on DRQA is $0\%$ irrespective of model size, underscoring that internal knowledge is inherently insufficient on this benchmark.
• DCCD outperforms all baselines on DRQA by absolute margins exceeding $4$ accuracy points (@5), and produces consistent, though smaller, improvements across other multi-document QA datasets.
• In ablation, dual-confidence is necessary: the removal of document-level confidence sharply degrades performance in conflict-heavy settings, while token-confidence alone lacks robustness in deeper bundles.

Qualitative inspection reveals that DCCD is consistently able to extract correct, context-grounded answers, while all baseline approaches frequently produce uncertain or contradictory outputs—particularly when the retrieved context includes realistic and unstated internal conflict.

Implications

The theoretical contribution of DCCD lies in operationalizing confidence-induced evidence arbitration at decoding-time without task or domain-specific fine-tuning. This yields several implications:

• Practical: DCCD offers a training-free, broadly-applicable solution to enterprise and web-scale RAG deployments, where retrieved bundles are often noisy, mismatched, or deliberately adversarial. The independence from finetuning is valuable in regulated or highly dynamic settings.
• Theoretical: The approach of dual-gated contrast augments the standard contrastive decoding paradigm by explicitly modeling intra-context discrimination, rather than assuming the retrieved context is monolithic or uniformly trustworthy.
• Future-Work Potential: The paper suggests avenues such as learned calibration of source and token confidence, asynchronous or sparse application for runtime reduction, and extensions to multi-modal and long-form deep-research tasks. In high-recall retrieval settings, further research into confidence-based document pruning is critical for amortizing inference cost.

Conclusion

Dual-Confidence Contrastive Decoding (DCCD) establishes a rigorous and scalable framework for retrieval-grounded intra-context arbitration within RAG QA generation. By unifying document-level and token-level source confidence, DCCD produces state-of-the-art results on the DRQA benchmark and consistently improves on established multi-document QA tasks, particularly in settings where the only reliable signal is present in the retrieved evidence. The algorithm's principled, training-free design links uncertainty estimation with contrastive signal gating, paving the way for robust, source-aware generation in domains where retrieval is necessary but unreliable by default (2607.00570).