Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding

Abstract: LLMs often produce content that contradicts or overlooks information provided in the input context, a phenomenon known as faithfulness hallucination. In this paper, we propose Context-Fidelity Boosting (CFB), a lightweight and general decoding-time framework that reduces such hallucinations by increasing the generation probability of source-supported tokens. Motivated by logit-shaping principles from watermarking techniques, CFB applies additive token-level logit adjustments based on a token's degree of support from the input context. Specifically, we develop three boosting strategies: static boosting, which applies a fixed bias to source-supported tokens; context-aware boosting, which scales this bias using the divergence between next-token distributions with and without context; and token-aware boosting, which further redistributes the adaptive bias according to local relevance estimated from source-position attention and source-scoped semantic similarity. CFB requires no retraining or architectural changes, making it compatible with a wide range of LLMs. Experiments on summarization and question answering tasks across multiple open-source LLMs show that CFB consistently improves faithfulness metrics with minimal generation overhead. Our implementation is fully open-sourced.

• The paper introduces a novel, decoding-time framework that enhances context fidelity in LLMs using watermark-inspired logit-shaping.
• It presents three boosting strategies—static, context-aware, and token-aware—that adaptively adjust token probabilities to align generated text with source evidence.
• Empirical results demonstrate improved factual consistency and summarization metrics across both summarization and question-answering tasks using state-of-the-art models.

Context-Fidelity Boosting: Watermark-Inspired Decoding for Faithful Generation

Introduction

LLMs exhibit systematic failures in context-faithful text generation, often producing content that, while factually plausible, contradicts provided evidence—so-called "faithfulness hallucination" is now decoupled from general factuality errors in the literature. The paper "Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding" (2604.22335) presents a formal and scalable solution targeting the context-fidelity issue via a lightweight, general-purpose, decoding-time framework denoted as Context-Fidelity Boosting (CFB). This approach leverages logit-shaping methods originally developed in the LLM watermarking domain but redirects the principle toward boosting source-supported tokens adaptively during token sampling.

Technical Approach

CFB intervenes during generation by adjusting token probabilities through additive logit-shaping—no model retraining or architectural changes are required. The framework comprises three operational modes, each reflecting a trade-off between granularity of adaptation and computational cost.

• Static Boosting: Uniform bias is applied to all tokens appearing in the extractive source context.
• Context-Aware Boosting: Bias magnitude is adaptively scaled based on the Jensen-Shannon divergence between next-token distributions with and without context, quantifying the context's contribution.
• Token-Aware Boosting: The adaptive bias is distributed across source tokens in proportion to their attention-based position relevance and source-scoped semantic similarity with respect to the decoded prefix, using token embeddings for a fine-grained, dynamic control of context reinforcement during decoding.

This hierarchy of adaptation provides both practical speed and metric-optimal context-grounding, depending on the downstream deployment needs.

Figure 1: Illustration of context-faithful decoding, where CFB logit-shaping dynamically aligns token probabilities with the context (e.g., "Paris 2024") against parametric memory preference (e.g., "Tokyo").

Figure 2: The CFB framework with three boosting strategies, ranging from static to token-level adaptive logit-shaping based on attention and semantic relevance.

Empirical Evaluation

CFB is evaluated across both summarization (CNN-DM, XSum) and question answering (NQ-Synth, NQ-Swap) tasks on SOTA open-weight LLMs (Mistral-7B, Llama2-13B, Llama3-8B). Evaluation metrics include ROUGE-L (surface overlap), FactKB (factual consistency), BERT-P (semantic preservation), and accuracy.

Summarization:

CFB (especially Token-Aware CFB) consistently increases FactKB and BERT-P over prior decoding-time faithfulness methods (CAD, ADACAD, COIECD). Gains on ROUGE-L are more model-dependent; however, factual consistency on abstractive, evidence-demanding datasets such as XSum is notably higher.

Figure 3: ROUGE and factual consistency improvement visualization on CNN-DM, demonstrating the practical impact of CFB logit-shaping on summarization.

Question Answering:

On context-complementary QA (NQ-Synth), CFB (especially Token-Aware) yields best-in-class accuracy and ROUGE-L. In knowledge-conflict settings (NQ-Swap), CFB is competitive on faithfulness, but ADACAD—an explicitly conflict-mitigating method—remains optimal in accuracy. Automatic and human evaluations corroborate that CFB enhances faithfulness and informativeness without inducing fluency loss, with lower hallucination and contradiction rates.

Analysis and Ablations

Ablation studies confirm that in Token-Aware CFB, semantic similarity scoring dominates contribution to context alignment, with attention signals adding complementary but not critical value. Removal of adaptive (JSD-based) scaling causes moderate performance degradation, further validating the design.

Attention to the boost value $\delta$ reveals a robust regime: moderate boosts are optimal, while extreme values trade off between under- and over-amplifying context, especially for datasets with strong parametric-context conflict.

Computational analysis establishes that CFB (Static/Context-aware variants) introduce negligible overhead (<0.003% base model FLOPS), with the more elaborate Token-Aware strategy still practically tractable.

Theoretical and Practical Implications

CFB demonstrates that fine-grained, additive logit-shaping at decoding time delivers practically significant gains in context-fidelity without retraining or heavy computation. This supports two broader claims:

• Watermarking-inspired control of decoding offers a modular foundation for a family of reliable, interpretable decoding techniques in LLMs beyond adversarial or post hoc solutions.
• Context-faithful decoding can be separated from factuality improvements, further refining the taxonomy and mitigation of neural "hallucination" phenomena.

In contexts demanding high reliability (e.g., legal, clinical, and financial RAG/QA), the cost-benefit ratio of CFB is especially compelling relative to heavyweight retraining or contrastive decoding paradigms.

Future Directions

CFB exposes the limitations of context-fidelity control in pure black-box LLMs—internals access (logits, attention, embeddings) is required. Addressing black-box or API-only settings might require gradient-free or surrogate-based scoring. Further, the manner of determining context-token relevance remains dependent on preexisting similarity and attention mechanisms; research into robust, context-sensitive relevancy estimation and adaptive context span extraction will bolster performance and robustness in open-world conditions.

Integration with task-specific or reasoning-aware augmentation (e.g., hybrid retrievers, compositional reasoning objectives) may further extend the applicability of logit-shaping to more complex dialog and knowledge discovery tasks.

Conclusion

"Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding" (2604.22335) establishes that decoding-time, watermark-inspired logit-shaping provides a practical, effective, and modular framework for context-fidelity in LLMs. The results redefine the landscape of controlled neural text generation, balancing interpretability, efficiency, and metric performance across models and tasks, and open theoretical and empirical directions for future LLM deployment and research.