Influence-Aware Contrastive Decoding

• Influence-aware contrastive decoding is an inference method that contrasts a primary predictive distribution with reference signals to dynamically regulate the influence of diverse information sources on token selection.
• It employs mechanisms such as entropy-adaptive interpolation, context perturbation, and attention modulation to improve reasoning, reduce bias, and mitigate the effects of irrelevant or noisy input.
• Empirical evaluations in retrieval-augmented, long-context, and multimodal generation demonstrate its effectiveness in enhancing factuality, contextual grounding, and overall model robustness.

“Influence-aware contrastive decoding” (Editor’s term) can denote a family of inference-time decoding methods that modify next-token selection by contrasting a primary predictive distribution with one or more reference distributions chosen to expose the effect of particular information sources—parametric memory, retrieved passages, irrelevant or adversarial context, positional access, multimodal evidence, or role-labeled prompt segments. In this framing, contrast is not only a device for improving generation quality; it is a mechanism for estimating and regulating how strongly a source is allowed to reshape token probabilities. The lineage runs from expert–amateur contrastive decoding for open-ended text generation (Li et al., 2022) to retrieval-grounded, perturbation-based, attention-steered, and hierarchy-sensitive variants that operationalize “influence” through uncertainty reduction, context perturbation, attention allocation, or role-ablation divergence (Kim et al., 2024).

1. Foundational formulation and historical development

The foundational formulation is “Contrastive Decoding: Open-ended Text Generation as Optimization” (Li et al., 2022). It defines generation as maximization of a contrastive objective between a large expert LLM and a smaller amateur model, subject to an adaptive plausibility constraint. At the token level, admissible candidates are restricted to

$$V (x_{<i}) = \{x_i \in \mathcal{V}: exp(x_i \mid x_{<i}) \geq \alpha \max_w exp(w \mid x_{<i}) \},$$

and then scored by

$\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$

This established the core architectural template: a high-capability distribution supplies plausibility, a contrastive reference suppresses undesirable modes, and decoding proceeds by token-level search rather than post hoc reranking (Li et al., 2022).

Subsequent work clarified that the reference branch need not be a smaller model. “Contrastive Decoding Improves Reasoning in LLMs” reformulated the score as

$$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$

and showed that this decoding rule improves chain-of-thought reasoning while reducing prompt-token copying, although it can increase arithmetic slip-ups (O'Brien et al., 2023). “Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM” then reinterpreted CD as linear extrapolation toward a hypothetical larger model and identified a specific failure mode—“obvious blindness,” where tokens already assigned high probability by the amateur can be over-penalized (Chang et al., 2024). This suggests that later “influence-aware” variants are best understood not as abandoning contrastive decoding, but as refining what the negative signal should represent and how strongly it should act.

2. Canonical mathematical patterns

Across the literature, the shared mathematical pattern is a base next-token score plus a signed contrastive displacement. In the original expert–amateur setting this displacement is model-scale disagreement, but later work replaces it with source-specific disagreements.

In retrieval-grounded question answering, “Enhancing Contextual Understanding in LLMs through Contrastive Decoding” uses three branches: no context, relevant context $c^+$, and irrelevant context $c^-$. Its token rule is

$$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$

or, equivalently,

$$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$

This formulation makes the contrast source-sensitive: a token is preferred when it is supported by relevant evidence and not supported by irrelevant evidence (Zhao et al., 2024).

In retrieval-augmented generation with a single retrieved passage $c$, “Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts” rewrites context amplification as

$$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$

or

$$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$

Here $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$0 is the contextual effect on the model’s next-token beliefs, and $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$1 directly controls how far decoding moves from the closed-book prior toward the context-conditioned distribution (Kim et al., 2024).

Other domains instantiate the same pattern with different reference conditions. “Surfacing Biases in LLMs using Contrastive Input Decoding” defines

$\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$2

and decodes from

$\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$3

so that generation follows the direction induced by an input perturbation (Yona et al., 2023). “Mitigating the Influence of Distractor Tasks in LMs with Prior-Aware Decoding” derives

$\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$4

where $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$5 is estimated from a weakened prompt intended to expose distractor-task or strong-local-prior behavior (Douglas et al., 2024). “Anti-LM Decoding for Zero-shot In-context Machine Translation” uses an anti-model term $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$6 from the source sentence alone, with a timestep-dependent decay $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$7, to penalize source-language continuation priors (Sia et al., 2023). Collectively, these formulations show that the contrastive branch can be induced by model size, context ablation, relevance polarity, perturbation, or prior stripping.

3. Estimating and modulating influence

The decisive methodological question is how the decoder estimates whether a contrastive signal should be trusted. Different papers instantiate this through uncertainty, confidence, perturbation sensitivity, attention, or divergence.

Mechanism	Influence proxy	Representative paper
Entropy-adaptive interpolation	$\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$8 vs. $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$9	(Kim et al., 2024)
Confidence-gated relevant/irrelevant contrast	$$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$0 and $$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$1 from max token probability	(Zhao et al., 2024)
Prompt weakening / prior stripping	$$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$2	(Douglas et al., 2024)
Input perturbation differential	$$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$3	(Yona et al., 2023)
Role-ablation divergence	$$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$4	(Liu et al., 29 Jun 2026)

ACD is the clearest direct instance of online modulation. It defines token-level entropy under the no-context and with-context distributions and sets

$$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$5

If conditioning on $$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$6 lowers entropy, $$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$7 approaches $$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$8; if it raises entropy, $$s_{CD}^{(i)}= \begin{cases} (1+\beta)s_e^{(i)}-\beta s_a^{(i)} & i\in V_{valid}\ -\infty & i\notin V_{valid}, \end{cases}$$9 moves toward $c^+$0. The resulting decoder is

$c^+$1

with greedy selection over the full vocabulary and no extra learned gate or relevance classifier (Kim et al., 2024). The paper explicitly interprets this as uncertainty-aware regulation of contextual influence rather than formal causal attribution.

The multi-input QA method in (Zhao et al., 2024) uses a different adaptive rule. It computes

$c^+$2

and sets

$c^+$3

This is a confidence-sensitive gate over the relative impact of parametric priors and relevant evidence (Zhao et al., 2024).

A stronger notion of source influence appears in IHDec for multi-turn instruction hierarchies. It defines role influence by

$c^+$4

declares a hierarchy violation when $c^+$5 for a lower-priority role $c^+$6, constructs

$c^+$7

and then injects a normalized contrastive steering vector into the raw logits: $c^+$8 This moves influence-aware contrastive decoding from heuristic confidence gating toward explicit source-ablation diagnostics (Liu et al., 29 Jun 2026).

4. Domain-specific instantiations

In retrieval-augmented generation, the central issue is balancing parametric knowledge against non-parametric evidence. The negative result motivating this line is that fixed context amplification can become brittle when the retrieved passage is irrelevant, incomplete, or misleading. The positive-context/irrelevant-context decoder in (Zhao et al., 2024) and the entropy-adaptive ACD rule in (Kim et al., 2024) are both designed for open-domain QA, but they differ in what they treat as the nuisance signal: irrelevant retrieved evidence in the former, and uncertainty-increasing contextual shift in the latter.

In long-context decoding, the operative source of influence is positional access. “Mitigating Posterior Salience Attenuation in Long-Context LLMs with Positional Contrastive Decoding” introduces Posterior Salience Attenuation (PSA), observes that the gold token often remains among the very top posterior candidates even when long-context performance degrades, and contrasts a standard long-aware branch with a local-aware branch produced by over-rotating low-frequency RoPE components. The combined logits are

$c^+$9

This can be read as a positional-influence-aware decoder: the contrast isolates what the standard branch gains from long-range positional awareness relative to a deliberately more local reference (Xiao et al., 10 Jun 2025).

In factuality-oriented decoding, LayerCake localizes influence inside the model stack. It identifies punctuation tokens as dominant in early layers and conceptual tokens as dominant in intermediate layers, suppresses their attention selectively, and constructs contrastive scores from the original and degraded branches. The paper reports stage-specific attention suppression rules such as

$c^-$0

for punctuation in early layers, and then averages punctuation- and concept-specific contrastive scores for final decoding (Zhu et al., 6 Jul 2025). This suggests a finer-grained notion of influence tied to token type $c^-$1 layer interactions rather than to whole prompts or whole documents.

Multimodal variants generalize the same pattern. ASCD argues that the effectiveness of prior visual and instruction contrastive decoding stems from altered internal attention allocation, then directly steers text-centric heads and critical visual tokens before fusing positive and negative branches as

$c^-$2

with a plausibility-style cutoff on the positive branch (Wang et al., 17 Jun 2025). AVCD estimates modality dominance from attention mass,

$c^-$3

masks high-attention tokens in less dominant modalities, and combines intact and perturbed audio-visual branches through a trimodal logit formula (Jung et al., 27 May 2025). SafeCoDe contrasts real and Gaussian-noised images,

$c^-$4

then modulates refusal-token logits according to a global safe/unsafe verdict (Liu et al., 23 Sep 2025). CHASD gates localized visual perturbation by uncertainty, activating the negative branch only when the maximum next-token probability falls below a threshold, and perturbs only the top-$c^-$5 salient visual tokens identified from cross-attention (Huang et al., 22 May 2026). These methods collectively indicate that “influence” in multimodal contrastive decoding is often carried by attention pathways, salient regions, or perturbation sensitivity rather than by explicit symbolic relevance labels.

5. Empirical behavior under noise, conflict, and bias

The most detailed empirical validation of influence-aware decoding in the supplied literature is ACD. On TriviaQA, Natural Questions, and PopQA with top-1 retrieved context from CONTRIEVER-MSMARCO and 5-shot prompting, ACD yields the best exact match on the full test set for LLAMA2-7B: TriviaQA $c^-$6, NQ $c^-$7, and PopQA $c^-$8, compared with MICD$c^-$9 at $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$0, $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$1, and $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$2, MICD$$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$3 at $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$4, $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$5, and $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$6, CAD at $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$7, $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$8, and $$y_t \sim \mathrm{softmax}(z_t + \alpha (z_t^+ - z_t^-)),$$9, and closed-book decoding at $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$0, $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$1, and $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$2. On the noisy subset, ACD remains especially strong: TriviaQA $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$3, NQ $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$4, and PopQA $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$5. The paper also shows that its adaptive coefficient tracks context quality better than MICD$$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$6: for LLAMA2-7B on the “First” AUROC metric, ACD reaches $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$7 versus $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$8 on NQ, $$y_t \sim p_\theta(y_t|x, y_{<t}) \left( \frac{p_\theta(y_t|c^+, x, y_{<t})} {p_\theta(y_t|c^-, x, y_{<t})} \right)^\alpha .$$9 versus $c$0 on TriviaQA, and $c$1 versus $c$2 on PopQA (Kim et al., 2024).

The positive-context/irrelevant-context decoder in (Zhao et al., 2024) exhibits a related robustness pattern. With Llama-2 70B, exact match improves from regular open-book decoding to the proposed method on NQ from $c$3 to $c$4 in the fixed setting, on TriviaQA from $c$5 to $c$6 with dynamic $c$7, and on PopQA from $c$8 to $c$9 with dynamic $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$0. On NQ-SUB, where retrieved context deliberately contradicts parametric memory, the method also outperforms CAD and regular open-book decoding, indicating that its contrastive signal is not simply anti-context but source-selective (Zhao et al., 2024).

Outside RAG, the same pattern appears under different operational definitions of influence. PCD improves Llama-3-8B (262k) on InfiniteBench KV Retrieval from $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$1 at $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$2 to $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$3, and on RULER Variable Tracking from $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$4 to $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$5, supporting the claim that contrasting long-aware and local-aware positional branches can alleviate PSA (Xiao et al., 10 Jun 2025). LayerCake improves LLaMA-2-7B on TruthfulQA from $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$6 to $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$7 on MC1/MC2/MC3, HellaSwag from $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$8 to $$P_\theta(Y_t \mid x, y_{<t}) = \operatorname{softmax}\!\left(z_t + \alpha (z_t^c - z_t)\right)$$9, StrategyQA from $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$0 to $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$1, and FACTOR from $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$2 to $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$3, consistent with the claim that token-type × layer interventions can construct a useful degraded comparator (Zhu et al., 6 Jul 2025).

Multimodal safety and hallucination studies reinforce the same theme. SafeCoDe raises Qwen-VL-7B-Instruct on MSSBench overall average from $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$4 to $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$5 while reducing MOSSBench average rejection from $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$6 to $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$7, and raises LLaVA-1.6-7B on MSSBench from $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$8 to $$\operatorname{softmax}\!\left((1-\alpha) z_t + \alpha z_t^c\right).$$9 while lowering MOSSBench average rejection from $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$00 to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$01 (Liu et al., 23 Sep 2025). CHASD improves LLaVA-1.5 on POPE adversarial accuracy to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$02 and F1 to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$03, AMBER to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$04, and MMHal-Bench to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$05, while retaining competitive inference efficiency relative to always-on contrastive baselines (Huang et al., 22 May 2026). IHDec raises Llama-3.1-8B-Instruct on IHEval Rule-Following conflict average from $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$06 to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$07 and on Safety Defense conflict average from $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$08 to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$09, while MT-Bench-101 changes only from $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$10 to $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$11, indicating that token-level hierarchy steering can materially improve multi-turn conflict robustness without major general-quality loss (Liu et al., 29 Jun 2026).

6. Limitations, misconceptions, and open directions

A recurring misconception is that these methods compute “influence” in a formal interpretability sense. Most do not. ACD explicitly does not measure causal influence, document attribution, or token-level relevance maps; it uses entropy reduction as a coarse proxy for whether context is clarifying the prediction (Kim et al., 2024). LayerCake infers influence from attention concentration plus performance degradation under attention suppression, not from gradients or mediation analysis (Zhu et al., 6 Jul 2025). ASCD and AVCD likewise operationalize influence through attention heads, salient tokens, or modality-dominance statistics rather than through explicit causal decomposition (Wang et al., 17 Jun 2025). SafeCoDe uses response sensitivity to a Gaussian-noised image as a practical visual influence proxy, not a principled attribution measure (Liu et al., 23 Sep 2025). CHASD uses max-probability gating and cross-attention saliency, which means high-confidence hallucinations can bypass calibration altogether (Huang et al., 22 May 2026).

A second limitation is compute and access. ACD and CAD require roughly twice the inference cost of conventional greedy decoding, while MICD requires roughly three times the cost (Kim et al., 2024). The relevant/irrelevant-context decoder is about three times slower than regular decoding because it runs the same LLM three times per step (Zhao et al., 2024). PCD uses two forward branches with distinct RoPE settings (Xiao et al., 10 Jun 2025). Attention-steered methods such as LayerCake and ASCD require white-box access to internal attention maps, and ASCD is explicitly incompatible with FlashAttention (Wang et al., 17 Jun 2025). IHDec needs batched role-ablated forward passes plus role annotations, and its printed formula for $\score(x_i; x_{<i}) = \begin{cases} \log \frac{exp ( x_i \mid x_{<i})}{ ama( x_i \mid x_{<i})}, & \text{if } x_i \in V(x_{<i}),\ -\inf, & \text{otherwise.} \end{cases}$12 is slightly inconsistent with the surrounding prose, leaving some implementation ambiguity (Liu et al., 29 Jun 2026).

A third limitation concerns calibration of the negative signal itself. APD shows that plain linear expert–amateur subtraction can over-penalize “obvious” high-probability answers already known by the amateur, and proposes asymptotic probability modeling as a remedy (Chang et al., 2024). This suggests that fixed-magnitude negative influence is often too crude. A plausible implication is that future influence-aware decoders will need token-specific or source-specific reliability estimates rather than a single global subtraction coefficient.

The literature therefore points toward several open directions. One is finer decomposition across multiple retrieved passages, spans, or roles rather than binary with/without comparisons; ACD, for example, compares only no-context and top-1-context distributions and does not decompose influence across documents (Kim et al., 2024). Another is richer source-reliability estimation, since current methods rely heavily on entropy, max-probability, attention mass, or perturbation sensitivity. A third is efficiency: multiple branches remain the dominant systems bottleneck across RAG, long-context, and multimodal instantiations. Finally, a more explicit theory of influence-aware decoding would need to connect these practical proxies—uncertainty reduction, role-ablation JSD, attention dominance, and perturbation deltas—to formal causal or attributional notions of source contribution. The existing papers show that such proxies can already improve robustness, factuality, grounding, and hierarchy adherence, but they stop short of a complete theory.