Adaptive Path-Contrastive Decoding (APCD)

• Adaptive Path-Contrastive Decoding (APCD) is a novel inference-time strategy that enhances LLM reliability by exploring multiple decoding paths to mitigate hallucinations.
• It employs Entropy-Driven Path Expansion to delay branching until decision uncertainty peaks and uses Divergence-Aware Path Contrast to maintain useful diversity among candidates.
• Empirical evaluations show that APCD yields improved factual accuracy and efficiency compared to beam search, with controlled complexity and significant performance gains.

Adaptive Path-Contrastive Decoding (APCD) is a training-free, inference-time multi-path decoding framework for reliable LLM generation. It is designed for settings in which hallucinations arise from error accumulation in autoregressive decoding: an early suboptimal token choice can redirect the entire continuation into a self-reinforcing but incorrect trajectory. APCD addresses this failure mode by combining Entropy-Driven Path Expansion (EDPE), which delays branching until multiple continuations are genuinely plausible, with Divergence-Aware Path Contrast (DAPC), which encourages useful path diversity while attenuating inter-path interaction as the paths become sufficiently different. After generation, APCD selects the path with the lowest perplexity as the final answer (Zheng et al., 10 May 2026).

1. Conceptual basis and problem formulation

APCD begins from the claim that hallucination is not only a modeling problem but also a decoding problem. Standard left-to-right generation commits to a single path, so once decoding enters a bad local optimum, subsequent conditionals are computed under a flawed prefix and may amplify the original mistake. The paper therefore treats reliability as a search problem over alternative trajectories rather than as a property of a single greedy continuation (Zheng et al., 10 May 2026).

The underlying autoregressive formulation is standard. For a query $q$ and trajectory $y=(y_1,\dots,y_L)$,

$$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$

At step $t$, the model emits logits $z_t \in \mathbb{R}^M$ over vocabulary $V$ of size $M$, with token probability

$$P_k = P(y_t = k \mid y_{<t}, q) = \frac{e^{z_{t,k}}}{\sum_{j=1}^{M} e^{z_{t,j}}}.$$

The framework is motivated by three questions. The first is when branching should occur: branching too early wastes computation on low-probability noise. The second is how to maintain diversity without the mode collapse characteristic of ordinary beam search. The third is how to regulate interaction among paths so that diversity is encouraged when useful but not over-enforced once paths have already separated. APCD answers these questions with delayed branching and divergence-regulated contrast.

A plausible implication is that APCD should be understood less as a generic diversity heuristic than as a controlled trajectory-management scheme: it branches only at uncertainty spikes and treats contrast as a transient coordination mechanism rather than a permanent penalty.

2. Entropy-Driven Path Expansion

EDPE determines when a single decoding path should split into multiple paths. Instead of maintaining several beams from the first step, APCD monitors uncertainty in the current next-token distribution and branches only when the model reaches what the paper characterizes as a meaningful decision point (Zheng et al., 10 May 2026).

The uncertainty signal is Shannon entropy over the top-$k$ candidate tokens, not over the full vocabulary. If the top-$k$ probabilities are renormalized as

$y=(y_1,\dots,y_L)$0

then APCD computes

$y=(y_1,\dots,y_L)$1

Branching is triggered when

$y=(y_1,\dots,y_L)$2

where $y=(y_1,\dots,y_L)$3 is a predefined entropy threshold. Low $y=(y_1,\dots,y_L)$4 indicates that one token dominates and that branching is likely unproductive; high $y=(y_1,\dots,y_L)$5 indicates several comparably plausible continuations.

Once the threshold is crossed, APCD initializes $y=(y_1,\dots,y_L)$6 paths from the current top-$y=(y_1,\dots,y_L)$7 tokens. EDPE therefore performs delayed expansion rather than continuous multi-path decoding. Algorithmically, the framework first runs as a single-path decoder, repeatedly computing logits, extracting the top-$y=(y_1,\dots,y_L)$8 set, renormalizing those probabilities, and checking normalized entropy. The branch point is the first step at which the entropy criterion is met.

For long-form generation, the reported default hyperparameters are $y=(y_1,\dots,y_L)$9 and $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$0. In short-text settings, the paper uses model- and dataset-specific settings: for TruthfulQA, Llama-2-7b-chat uses $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$1, Llama-2-13b-chat uses $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$2, and Llama-3-8b-instruct uses $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$3; for TriviaQA, HotPotQA, and NQ, the reported setting is $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$4. The appendix summary further states that smaller or weaker models benefit from more paths and lower entropy thresholds, while stronger models tend to prefer fewer paths and higher thresholds.

This component is the main answer to the criticism that ordinary multi-path decoding branches indiscriminately. APCD treats branching as an uncertainty-conditioned event rather than as a fixed decoding mode.

3. Divergence-Aware Path Contrast

After expansion, APCD enters a multi-path regime governed by DAPC. Its objective is to keep the paths from collapsing into near-duplicates while avoiding the rigidity of fixed diversity penalties such as those used in Diverse Beam Search. The central mechanism is pairwise contrastive logit adjustment whose strength decays with distributional divergence (Zheng et al., 10 May 2026).

For path $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$5, APCD updates the logit of candidate token $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$6 as

$$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$7

The update is applied only over a valid token subset,

$$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$8

with $$P(y \mid q) = \prod_{t=1}^{L} P(y_t \mid y_{<t}, q).$$9. In the reported experiments, $t$0. This restricts contrast to near-top candidates rather than the full vocabulary.

The contrast weights are computed from Jensen-Shannon divergence: $t$1 Here, $t$2 is the maximum pairwise contrast strength and $t$3 is the divergence threshold. If two path distributions remain similar, $t$4 is small and the contrast weight remains high. As the paths diverge, the weight decreases. Once normalized JSD exceeds $t$5, the weight becomes zero and contrast between that pair terminates.

This dynamic attenuation is APCD’s distinguishing feature. It differs from fixed diversity promotion in two ways. First, it uses actual path distribution divergence rather than overlap heuristics. Second, it stops interfering once diversity has already emerged. The result is not simply “push beams apart,” but “push beams apart while they are insufficiently distinct.”

A plausible implication is that DAPC defines a middle ground between ordinary beam search and permanently contrastive decoding. It supplies cross-path regularization only while it remains informative.

4. Inference procedure, pruning, and computational profile

The full APCD procedure has two phases. In the single-path phase, EDPE monitors top-$t$6 entropy and continues greedy decoding until the entropy threshold is reached. In the multi-path phase, APCD replicates the KV cache and text state across $t$7 paths, seeds each path with one of the top-$t$8 candidate tokens from the expansion point, and then applies DAPC at each subsequent step (Zheng et al., 10 May 2026).

Path quality is tracked with perplexity,

$t$9

Perplexity is used in two places. During decoding, paths whose perplexity exceeds a threshold are pruned; in long-text experiments the reported threshold is $z_t \in \mathbb{R}^M$0. After decoding, APCD returns the path with minimum perplexity. The paper uses lowest perplexity as a proxy for coherence and model confidence.

The framework adds overhead relative to greedy decoding through entropy computation, maintenance of multiple paths, pairwise JSD computations, and contrastive logit updates. Its complexity in the multi-path regime is roughly $z_t \in \mathbb{R}^M$1 in path-pair interactions per step, although the effective cost is reduced by vocabulary restriction to $z_t \in \mathbb{R}^M$2, path pruning, and the fact that many $z_t \in \mathbb{R}^M$3 weights eventually become zero.

The implementation details reported in the paper include KV-cache replication at expansion time, greedy token selection after logit update, maximum generation lengths of 4096 tokens for long-form generation and 100 tokens for short-text generation, and experiments run on four NVIDIA Tesla V100 32GB GPUs.

Latency results on TriviaQA show that APCD is close to beam search and substantially faster than Diverse Beam Search. For Llama-2-7b-chat, the reported average latency is 21.8 ms/token for greedy decoding, 27.0 for beam search, 64.3 for DBS, and 26.9 for APCD. For Llama-2-13b-chat, the corresponding numbers are 43.5, 52.3, 90.0, and 50.1. For Llama-3-8b-it, they are 24.2, 34.2, 69.1, and 35.9. The paper summarizes APCD as only 5.1–11.7 ms/token slower than baseline and 44–58% faster than DBS.

5. Empirical performance and ablation evidence

APCD is evaluated on eight benchmarks: NEJM, MedMCQA, MedQA, MedB-4, TruthfulQA, TriviaQA, HotPotQA, and Natural Questions. The model set comprises LLaMA-2-7B-Chat, LLaMA-2-13B-Chat, LLaMA-3-8B-Instruct, and II-Medical-8B-1706. Reported metrics are %Truth, %Info, %T$z_t \in \mathbb{R}^M$4I, and %Reject for TruthfulQA; Exact Match and F1 for TriviaQA, HotPotQA, and NQ; and Strict Match and Fuzzy Match for the medical datasets (Zheng et al., 10 May 2026).

On TruthfulQA, APCD improves the joint truthfulness-and-informativeness measure while reducing rejection. For Llama-2-7b-chat, %T$z_t \in \mathbb{R}^M$5I rises from 39.5 to 45.9 and %Reject falls from 19.5 to 12.9. For Llama-2-13b-chat, %T$z_t \in \mathbb{R}^M$6I rises from 44.0 to 49.0 and %Reject falls from 15.8 to 11.0. For Llama-3-8b-instruct, %T$z_t \in \mathbb{R}^M$7I rises from 39.8 to 49.8 and %Reject falls from 19.9 to 17.2.

On factual QA benchmarks, the paper reports consistent gains over baseline. For Llama-2-7b-chat, TriviaQA improves from 58.2 to 61.0 EM and from 54.8 to 59.4 F1; HotPotQA improves from 25.5 to 28.4 EM and from 23.4 to 29.4 F1; NQ improves from 34.2 to 36.0 EM and from 31.2 to 34.0 F1. Similar gains are reported for the larger chat and instruct models.

On II-Medical-8B-1706, APCD outperforms all baselines across NEJM, MedMCQA, MedQA, and MedB-4. The reported improvements over baseline are 67.7 to 76.6 SM and 71.5 to 78.0 FM on NEJM, 67.9 to 75.3 SM and 71.0 to 76.9 FM on MedMCQA, 79.2 to 85.0 SM and 82.8 to 86.7 FM on MedQA, and 68.4 to 76.0 SM and 75.6 to 78.3 FM on MedB-4.

The ablations clarify the relative contribution of the two APCD components. Removing EDPE by setting $z_t \in \mathbb{R}^M$8, which causes immediate branching, produces a substantial drop: on MedMCQA, SM falls from 75.3 to 61.6 and FM from 76.9 to 70.3. Removing DAPC by setting $z_t \in \mathbb{R}^M$9 yields a smaller but still consistent loss: on MedMCQA, SM falls from 75.3 to 73.9 and FM from 76.9 to 75.1. The full system performs best, indicating that EDPE provides the dominant structural gain while DAPC adds further reliability.

Hyperparameter analysis on Llama-3-8b-instruct for TruthfulQA reports the best setting at $V$0 and $V$1. The appendix summarizes a practical range of $V$2 between 2 and 5 and $V$3 between 0.8 and 0.95 in typical settings. The branch-timing analysis further shows low entropy early in generation and a later rise at reasoning decision points, which the paper uses to justify delayed branching.

6. Position within contrastive decoding research and major limitations

APCD belongs to the broader contrastive-decoding family but differs from earlier methods in where contrast is applied and how adaptivity is defined. Standard Contrastive Decoding uses expert–amateur model differences plus a plausibility constraint and was originally formulated as a constrained contrastive objective over continuations (Li et al., 2022). Later work showed that such token-level contrast can improve reasoning trajectories, but it remained locally stepwise and used fixed contrast weights (O'Brien et al., 2023). APCD instead performs cross-path contrast within a single model, after uncertainty-triggered branching, and attenuates that contrast as trajectories separate (Zheng et al., 10 May 2026).

This distinction is especially important when APCD is compared with adaptive but non-path-based methods. Adaptive Contrastive Search modulates $V$4 and $V$5 by token-level uncertainty, but it still reranks immediate next-token candidates rather than competing paths (Arias et al., 2024). PromptCD uses synchronized positive and negative prompts and a fixed per-step contrastive rule with a dynamic plausibility mask, yet its decoding remains single-track rather than path-branch adaptive (Bi et al., 24 Feb 2026). AdaCD, despite the name “Adaptive Contrastive Decoding,” is a token-level safety method that switches between adding and subtracting a refusal distribution for the first $V$6 tokens and does not define path-level contrast or trajectory search (Qi et al., 18 Apr 2026). ARR in conflict-aware decoding routes between interpolation and extrapolation over prior and context distributions at each step, which is adaptive in authority allocation, but it is still step-local rather than a multi-path framework (Jiang et al., 9 Jun 2026).

A recurrent misconception is therefore that APCD is interchangeable with other adaptive contrastive methods. The literature indicates otherwise. APCD is specifically a multi-path framework with entropy-triggered expansion, cross-path contrast, divergence-aware attenuation, path pruning, and final path selection by perplexity. That combination distinguishes it from token-local adaptive decoders.

The paper also states clear limitations. APCD relies on sufficient reasoning length: EDPE needs enough steps for entropy to evolve meaningfully, and DAPC needs multiple steps for divergence-aware weights to matter. It is therefore less suitable for ultra-short text scenarios. Its behavior depends on hyperparameters such as $V$7, $V$8, $V$9, and $M$0. If branching occurs too early, noisy paths can be introduced; if too late, useful alternatives may be missed. Perplexity is only a proxy for correctness, so selecting the lowest-perplexity path does not guarantee factual accuracy in every case. The paper identifies tighter integration with hidden states across layers as a promising future direction (Zheng et al., 10 May 2026).