Deferred Commitment Decoding (DCD)

• Deferred Commitment Decoding (DCD) is an inference algorithm for discrete diffusion language models that defers low-confidence token commitments using a dynamic, confidence-aware sliding window.
• It tackles limitations of block-based diffusion decoding by mitigating Boundary-Induced Context Truncation and enabling effective bidirectional information flow.
• Empirical evaluations show that DCD improves accuracy on tasks like math problem solving and code synthesis while maintaining inference efficiency comparable to traditional methods.

Deferred Commitment Decoding (DCD) is a training-free inference algorithm for discrete diffusion LLMs (DLMs) that addresses the limitations of block-based diffusion decoding, particularly the problem of Boundary-Induced Context Truncation (BICT), by dynamically deferring low-confidence token commitments within a confidence-aware sliding window. DCD enables effective bidirectional information flow during generation, substantially improving accuracy—especially on tasks requiring precise contextual reasoning—while maintaining comparable inference efficiency to traditional block-based methods (Shu et al., 5 Jan 2026).

1. Motivation and Problem Formulation

Diffusion LLMs generate text by starting from an all-masked sequence and iteratively denoising masked tokens in parallel. For sequence $x = (x_1, \ldots, x_T)$, the state at each timestep is $$x^{(t)} \in (V \cup \{\langle \mathrm{MASK} \rangle\})^T$$, and the reverse process factorizes as

$$p_\theta(x^{(t-1)}|x^{(t)}) = \prod_{i \in \mathcal{M}^{(t)}} p_\theta(x_i \mid x^{(t)})$$

where $\mathcal{M}^{(t)}$ indicates the set of masked positions at step $t$.

For efficient inference and compatible key/value (KV) caching in Transformers, conventional DLMs partition the sequence into $K$ contiguous blocks $\{\mathcal{B}_1, \ldots, \mathcal{B}_K\}$, and decode all tokens in $\mathcal{B}_k$ before moving to $\mathcal{B}_{k+1}$. This block-based approach yields efficient reuse of cache, but imposes a rigid left-to-right (or blockwise) commitment order, causing the BICT issue: tokens at block boundaries must be committed without access to right context, reducing certainty for predictions whose attention horizon ideally extends beyond the current block. Empirical evidence shows diminished performance on tasks such as mathematical problem solving and code generation under this paradigm (Shu et al., 5 Jan 2026).

2. Mathematical Framework of DCD

DCD introduces dynamic, confidence-guided decoding within a sliding window, decoupling the commitment order from the block structure and allowing high-uncertainty tokens to defer until further context is available.

Let $$c_i = \max_{v \in V} p_\theta(x_i = v \mid x^{(t)})$$ denote the confidence score at position $i$. DCD maintains a window %%%%10%%%% over masked tokens, parametrized by initial and maximum size ($s_\mathrm{init}$, $s_\mathrm{max}$). Within this window, a token at $i$ is committed if $c_i \geq \tau_{\text{conf}}$ for a preselected threshold $\tau_{\text{conf}}$. If no token within the window exceeds $\tau_{\text{conf}}$, the highest-confidence token is committed: $$S^{(t)} = \{i \in E^{(t)} \mid c_i \geq \tau_{\text{conf}}\} \cup \left\{ \arg\max_{i \in E^{(t)}} c_i \right\}$$ where $$E^{(t)} = [L^{(t)}, R^{(t)}) \cap \mathcal{M}^{(t)}$$.

The sliding window advances as masked positions are resolved, and the algorithm iterates until all positions are filled. The logic is captured below in pseudocode:

Algorithm 1: Deferred Commitment Decoding (DCD) Input: x^{(t)}_{[l:r]}, DLM p_θ(·|x^{(t)}), window params s_init, s_max; cache type; B', r; τ_conf 1. Initialize L^{(t)} ← l, R^{(t)} ← l + s_init; cd ← 0 2. while M^{(t)} ≠ ∅ do 3. E^{(t)} ← [L^{(t)}, R^{(t)}) ∩ M^{(t)} 4. if cache_type ≠ none and cd ≤ 0 then 5. Refresh KV cache over window 6. cd ← B' 7. end if 8. For each i ∈ E^{(t)} compute c_i ← max_v p_θ(x_i = v | x^{(t)}) 9. S^{(t)} ← {i | c_i ≥ τ_conf} ∪ {arg max_i c_i} 10. Commit x_{S^{(t)}} via Eq.(2) 11. Recompute L, R via (6)–(7) on x^{(t-1)} 12. cd ← cd – |S^{(t)}|; t ← t – 1 13. end while 14. return x^{(t)}_{[l:r]}

3. Algorithmic Design and Implementation

DCD is compatible with both prefix and dual KV-caching strategies as employed in Fast-dLLM. The attention window $$\mathcal{W}^{(t)} = \{ x^{(t)}_i \mid i \in [L^{(t-1)} - r, R^{(t)} + r) \}$$ is maintained, and KV-cache is refreshed every $B'$ committed tokens to maximize cache reuse.

Recommended hyperparameter settings are:

• For full-attention DLMs: $s_\mathrm{init} = 16$, $s_\mathrm{max} = 128$, $\tau_{\text{conf}} = 0.9$, $B' = 32$, $r = 2$.
• For semi-causal DLMs: $s_\mathrm{init}=8$, $s_\mathrm{max} = \infty$, constrained only by block size.
• Experimentally, block size $B=32$ and sub-block size $b=8$ (for Fast-dLLM-v2) were used (Shu et al., 5 Jan 2026).

4. Complexity Analysis

DCD maintains computational complexity and memory requirements essentially identical to block-based decoding: $O(T \cdot d_\mathrm{model} \cdot \text{blocks})$. The additional overhead from maintaining the sliding window and dynamic commitment logic is negligible in practice. Decoding latency is within $±10\%$ of block-based methods across benchmarked settings and sometimes marginally faster due to reduced low-confidence commits.

5. Empirical Evaluation

DCD was evaluated on four pretrained DLMs, under three cache configurations (none, prefix, dual), across five benchmarks covering code synthesis, math, and instruction tasks. The main results are summarized in the table below (accuracy is pass@1 for coding and standard accuracy for math/instruction):

Model	Cache	Decoding	Time(s)	Hum.	MBPP	MATH500	GSM8K	IFEval
LLaDA-8B	dual	block-based	18617	44.5	36.4	36.2	75.7	53.2
LLaDA-8B	dual	DCD	18501	44.5	37.2	39.0	79.2	53.6
Dream-v0-Inst-7B	dual	block-based	9273	56.7	52.8	44.4	74.8	47.7
Dream-v0-Inst-7B	dual	DCD	9284	59.8	58.8	45.2	77.3	56.7
Dream-v0-Base-7B	dual	block-based	10189	57.3	13.4	13.2	73.8	–
Dream-v0-Base-7B	dual	DCD	9406	56.1	13.2	13.2	74.7	–
Fast-dLLM-v2-7B	dual	sub-block	11379	57.9	46.0	52.4	76.0	60.3
Fast-dLLM-v2-7B	dual	DCD	10498	59.1	49.0	51.6	77.8	60.8

Key empirical findings:

• DCD yields an average accuracy improvement of +1.39% over block/sub-block baselines, with decoding time within ±4.4%.
• The maximum observed improvement is +9.0% (MBPP, Dream-v0-Instruct-7B, dual cache).
• Decoding latency remains comparable, sometimes slightly faster due to fewer low-confidence steps (Shu et al., 5 Jan 2026).

6. Discussion, Limitations, and Prospective Directions

DCD is particularly impactful on full-attention DLMs and tasks demanding extensive local context integration, such as reasoning and code synthesis. For semi-causal DLMs, DCD's flexibility is constrained to sub-block-level modifications, resulting in smaller gains (e.g., +0.62% for Fast-dLLM-v2), as the underlying block structure remains fixed.

Instances where DCD underperforms relative to block-based decoding are rare and attributed to stochastic decoding intricacies. Areas for further research include adapting semi-causal architectures to better support deferred commitment, learning adaptive thresholds or window sizes per instance, and integrating entropy-based deferral metrics for more robust uncertainty modeling.

DCD demonstrates that deferring token commitments based on confidence is a principled and effective technique to enhance the quality and maintain the efficiency of diffusion LLM decoding (Shu et al., 5 Jan 2026).