TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

Abstract: Diffusion LLMs (DLLMs) generate text by iteratively denoising masked positions, exposing a trajectory of predictive distributions rather than a single instantaneous belief. Most existing decoders ignore this trajectory and commit tokens from the current snapshot alone, conflating confidence with commitment readiness: a transient top-1 peak under incomplete context can be locked in, while candidates with consistent cross-step support are delayed. We propose Trajectory-Aware Commit Gating (TACG), a training-free gate-level decoder that anchors token identities to the base posterior and uses trajectory-aware signals only to decide whether the current proposal is ready to commit. TACG combines Temporal Implicit Logits Guidance (TILG), which keeps an exponential moving average of past logits as a self-reference and contrasts the current logits against this reference in natural-parameter space, with a History Gate (HG) that enforces short-term proposal persistence before commitment. Together with a capped extra-promotion budget, these components yield a stability-constrained commit rule without auxiliary networks or extra forward passes. We evaluate TACG on LLaDA, Dream, and LLaDA2-Mini across code (HumanEval, MBPP) and math (GSM8K, MATH500) benchmarks; it typically improves or preserves accuracy while reducing denoising steps and increasing tokens per forward (TPF). The code is publicly available at https://github.com/Clarence-CV/TACG-DLLM.

• The paper introduces TACG, a training-free method that separates token identity from commit timing using trajectory-aware signals (TILG and HG).
• It demonstrates enhanced decoding performance, with improvements in accuracy (+4.9% on HumanEval) and token throughput without extra model overhead.
• The approach offers a robust, tunable framework that balances efficiency and quality by harnessing temporal signal stability for early, reliable token commitments.

Trajectory-Aware Commit Gating for Diffusion LLM Decoding (TACG)

Motivation and Context

Diffusion LLMs (DLLMs) perform text generation by reversing a masking process, iteratively denoising and revealing masked tokens. Unlike autoregressive models, which generate a single token per step, DLLMs work in parallel, producing distributions over many masked positions in each step. This exposes a temporal trajectory of beliefs for each position. Current DLLM decoders, however, typically make commitment decisions (i.e., when to unmask a token) based on snapshot confidence, without explicit consideration of these belief trajectories. This often conflates transient confidence spikes with genuine commitment readiness, leading to suboptimal early commitments or unnecessary delays, both degrading efficiency and final sequence quality.

TACG introduces a principled, training-free method to leverage the evolution of the denoising trajectory, providing more robust and efficient commit gating. It identifies and explicitly separates the decision of what token should be written and when the position should leave the masked state, solving a key fragility in practical DLLM decoding.

TACG Framework: Components and Mechanisms

TACG augments the standard base posterior commit strategy (e.g., confidence thresholding) with two trajectory-aware signals:

1. Temporal Implicit Logits Guidance (TILG): Maintains an exponential moving average (EMA) self-reference of historical logits at each position. By contrasting current logits against this EMA in the natural-parameter (logit) space, TILG quantifies whether support for the base proposal is increasing, offering a directional temporal support statistic—distinct from merely sharpened confidence (see Figure 1).
2. History Gate (HG): Enforces short-term persistence of a proposal (i.e., the base token proposal must remain stable for a minimum number of steps) before it is eligible for commitment. This mitigates the risk of committing to transient top-1 spikes due to incomplete context.

When combined, these signals form a robust commit gating rule: positions are eligible for commitment if their proposals are both persistent and exhibit positive temporal support relative to their own belief history. Additionally, a capped "extra promotion" mechanism allows a bounded number of tokens with significant readiness (via combined confidence and temporal support) to be promoted per step, limiting additional risk while maintaining decoder parallelism.

Figure 1: TACG mechanism with a TILG temporal-support branch. The base posterior proposes token identity; TILG contrasts current logits with the EMA reference to score commitment readiness, leaving token selection anchored to the base posterior.

Empirical Analysis of Historical Signals

Extensive diagnostics on GSM8K reveal that:

• History Gate support (length of uninterrupted proposal stability) correlates with increased token-label match, even when confidence is held constant.
• History-logit support (EMA-driven temporal support) identifies tokens in lower-confidence regions already supported by the denoising trajectory, enabling earlier yet reliable commitments.

  Figure 2: Diagnostic historical signals on GSM8K. Higher history gate and history-logit support strongly enhance token-level match, implying both reliable commitment and early token promotion are achievable with trajectory-aware signals.

Experimental Results

Main Evaluation

TACG is integrated into LLaDA, Dream, and LLaDA2-Mini and evaluated on HumanEval, MBPP, GSM8K, and MATH500, under both confidence thresholding and KLASS-based gates. TACG improves or matches accuracy and increases tokens-per-forward (TPF) in nearly all scenarios. For instance, on LLaDA/HumanEval, TACG achieves +4.9% accuracy (42.1% vs. 37.2%) and +0.56 TPF (7.01 vs. 6.45) over the base confidence decoder. KLASS with TACG gains more modestly in accuracy but shows task-dependent TPF changes, validating the complementary nature of trajectory-aware and distribution-level stability signals.

Generalization

Transfer experiments on Dream and adaptation to block-wise LLaDA2-Mini show similar gains, confirming the effectiveness of TACG across architectures and inference paradigms.

Decoding Dynamics

Analysis of per-step decoding reveals that TACG yields a distinct early peak in accelerated token commitments relative to baselines, demonstrating that the integration of TILG and HG not only increases parallelism but also selectively advances stable regions rather than uniformly increasing risk.

Figure 3: Per-step decoding dynamics on HumanEval. Positive values indicate a larger number of tokens decoded per step than the baseline, illustrating TACG's ability to safely accelerate the decoding process.

Sensitivity and Ablations

Comprehensive ablations demonstrate the distinct contributions of TILG (efficiency gains via aggressive yet temporally guided promotion) and HG (reliability through consistency enforcement). Hyperparameter sweeps over the temporal support weight and confidence floor reveal stable performance and tunable accuracy-efficiency tradeoffs.

Figure 4: Sensitivity to support weight $\lambda$ and confidence floor on HumanEval. TACG remains robust under a wide range of parameter settings, balancing quality and efficiency.

The method demonstrates resilience to historical-memory window and EMA decay settings, with moderate windows and higher decay rates yielding the most favorable tradeoff between accuracy and token throughput.

Theoretical and Practical Implications

TACG reformulates DLLM decoding as a joint stopping-and-labeling problem, advocating that commitment readiness should be assessed using trajectory-based statistics rather than instantaneous confidence alone. The approach empirically demonstrates that trajectory-aware commit gating can yield both higher quality and more parallel generation—without retraining, extra model parameters, or additional forward passes. This modularity enables immediate adoption in existing diffusion LLM frameworks.

Theoretically, this work positions trajectory-level signals (notably, candidate-conditioned temporal support) as a new class of model-agnostic, training-free readiness indicators that are both computable at inference and directly actionable for fast, reliable generation. The extra-promotion budget further provides an explicit, tunable bound on additional risk per step, introducing a controllable efficiency–robustness tradeoff.

Conclusion

TACG presents a simple yet powerful advance in DLLM inference, separating token identity from commit readiness while integrating temporally-aware and persistence-based signals for commitment gating. The method consistently improves or preserves accuracy and increases decoding efficiency across diverse settings, supporting its role as a general decoding policy improvement for DLLMs. Future work may extend TACG with group-level commit rules or integrate it with more dependency-aware schedulers, facilitating even broader adoption in high-throughput, robust diffusion LLM pipelines.

---

Reference: "TACG: Trajectory-Aware Commit Gating for Diffusion LLM Decoding" (2607.03236)