DSB: Dynamic Sliding Block Scheduling for Diffusion LLMs

Abstract: Diffusion LLMs (dLLMs) have emerged as a promising alternative for text generation, distinguished by their native support for parallel decoding. In practice, block inference is crucial for avoiding order misalignment in global bidirectional decoding and improving output quality. However, the widely-used fixed, predefined block (naive) schedule is agnostic to semantic difficulty, making it a suboptimal strategy for both quality and efficiency: it can force premature commitments to uncertain positions while delaying easy positions near block boundaries. In this work, we analyze the limitations of naive block scheduling and disclose the importance of dynamically adapting the schedule to semantic difficulty for reliable and efficient inference. Motivated by this, we propose Dynamic Sliding Block (DSB), a training-free block scheduling method that uses a sliding block with a dynamic size to overcome the rigidity of the naive block. To further improve efficiency, we introduce DSB Cache, a training-free KV-cache mechanism tailored to DSB. Extensive experiments across multiple models and benchmarks demonstrate that DSB, together with DSB Cache, consistently improves both generation quality and inference efficiency for dLLMs. Code is released at https://github.com/lizhuo-luo/DSB.

• The paper introduces Dynamic Sliding Block scheduling, which adapts block boundaries to token confidence and semantic difficulty to improve the quality–speed trade-off in diffusion LLM decoding.
• DSB Cache refreshes the active block and a preceding prefix window while reusing stable earlier states; on LLaDA-8B-Instruct, it reaches 80.29 GSM8K accuracy at 99.61 tokens per second.
• Results show stronger gains on diffusion-trained LLaDA models than on AR-initialized Dream models, highlighting the importance of model-training alignment and model-specific cache settings.

Motivation and problem statement

Diffusion LLMs (dLLMs) such as LLaDA and Dream decode multiple positions in parallel over iterative denoising steps, but global bidirectional decoding misaligns the generation order with the causal structure of natural language. The standard remedy is block-diffusion inference: the response is partitioned into fixed blocks that are decoded sequentially (semi-autoregressively), with parallel updates inside each block. While this improves quality substantially, the fixed block schedule is agnostic to semantic difficulty. It forces premature commitment to low-confidence positions inside the active block while delaying high-confidence positions just outside its boundary, producing a suboptimal quality–speed trade-off. Existing semantic-aware scheduling attempts rely on handcrafted heuristics or task-specific cues; WeDLM learns a flexible schedule via training, whereas this paper targets a training-free alternative.

Method

Dynamic Sliding Block (DSB) maintains an active block $B^{(t)} = [s^{(t)}, e^{(t)})$ whose boundaries advance at different rates. After each denoising iteration, the left boundary moves to just before the first still-masked position in the block (or to $e^{(t)}$ if none remain); the right boundary expands to $$\min(\ell_{prompt} + S_{init} + U,\; s^{(t+1)} + S_{max})$$, where $U$ counts unmasked positions. This guarantees at least $S_{init}$ unresolved tokens are available whenever possible, capped at $S_{max}$. Two special cases recover useful variants: $S_{max} = S_{init}$ yields a constant-size sliding block (DSB const.), and unbounded $S_{max}$ yields an aggressive variant (DSB greedy) with higher throughput but a weaker causal constraint.

DSB Cache. Applying existing training-free caches for fixed-block inference—prefix or dual caching with block-completion updates—to a sliding block causes severe degradation, because newly exposed boundary positions have transient KV states and get invalidated or recomputed frequently. DSB Cache addresses this by refreshing, at every step, both the active block and a prefix window immediately preceding it, whose length is $$\ell_{pw}^{(t)} = \max(\ell_{pmin},\, s^{(t)} - s^{(t-1)})$$, so it always covers newly exposed positions. Positions further left are cached and reused; a periodic global refresh after every $S_{init}$ decoded tokens re-synchronizes cached states. An appendix study shows the symmetric suffix-window design does not yield consistent benefits (accuracy stays below SWL=0 on LLaDA-1.5), so it is not adopted.

Experimental results

Experiments cover LLaDA-8B-Instruct, LLaDA-1.5, Dream-v0-Base-7B, and Dream-v0-Instruct-7B on GSM8K, MATH, HumanEval, MBPP, and BBH, using confidence-aware parallel decoding from Fast-dLLM (threshold 0.9), generation length 256, block size 32, on a single NVIDIA H200 via lm-eval.

Key findings:

• Without caching, DSB gives stable gains on the LLaDA family. On GSM8K with LLaDA-8B-Instruct, DSB greedy reaches 78.54 accuracy at 51.03 TPS versus 77.26/48.70 for naive block; on HumanEval, DSB const. reaches 42.07 accuracy at 124.6 TPS versus 40.85/119.1.
• With caching, gains widen and DSB sometimes exceeds vanilla Top-1 sampling quality: on GSM8K, DSB greedy achieves 80.29 (LLaDA-8B-Instruct) and 81.96 (LLaDA-1.5)—both above vanilla accuracy—at 99.61 and 95.54 TPS respectively, versus 92.26/86.42 TPS for naive block with Dual Cache.
• Gains on Dream are less consistent. The paper attributes this to Dream's AR-initialized, shift-aligned training; several Dream configurations show small regressions (e.g., DSB greedy drops GSM8K accuracy to 72.48 on Dream-v0-Base-7B under caching). This is a notable caveat: DSB's benefits appear strongest for diffusion-trained models rather than AR-initialized ones.
• Ablations confirm DSB Cache's necessity: replacing DSB Cache with Dual Cache on sliding blocks drops GSM8K accuracy from 80.14 to 76.42 and TPS from 98.10 to 78.93 (DSB const.), showing naive cache update rules fail under block movement.
• Sensitivity studies show DSB is robust across initial block lengths ($e^{(t)}$0 slightly degrades accuracy by weakening causality), across generation lengths, and exhibits a clear quality–speed trade-off in $e^{(t)}$1 on LLaDA (throughput rises, accuracy can fall), while Dream shows the opposite throughput trend with a turning point around $e^{(t)}$2.

Limitations and open questions

The method's effectiveness depends on the underlying model family: results on AR-initialized Dream models are mixed, and the paper offers only the training-alignment hypothesis as explanation without isolating the cause. Hyperparameter choices ($e^{(t)}$3, $e^{(t)}$4 for LLaDA vs. 4 for Dream) are model-dependent and tuned empirically rather than derived. The prefix window is justified heuristically by the left-context dependence of natural language, not by any formal analysis of KV-state stability. Open questions include whether DSB-like schedules should be incorporated into pretraining or post-training to align models with dynamic scheduling, and how DSB interacts with trained block-causal dLLMs beyond the inference-time settings evaluated here.

Conclusion

This paper identifies the fixed block schedule as a bottleneck in semi-autoregressive dLLM inference and proposes a training-free remedy: a dynamically sized, sliding active block (DSB) paired with a tailored KV-cache scheme (DSB Cache) that refreshes a prefix window to absorb boundary instability. Across four models and five benchmarks, the combined method consistently improves the quality–speed frontier, occasionally surpassing vanilla sampling accuracy at multi-fold higher throughput. The main qualification is reduced consistency on AR-initialized models, which leaves open how scheduling-aware training could close that gap.