Streaming-dLLM: Accelerated Diffusion LLM

• Streaming-dLLM is a training-free framework that accelerates diffusion-based LLM inference by applying suffix pruning and adaptive decoding.
• It achieves spatial efficiency by reducing context via attenuation-guided modeling and improves temporal efficiency with dynamic confidence thresholds.
• Empirical results demonstrate significant throughput gains and lower latency, making it ideal for long-form, latency-sensitive applications.

Streaming-dLLM refers to a training-free acceleration framework for diffusion-based LLMs (dLLMs) focused on optimizing spatial and temporal efficiency in diffusion decoding. It targets the inefficiencies of block-wise dLLM inference—specifically, spatial redundancy in attending to long, uninformative suffixes and temporal inefficiency from fixed masking schedules—by introducing suffix pruning and dynamic, confidence-aware decoding. Streaming-dLLM can be deployed as a plug-and-play module for dLLM inference, yielding significant improvements in throughput and latency with negligible impact on output quality (Xiao et al., 25 Jan 2026).

1. Diffusion-Based LLMs and Inference Inefficiency

Diffusion-based LLMs (dLLMs) generate target sequences by iterative refinement of masked token blocks. For a target sequence of length $L$, the initial state consists of a prompt $p_0$ followed by $L$ masked positions:

$x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$

Tokens are grouped into $N$ non-overlapping blocks of size $K$ (thus $L = N \cdot K$). At each diffusion step $t$ (for a total $T = N \cdot M$ diffusion steps), the model $f_\theta$ predicts logits for all masked positions:

$p_0$0

A selection rule determines which masked positions are updated. Bidirectional attention across prompt plus all masked tokens enables superior global coherence relative to autoregressive models and allows tokens within a block to be finalized in parallel. However, standard dLLM inference attends over the entire masked suffix at each step, incurring unnecessary computation as $p_0$1 increases, and applies fixed confidence thresholds, causing either excess waiting for high-confidence tokens or premature updates for uncertain positions (Xiao et al., 25 Jan 2026).

2. Spatial Acceleration: Suffix Pruning via Attenuation-Guided Modeling

Streaming-dLLM introduces an attenuation-guided suffix modeling strategy. Empirically, attention scores from the current block to distant suffix blocks decay rapidly, indicating that only a narrow window of suffix blocks and the end-of-sequence token provide significant contextual utility. The framework constructs a sliding window of $p_0$2 contiguous suffix blocks $p_0$3 tokens per block$p_0$4 and includes the final token to capture necessary position information. The effective context at step $p_0$5 is then:

$p_0$6

where $p_0$7 covers the prompt and all decoded blocks, $p_0$8 the active block, and $p_0$9 the pruned subset of the suffix. This reduces the per-layer attention cost from $L$0 to $L$1. Empirical results confirm that using a much smaller context (e.g., $L$2 for $L$3) does not degrade output quality (Xiao et al., 25 Jan 2026).

3. Temporal Acceleration: Dynamic Decoding with Adaptive Thresholding

Classical dLLMs employ a fixed confidence threshold $L$4 to determine when masked positions are updated. Streaming-dLLM instead adopts an adaptive thresholding rule:

$L$5

where $L$6 is the fraction of still-masked positions in the current block, $L$7 is a base threshold, and $L$8 determines how aggressively the threshold adapts. At each iteration, all tokens $L$9 with $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$0 are unmasked; if none pass the threshold, the most confident token is updated to ensure progress. An early exit mechanism halts decoding if a high-confidence EOS token is produced, thus saving unnecessary iterations for converged outputs. This dynamic policy improves sample efficiency and model throughput (Xiao et al., 25 Jan 2026).

4. Streaming-dLLM Inference Algorithm

Streaming-dLLM simply replaces full-suffix attention with pruning (as above) and fixed-threshold unmasking with dynamic thresholding and early exit. No model retraining is required. The inference pseudocode is as follows: $K$1 Practical values typically set $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$1 ($x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$2), $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$3, and $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$4 (Xiao et al., 25 Jan 2026).

5. Empirical Performance and Comparative Evaluation

Streaming-dLLM achieves substantial improvements in speed and computational efficiency while preserving or slightly improving output quality:

• Throughput: Up to $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$5 speedup on MBPP@512 tokens; up to $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$6 for $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$7.
• Latency: Up to $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$8 reduction in per-sample inference time.
• Quality: Output accuracy remains within $x^{(0)} = [p_0, [MASK]_1, \ldots, [MASK]_L]$9 of the full-suffix baseline. Compared to dKV-Cache, Prefix-Cache, and Fast-dLLM baselines, Streaming-dLLM delivers higher throughput and comparable or superior accuracy (e.g., on GSM8K@512, Fast-dLLM achieves $N$0 TPS vs. $N$1 TPS for Streaming-dLLM). Ablation studies show that each component—suffix pruning, dynamic decoding, early exit—contributes to the overall speedup (Xiao et al., 25 Jan 2026).

Method	Speedup (vs baseline)	Quality Δ
Suffix pruning	$N$2	Slight gain
+ Dynamic decoding	$N$3	Minor change
+ Early exit	$N$4	None

6. Implementation and Deployment Considerations

• Hyperparameter tuning: The suffix window ($N$5) controls the balance between speed and quality; recommended $N$6 values are $N$7. Setting $N$8 too low or $N$9 too high may cause premature unmasking and reduced quality.
• Applicability: Best suited for long-form generation tasks ($K$0), block-wise diffusion architectures (e.g., Dream-7B, LLaDA-1.5/8B), and interactive systems where latency is critical.
• Plug-and-play: Streaming-dLLM operates as a training-free wrapper, requiring only modifications to the inference routine without retraining the underlying dLLM (Xiao et al., 25 Jan 2026).

7. Significance and Broader Impact

Streaming-dLLM represents a practical advance for accelerating natural language generation in diffusion-based LLMs, effectively addressing inefficiencies that scale with output length and enabling near-real-time inference in settings that previously suffered from substantial computational overhead. Its design principles—attenuation-guided context pruning, adaptive masking, and early exit—may influence efficient inference in other non-autoregressive generative frameworks and facilitate broader adoption of dLLMs in latency-sensitive and high-throughput applications (Xiao et al., 25 Jan 2026).