DFlash: Accelerating LLMs with Block Diffusion

• The paper demonstrates that a lightweight, context-conditioned block diffusion drafter can accelerate speculative decoding in LLMs with speedups exceeding 6× while maintaining lossless output.
• It integrates a block diffusion model with autoregressive verification using deep key-value injection to maximize draft acceptance and enhance hardware efficiency.
• Empirical results reveal significant throughput gains, achieving up to 4.8× tokens/sec and over 90% GPU utilization compared to traditional autoregressive methods.

DFlash: Block Diffusion for Flash Speculative Decoding is a decoding framework for LLMs that leverages blockwise diffusion modeling for accelerated, lossless speculative decoding. By employing a lightweight, context-conditioned block diffusion drafter tightly integrated with standard transformer architectures, DFlash enables highly parallel block-level draft generation, followed by efficient verification against a target autoregressive (AR) LLM. This approach yields substantial improvements in throughput and GPU utilization compared to traditional autoregressive speculative decoding methods, enabling lossless (distribution-preserving) speedups exceeding 6× and providing up to 2.5× higher acceleration than previous state-of-the-art speculative decoding pipelines (Chen et al., 5 Feb 2026, Sandler et al., 1 Nov 2025, Christopher et al., 2024).

1. Background: Autoregressive Decoding, Diffusion Models, and Speculative Decoding

Autoregressive LLMs generate output sequences token by token, each prediction conditioned on the previous context: $x_i \sim p_{\rm target}(x_i \mid x_{<i})$. This strict sequential dependency results in suboptimal GPU utilization and high latency during inference, since only a single token is produced per forward pass. Speculative decoding mitigates this bottleneck by employing a lightweight draft model to propose multiple future tokens (“draft block”) in parallel. The target LLM then verifies (in parallel) how many of these drafts are correct and accepts as many as match its own greedy output, appending these to the accepted sequence and resuming from the next position if a mismatch occurs. In previous speculative decoding methods (e.g., EAGLE-3), the drafter itself is autoregressive, limiting scalability because its generation cost grows linearly with block size $\gamma$, which caps practical speedup at approximately 2–3× (Chen et al., 5 Feb 2026).

Diffusion LLMs provide an alternative by relaxing strict AR dependencies, enabling the simultaneous inference of entire token blocks. However, standard diffusion models typically require multiple denoising steps per token or block, and their open-source instantiations can lag behind AR models in both speed and quality if not carefully designed. DFlash introduces a tightly coupled, context-aware block diffusion drafter capable of proposing full token blocks in a single forward pass, efficiently conditioned on the target LLM’s internal features, and supporting highly parallel verification (Chen et al., 5 Feb 2026).

2. DFlash Block Diffusion Draft Model

The DFlash drafter operates via a continuous-time diffusion process defined over embeddings for entire token blocks. For a ground-truth block $x_0 \in \{1,\dots,V\}^\gamma$ with embeddings $E(x)\in\mathbb{R}^{\gamma\times d}$, the forward process adds Gaussian noise to $E(x_0)$ following a variance schedule $\{\alpha_t\}$:

$$\bar\alpha_t = \prod_{s=1}^t \alpha_s, \quad q(x_t \mid x_0) = \mathcal{N}(x_t; \sqrt{\bar\alpha_t} E(x_0), (1-\bar\alpha_t) I)$$

The reverse (“denoising”) process is parameterized as:

$$x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\Big(x_t - \frac{1-\alpha_t}{\sqrt{1-\bar\alpha_t}}\,\epsilon_\theta(x_t, t, c)\Big) + \sigma_t z,\quad z\sim\mathcal{N}(0, I)$$

Here, $\epsilon_\theta$ is a neural network predicting the noise residual, and $c$ is a conditioning vector derived from the target model’s hidden states (see below). Unlike traditional diffusion LLMs requiring many denoising steps, DFlash collapses the process into a single (or few) diffusion passes, with learned noise-scheduling integrated into linear layers for efficiency.

Contextual Conditioning via KV-Injection:

DFlash achieves high draft quality by injecting rich, deep context from the target LLM. After a prefill pass over the prompt and any previously accepted tokens, several target LLM hidden layers are concatenated and linearly fused to yield context vector $c$:

$$c = W_{\rm fuse}[h^{(\ell_1)};\dots;h^{(\ell_k)}] \in \mathbb{R}^{d_c}$$

This is injected as key and value projections into every drafter block layer, allowing the drafter to attend to the target's deep semantic features, which substantially increases average acceptance length $\tau$ (tokens accepted per draft) (Chen et al., 5 Feb 2026).

Draft Block Generation Algorithm:

The full block generation proceeds as follows:

1. Extract $c$ from chosen hidden layers of the target model.
2. Prepare a block of $\gamma$ masked tokens at the output positions.
3. Run the block diffusion drafter, conditioned on $c$, to generate logit distributions for all block tokens in parallel.
4. Sample each token independently to form the draft block.

3. DFlash Speculative Decoding: Pipeline and Acceptance

Given a prompt + previous output and a block size $\gamma$, the DFlash decoding loop per block is:

1. Block Drafting: Use the block diffusion drafter to propose $\gamma$ candidate tokens in parallel.
2. Block Verification: With the target LLM, compute logits for each position in the extended context—including the full draft block—enabling all verification steps in a single batched forward pass.
3. Acceptance Criterion: Sequentially compare each draft token $\hat{x}_i$ against the target model’s greedy choice at position $i$. Acceptance proceeds tokenwise until the first mismatch, which ends the acceptance streak. Accepted tokens are appended to output; on a mismatch, one AR step of the verifier handles the rejected position, then the process resumes.

The per-token lossless acceptance result ensures that the output distribution precisely matches that of the target LLM under greedy generation (Chen et al., 5 Feb 2026, Sandler et al., 1 Nov 2025).

4. Performance Analysis: Complexity, Throughput, and Empirical Results

DFlash structurally decouples draft block size $\gamma$ from draft time complexity, since the block diffusion model generates an entire block in $O(1)$ time (relative to $\gamma$). The target LLM verifier operates in embarrassingly parallel mode, benefiting from high hardware occupancy:

$$L_{\rm DFlash} = \frac{t_{\rm parallel} + t_{\rm verify}}{\mathbb{E}[\tau]}$$

$$\eta_{\rm speedup} = \frac{L_{\rm AR}}{L_{\rm DFlash}} \approx \frac{\mathbb{E}[\tau]}{1 + t_{\rm parallel}/t_{\rm step}}$$

Empirical findings demonstrate:

• DFlash with block size $\gamma=16$ attains acceptance lengths $\tau\approx 6.5$ (vs. $\tau\approx 3.4$ for EAGLE-3) and yields $4.86\times$ speedup on Qwen3-8B.
• On Qwen3-4B with Math500, DFlash achieves throughput of $1531$ tokens/sec versus $316$ baseline ($4.8\times$), with average blockwise $\tau=8.01$.
• With optimal design (“mini” denoiser, $k=5$ context vectors), >6× speedup is consistently observed; deeper drafter layers provide diminishing returns beyond 5–8 layers.
• GPU utilization frequently exceeds 90% due to batch-parallel drafter and verifier execution.

5. Comparison with Related Approaches

Against AR-based Speculation:

Autoregressive speculative decoding suffers from linearly growing draft cost in $\gamma$ and hardware underutilization. DFlash’s all-at-once block diffusion design allows for aggressive batching with constant-time drafting per block, significantly higher acceptance rates, and better alignment with modern GPU hardware.

Spiffy and Block Diffusion:

Spiffy (Agrawal et al., 22 Sep 2025) introduced auto-speculative draft graphs for masked diffusion LLMs, with strategies for parallel draft/verify using directed graphs and calibration. The Spiffy “directed draft graph” abstraction directly anticipates DFlash’s workflow but operates natively on masked diffusion LLMs rather than as a speculative proxy for AR LLMs. DFlash can be viewed as adapting this principle, emphasizing tight context-injection, blockwise diffusion, and practical integration with AR verifiers for maximal system-level speedup.

SpecDiff and SpecDiff-2:

SpecDiff (Christopher et al., 2024) and SpecDiff-2 (Sandler et al., 1 Nov 2025) propose using discrete diffusion models as block drafters, leveraging single-step masked denoisers, “streak distillation” for blockwise alignment, and test-time block self-selection. SpecDiff-2 achieves up to $4.6\times$–$4.9\times$ speedup, with further improvements of $10$–$20\%$ via multi-draft selection. DFlash’s architectural innovations (embedding-diffusion, deep KV conditioning) allow it to outperform these methods, particularly at large block sizes and with aggressive context fusion (Chen et al., 5 Feb 2026).

Deferred Commitment Decoding (DCD):

DCD (Shu et al., 5 Jan 2026) improves qualitative accuracy in block-based diffusion decoding by deferring uncertain token commitments within a sliding window, mitigating the boundary-induced context truncation (BICT) issue. DCD is scheduling-based and model-agnostic, while DFlash applies a two-stage speculative pipeline. Hybridizing DCD with DFlash may further combine DCD’s dynamic uncertainty handling with DFlash’s parallel, low-latency speculative pipeline.

6. Limitations, Ablations, and Extensions

• DFlash currently relies on a single-step or shallow denoiser. Increasing diffusion depth (i.e., more denoising steps) provides marginal gains in acceptance length at a linear computational cost.
• Block size $\gamma$ is typically fixed, and performance is sensitive to its value; adaptive block sizing according to context or workload may further optimize throughput.
• The exponentially decaying output loss weight for token positions in training increases accuracy and acceptance for early tokens in block, which is critical since later tokens’ acceptance depends on early correctness.
• DFlash leverages strict block-level scheduling for KV-cache re-use and maximal hardware efficiency, though further improvements could come from hybridizing with confidence-based sliding window techniques (as in DCD).
• Extending DFlash to multimodal data or longer context regimes will require architectural tailoring of block diffusion drafters and context extraction schemes.

7. Practical Implementation and Empirical Scaling

Implementation details for DFlash include:

• Block size $\gamma$ of 16–32 empirically yields peak acceptance/latency tradeoff (higher blocks are possible with sufficient memory).
• Five-layer block diffusion drafters with fusion of deep contextual features provide near-optimal acceptance and speedup; additional layers yield diminishing return.
• For alignment between drafter and verifier, “streak distillation”—maximizing expected block acceptance—is employed to ensure high blockwise agreement.
• In batched servers, DFlash achieves >90% hardware utilization.
• Across multiple benchmarks and target LLMs, DFlash consistently outperforms prior speculative and block-diffusion approaches on both throughput and quality.
• Integration with next-generation attention kernels (e.g., FlashAttention) and efficient cache management is essential for maximal gain.
• Empirical ablations show that optimizing drafter step count, context vector depth, and training loss weighting all materially contribute to throughput and acceptance (Chen et al., 5 Feb 2026).

---

DFlash represents an overview of blockwise diffusion modeling, context-conditioned draft generation, and speculative decoding verification, yielding a highly parallel, rapid, and lossless solution for AR LLM decoding acceleration. The key architectural insight is the tight coupling between block diffusion drafters and deep context extraction from the target LLM, enabling high acceptance rates and throughput while strictly preserving output fidelity. This approach is supported by, and extends, recent research on diffusion-based speculative decoding and blockwise generation, setting a new benchmark for practical accelerated inference in LLMs (Chen et al., 5 Feb 2026, Agrawal et al., 22 Sep 2025, Sandler et al., 1 Nov 2025, Christopher et al., 2024).