DFlash: Block Diffusion for Speculative Decoding

• DFlash is a block diffusion framework that replaces token-by-token drafting with a parallel draft block, achieving lossless acceleration in language model decoding.
• It leverages a lightweight Transformer draft network conditioned on target-model hidden features to propose full blocks verified via an acceptance-correction rule.
• Empirical evaluations show significant speedups, while successive extensions like DDTree and CaDDTree further enhance throughput and acceptance length.

DFlash is a speculative decoding framework for autoregressive LLMs in which the conventional autoregressive drafter is replaced by a lightweight block diffusion model that predicts an entire draft block in a single forward pass. The target LLM remains authoritative: it verifies the proposal in parallel and applies the standard acceptance-correction rule, so the final output distribution is preserved exactly. In this sense, DFlash is best understood as a lossless acceleration method that trades sequential drafting for block-parallel drafting, while using target-model hidden features to improve draft quality and acceptance length (Chen et al., 5 Feb 2026).

1. Conceptual position within speculative decoding

Speculative decoding separates generation into two roles: a fast drafter proposes future tokens, and a slower target model verifies them in parallel. Earlier high-performing systems such as EAGLE-3 still draft autoregressively, so the draft cost grows linearly with the number of proposed tokens. DFlash was introduced to remove that residual sequential bottleneck by using block diffusion for drafting: instead of proposing tokens one by one, it predicts a full block in one shot, conditioned on the verified prefix and target-model context features (Chen et al., 5 Feb 2026).

The central metric in this setting is the accepted prefix per verification round. In the original DFlash presentation, speedup is described through the per-token latency

$$L = \frac{T_{\mathrm{draft}} + T_{\mathrm{verify}}}{\tau},$$

with $\tau$ denoting the expected number of accepted tokens per cycle, including the bonus token. The key asymmetry is that an autoregressive drafter incurs $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$, whereas DFlash replaces this with a single parallel draft pass whose cost is much less sensitive to block length. This makes deeper and more expressive draft models feasible without proportional latency growth, which is the main architectural reason DFlash can occupy a different part of the speed–quality Pareto frontier than autoregressive drafters (Chen et al., 5 Feb 2026).

Later work sharpened this interpretation. WhiFlash characterizes DFlash as especially effective in low-entropy, highly structured regimes such as coding and many math tasks, where block-parallel commitments are relatively reliable and the low per-round latency dominates throughput. The same analysis identifies a complementary weakness in high-entropy, open-ended continuations, where committing multiple positions that do not condition on each other increases the likelihood of early mismatch (Kwon et al., 5 Jun 2026).

2. Architecture and block diffusion drafting

The DFlash drafter is a small Transformer block diffusion network with 3–8 layers, typically 5 layers and 8 layers for Qwen3-Coder. It shares the token embedding and LM head with the target model, both kept frozen, and only the draft Transformer layers are trained. The draft does not operate as an independent LLM; rather, it is a specialized adapter for masked block prediction under target verification (Chen et al., 5 Feb 2026).

Its defining architectural feature is persistent conditioning on target-model hidden states. DFlash extracts hidden features from a fixed set of target layers, typically five layers sampled from shallow to deep, concatenates and fuses them, and injects the fused context directly into the Key and Value streams of every draft layer. In the notation given for layer $l$,

$Q^{(l)} = X^{(l)} W_Q^{(l)},$

$$K^{(l)} = [X^{(l)} W_K^{(l)};\; c\, U_K^{(l)}], \qquad V^{(l)} = [X^{(l)} W_V^{(l)};\; c\, U_V^{(l)}].$$

Attention is then computed over token KV and injected context KV together. This differs from input-only fusion schemes, whose conditioning signal can dilute with depth; DFlash’s conditioning is persistent across all draft layers and is cached for reuse across iterations (Chen et al., 5 Feb 2026).

At inference time, each draft block has an anchor token in the first position and masked future positions in the remainder of the block. The drafter predicts all masked positions simultaneously in a single forward pass. The original system uses block size 16 for Qwen3 targets and block size 10 for LLaMA-3.1-8B-Instruct (Chen et al., 5 Feb 2026). Subsequent work treated these per-position marginals as a reusable object in their own right. DDTree shows that vanilla DFlash verifies only one drafted trajectory per round even though the one-pass block drafter exposes a full set of per-position distributions $q_i(\cdot \mid c,b)$; it constructs a tree of multiple candidate continuations from those marginals and verifies them in one target pass using ancestor-only attention (Ringel et al., 14 Apr 2026).

This reuse of DFlash marginals became a distinct line of work. CaDDTree formalizes the same per-position outputs as sufficient statistics for a cost-aware objective, optimizing tokens per unit time rather than acceptance length alone. The result is a dynamic per-round budget selector for DFlash-style diffusion draft trees rather than a fixed offline node budget (Zhang et al., 1 Jun 2026).

3. Training objective, masking scheme, and exactness

DFlash uses discrete masked block denoising rather than a continuous Gaussian diffusion parameterization. During training, the anchor token is left clean and the remaining positions in the block are masked; the model predicts the masked tokens in parallel through the shared LM head. Multiple blocks are sampled from a training sequence, and attention is bidirectional within each block but disallowed across blocks. The implementation uses Flex Attention to train many sampled blocks jointly in a single forward/backward pass (Chen et al., 5 Feb 2026).

The training loss is a position-weighted cross-entropy that emphasizes early block positions, because an error near the start of a speculative block invalidates all later positions under left-to-right verification. For position $k$ in the block,

$w_k = \exp\!\left(-\frac{k-1}{\gamma}\right),$

with $\gamma = 7$ for block size 16, $\tau$0 for block size 10, and $\tau$1 for block size 8. The loss is then

$\tau$2

This weighting was reported to accelerate convergence and improve acceptance length relative to uniform weighting (Chen et al., 5 Feb 2026).

The target responses used for training are themselves generated by the target model. The reported training corpus comprises approximately 800K samples from Nemotron Post-Training Dataset v2 and CodeAlpaca, with sequence length 3072 and 4096 for Qwen3-Coder. The optimizer is AdamW with learning rate $\tau$3, gradient clip 1.0, and cosine schedule with warmup ratio 0.04. Each sequence contributes 512 randomly sampled anchor positions per epoch. Random anchor sampling was later reported as materially better than uniform block partitioning, both in speedup and in acceptance length (Chen et al., 5 Feb 2026).

Exactness is not a property of the draft model alone but of the verification rule. DFlash adopts the standard speculative acceptance-correction mechanism: for drafted token $\tau$4 with draft probability $\tau$5 and target probability $\tau$6, the verifier accepts with probability

$\tau$7

On the first rejection, the remainder of the block is discarded and the target samples the correction token. If the entire block is accepted, the target emits a bonus token to preserve unbiasedness. Because the target always governs acceptance and correction, DFlash remains lossless under the same sampling configuration as the target model (Chen et al., 5 Feb 2026).

A later training paper, Spec-AUF, argues that mask-only block drafters such as DFlash are misaligned at train time because full-block cross-entropy supervises suffix positions that inference never reaches once an earlier token fails. Its Accept-Until-Fail objective truncates the cross-entropy support at the first predicted failure and, on Qwen3-8B, raises DFlash’s average emitted length $\tau$8 across six benchmarks from 2.40 to 2.61 under greedy verification, with gains on every benchmark (Yang et al., 2 Jul 2026).

4. Empirical performance and operating regimes

The original DFlash evaluation covered Qwen3-4B, Qwen3-8B, Qwen3-Coder-30B-A3B-Instruct, and LLaMA-3.1-8B-Instruct on math, code, and chat benchmarks. On the Transformers backend with Qwen3-8B at temperature 0, DFlash with block size 16 reports average speedup of approximately $\tau$9 with $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$0; on MATH-500 it reaches approximately $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$1–$$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$2 speedup with $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$3. At temperature 1 it maintains approximately $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$4 average speedup with $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$5. These figures support the paper’s headline claim of over $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$6 lossless acceleration across a range of models and tasks and up to $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$7 higher speedup than EAGLE-3 (Chen et al., 5 Feb 2026).

The serving-side picture is similar. Under SGLang with a single B200, Qwen3-8B on MATH-500 reaches speedups from $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$8 at concurrency 1 to $$T_{\mathrm{draft}} = \gamma \cdot t_{\mathrm{step}}$$9 at concurrency 32, with average $l$0; Qwen3-4B exhibits the same $l$1 and speedups from $l$2 to $l$3. On LLaMA-3.1-8B-Instruct with block size 10, DFlash reports speedups from $l$4 to $l$5 on GSM8K, from $l$6 to $l$7 on HumanEval, and from $l$8 to $l$9 on Alpaca across concurrency 1 to 32 (Chen et al., 5 Feb 2026).

Ablations clarify what drives these gains. On Qwen3-4B, five draft layers produced the best end-to-end speedup: three layers drafted more cheaply but with lower acceptance, while eight layers increased acceptance to $Q^{(l)} = X^{(l)} W_Q^{(l)},$0 on MATH-500 but lost some throughput because draft latency rose. Using five target hidden features rather than three improved both $Q^{(l)} = X^{(l)} W_Q^{(l)},$1 and speedup. Training with larger blocks generalized downward more effectively than training with smaller blocks and testing larger. A drafter without target-feature conditioning dropped to roughly $Q^{(l)} = X^{(l)} W_Q^{(l)},$2–$Q^{(l)} = X^{(l)} W_Q^{(l)},$3 speedups with $Q^{(l)} = X^{(l)} W_Q^{(l)},$4–$Q^{(l)} = X^{(l)} W_Q^{(l)},$5, underscoring that block parallelism alone is not sufficient; the conditioning pathway is central (Chen et al., 5 Feb 2026).

At the same time, later evaluations identify operating regimes in which vanilla DFlash degrades. Test-Time Speculation reports that the acceptance length of DFlash, EAGLE-3, and PARD declines with generation length on long-response tasks. For Qwen3-8B with DFlash on MATH-500, average acceptance length falls from 3.7 in the first 10K output tokens to 1.5 in the last 10K tokens. The diagnosis offered is train–inference mismatch: speculative drafters are typically trained offline on short sequences, often $Q^{(l)} = X^{(l)} W_Q^{(l)},$6K tokens, but must match the target on much longer outputs during inference (Kumar et al., 10 May 2026).

5. Subsequent variants and extensions

The DFlash design rapidly became a substrate for both algorithmic and systems work. Some extensions modify how a DFlash-style block drafter is trained; others retain the drafter and change how its per-position marginals are consumed at inference.

Method	Main modification to DFlash	Reported outcome
DDTree	Builds a tree from DFlash per-position marginals and verifies multiple continuations in one target pass	Improves speedup and mean acceptance length $Q^{(l)} = X^{(l)} W_Q^{(l)},$7 over vanilla DFlash across 60 dataset–model–temperature combinations (Ringel et al., 14 Apr 2026)
CaDDTree	Chooses the tree budget online by maximizing throughput rather than acceptance length alone	Matches or surpasses DDTree with oracle budget selection on nearly all tasks (Zhang et al., 1 Jun 2026)
Graft-DFLASH	Prunes low-confidence block positions and grafts retrieved tokens into released positions	Improves average speedup on Qwen3-8B from $Q^{(l)} = X^{(l)} W_Q^{(l)},$8 to $Q^{(l)} = X^{(l)} W_Q^{(l)},$9 for DFlash block size 16 (Shen et al., 19 May 2026)
WhiFlash	Token-level router switches between DFlash and an autoregressive drafter	Up to $$K^{(l)} = [X^{(l)} W_K^{(l)};\; c\, U_K^{(l)}], \qquad V^{(l)} = [X^{(l)} W_V^{(l)};\; c\, U_V^{(l)}].$$0 throughput gain over DFlash on Chat for Qwen3-8B (Kwon et al., 5 Jun 2026)
Spec-AUF	Truncates DFlash training loss support at first predicted failure	Raises DFlash average emitted length $$K^{(l)} = [X^{(l)} W_K^{(l)};\; c\, U_K^{(l)}], \qquad V^{(l)} = [X^{(l)} W_V^{(l)};\; c\, U_V^{(l)}].$$1 from 2.40 to 2.61 on Qwen3-8B (Yang et al., 2 Jul 2026)
Draft-OPD	On-policy distillation from verification-exposed error positions	Improves over DFlash by 13% end-to-end speedup under matched training FLOPs (Lei et al., 28 May 2026)
TTS	Online test-time distillation during generation	Improves acceptance length over DFlash by up to 72% and 41% on average (Kumar et al., 10 May 2026)
DFlare	Layer-wise fusion over a broader set of target layers	Improves over DFlash by roughly 11%, 8%, and 5% on Qwen3-4B, Qwen3-8B, and GPT-OSS-20B (Zhang et al., 1 Jun 2026)
HyperDFlash	Aligns DFlash-style block drafting to DeepSeek-V4 MHC residuals	Outperforms native MTP and vanilla DFlash adaptation on accepted length and speedup (Lin et al., 25 Jun 2026)

These papers collectively suggest two broad interpretations. First, DFlash’s block-parallel drafting is a durable primitive: it is reused by tree construction, routing, pruning-and-retrieval, and architecture-aligned variants rather than being displaced wholesale. Second, its weaknesses are now fairly well characterized. Token-level routing work treats DFlash as the preferable policy in low-entropy, structured segments but not universally across a sequence (Kwon et al., 5 Jun 2026). Training work treats offline full-block supervision as misaligned with the verifier’s prefix gate (Yang et al., 2 Jul 2026). Online distillation work treats long-horizon degradation as a distribution-shift problem rather than a fixed architectural ceiling (Kumar et al., 10 May 2026).

6. Limitations, scope, and terminological ambiguity

DFlash’s main limitation is structural: it predicts block positions in parallel from the prefix but not from one another. This makes it efficient, yet it also means that uncertain later positions can be committed without inter-position conditioning. Later analyses therefore describe DFlash as strong in structured outputs and weaker in high-entropy chat or novel reasoning, where early mismatch risk is higher and acceptance length is more volatile within a single sequence (Kwon et al., 5 Jun 2026).

A second limitation is temporal robustness. Offline supervised training on short target-generated trajectories produces strong initial performance but can plateau in acceptance length and then degrade on long outputs. Draft-OPD identifies this as an offline-to-inference mismatch because the drafter is evaluated on states induced by its own proposals rather than only on fixed target trajectories. Its on-policy replay of verification-exposed anchors improves DFlash serving throughput by 6–17% in SGLang and raises $$K^{(l)} = [X^{(l)} W_K^{(l)};\; c\, U_K^{(l)}], \qquad V^{(l)} = [X^{(l)} W_V^{(l)};\; c\, U_V^{(l)}].$$2 by approximately 11.2% on average, indicating that a nontrivial portion of DFlash’s residual error is training-induced rather than intrinsic to block diffusion itself (Lei et al., 28 May 2026).

A third limitation is architectural specificity. DFlash’s original conditioning path assumes a conventional residual stream and a single fused representation over selected target layers. HyperDFlash shows that this assumption is misaligned with DeepSeek-V4’s Multi-Hyper-Connection residual architecture; replacing the generic dense reducer with an inherited HC-Gate reducer cuts reducer parameters from 67M to 65K and raises average accepted length and speedup relative to a vanilla DFlash adaptation (Lin et al., 25 Jun 2026). DFlare reaches a related conclusion for standard Transformers: forcing all draft layers to share one fused target representation bottlenecks expressiveness and makes deeper drafts saturate earlier than necessary (Zhang et al., 1 Jun 2026).

The term itself is also potentially ambiguous. In contemporary arXiv usage, “DFlash” most prominently denotes the 2026 speculative decoding method “Block Diffusion for Flash Speculative Decoding” (Chen et al., 5 Feb 2026). It should be distinguished from the earlier portrait-relighting system “DeepFlash: Turning a Flash Selfie into a Studio Portrait” (Capece et al., 2019), and from summaries that use “DFlash” informally for durable flash-memory indexing around the FM Tree literature (III et al., 2012). In current LLM inference research, however, DFlash refers specifically to the block diffusion drafter for speculative decoding and to the family of methods that extend it.