DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding

Abstract: Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in parallel. Predicting an entire block at once requires a sufficiently capable draft model and effective utilization of the target model's internal knowledge. However, the state-of-the-art method DFlash constrains all draft layers to share a single fused representation derived from only a few target layers, limiting per-layer expressiveness and hindering further scaling of draft capacity. In this paper, we present \modelname, which flares out the narrow conditioning bottleneck of DFlash through a lightweight layer-wise fusion mechanism: each draft layer attends to its own learnable combination of a broad set of target layers at negligible overhead, simultaneously injecting richer target knowledge and providing every draft layer with a distinct input. This enhanced per-layer expressiveness enables scaling the draft model to deeper architectures with consistent gains. We further scale training data from 800K to 2.4M samples to fully exploit the enlarged capacity. On six benchmarks spanning mathematical reasoning, code generation, and conversation, \modelname attains average wall-clock speedups of 5.52x on Qwen3-4B, 5.46x on Qwen3-8B, and 3.91x on GPT-OSS-20B, improving over DFlash by roughly 11\%, 8\%, and 5\% respectively. Our code is available at https://github.com/Tencent/AngelSlim.

• The paper introduces DFlare's adaptive layer fusion, replacing a shared context with learnable, per-layer mixtures from target states.
• It employs heterogeneous key-value projections to preserve distinct representations, enhancing model expressiveness without additional overhead.
• Empirical results show up to 5.52x speedup and improved acceptance lengths, demonstrating robust gains over existing methods.

DFlare: Enhancing Draft Model Capacity for Block Diffusion Speculative Decoding

Introduction

The efficiency bottleneck in LLM inference, particularly for high-throughput applications, has prompted research into speculative decoding strategies that exploit parallelism via draft-then-verify paradigms. Block diffusion speculative decoding, which simultaneously predicts multiple tokens per block using a draft model and then verifies them en masse with the target model, can dramatically reduce latency. However, scaling the capacity and the representational power of the draft model without incurring counterproductive cost or redundancy remains an open problem.

The methodology in "DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding" (2606.02091) directly addresses the expressiveness bottleneck that exists in current block diffusion systems such as DFlash, where all draft layers share a fused and limited projection of the target's hidden features. DFlare proposes and empirically validates a lightweight, adaptive layer fusion over target states for every draft layer, along with heterogeneous KV projections and an improved training curriculum, providing substantial, consistent advancements in speculative inference speed and accepted sequence length.

Methodological Advancements

Adaptive Layer Fusion

DFlare's key innovation is the replacement of DFlash's single, shared fused context with a per-layer, learnable fusion—each draft Transformer layer receives a dedicated mixture over a broader set of target model hidden states. This layer-wise fusion is implemented as a scalar-weighted sum (normalized via softmax) over the selected target layers, with negligible computational overhead even as the number of conditioning layers increases.

This architectural change is critical: by providing each draft layer with a distinct view into the target model, specialization is enabled, and the information bottleneck present in DFlash is alleviated. Importantly, this also unlocks effective scaling of draft model depth—additional layers provide meaningful, non-redundant representational gains.

Figure 1: Overview of DFlare using adaptive layer fusion and heterogeneous KV projections for enhanced per-layer expressiveness.

Heterogeneous Key-Value Projections

In standard block diffusion drafting schemes (including DFlash), the projections for both draft states and injected target contexts share the same parameters. DFlare decouples these: it uses independent key and value projection matrices for the fused target context and the draft positions, ensuring that distinct representational information is preserved and utilized without forced compromise in the KV space. This further increases the per-layer capacity to incorporate and act upon target model knowledge.

Progressive Position-Weighted Training Loss

DFlare also improves the training objective by adopting a dynamic, position-weighted cross-entropy loss. Rather than a fixed exponential decay in importance for positions later in each block (as in DFlash), DFlare linearly increases the decay parameter from a low to a high value across training, emphasizing early positions during initial epochs and flattening the weighting as training progresses. This curriculum accelerates convergence while ultimately optimizing accuracy across all block positions.

Empirical Evaluation

The experiments encompass three target models (Qwen3-4B, Qwen3-8B, GPT-OSS-20B), six benchmarks spanning mathematical reasoning, code generation, and conversation tasks, and evaluation across greedy and stochastic decoding. The primary metrics are wall-clock speedup over baseline autoregressive decoding and acceptance length (the mean number of consecutive draft tokens accepted per verification).

DFlare demonstrates significant, consistent improvements:

• Average wall-clock speedups reach $5.52\times$ (Qwen3-4B), $5.46\times$ (Qwen3-8B), and $3.91\times$ (GPT-OSS-20B), corresponding to $10.6\%$, $8.1\%$, and $5.4\%$ gains relative to DFlash.
• Acceptance lengths improve by $15.5\%$, $14.7\%$, and $5.8\%$ under greedy decoding for the same set of models.
• Larger training sets (up to 2.4M samples) further boost performance, indicating that the enhanced capacity of DFlare can be fully realized with sufficient data.

In all settings, DFlare's improvements persist under stochastic decoding, and all gains are robust across application domains.

Figure 2: (Left) DFlare achieves largest speedup on all Qwen3-8B benchmarks under greedy decoding; (Right) as the training data increases, both speedup and acceptance length grow without saturation.

Layer and Feature Scaling Analysis

Experiments varying the number of draft layers unequivocally show that, for DFlare, both speedup and acceptance length scale with depth, in contrast to the saturation observed with DFlash. Adding target layers (i.e., fusing from more levels of the target model) further improves DFlare's efficacy, again without diminishing returns.

Ablation studies confirm that all components—softmax-normalized fusion, heterogeneous KV projections, and the progressive loss—independently contribute to performance, and omitting any of them degrades either speedup or acceptance length.

Figure 3: (Left) DFlare scales with draft model depth, achieving monotonicity in speedup and acceptance length, unlike DFlash; (Right) incorporating more target model layers yields continued improvements for DFlare.

Theoretical and Practical Implications

DFlare addresses two tightly connected limitations in block diffusion speculative decoding: depth scaling and target knowledge injection. The methodology provides architectural guidance for speculative decoding research and deployment—layer-wise specialization via fusion mechanisms and decoupled attention spaces are shown to be essential for efficient utilization of larger, deeper draft models.

Practically, the reduced inference latency per token, especially as block size and draft depth increase, yields substantial throughput improvements in serving environments. The cost of training the more expressive draft model can be amortized over numerous deployments, making DFlare attractive for industrial-scale deployment.

Future directions may include:

• Further scaling draft models and training sets, exploring if improvements saturate or continue;
• Integrating DFlare designs with other speculative acceleration paradigms, such as tree-based or classifier-based inference;
• Investigating fusion mechanisms informed by model interpretability analyses or uncertainty estimation.

Conclusion

DFlare constitutes a major architectural advance in scaling draft capacity for block diffusion speculative decoding. By employing adaptive, layer-specific fusion of target features and heterogeneous KV projections, as well as an improved training curriculum, DFlare achieves consistently superior speedup and acceptance length over prior methods across a robust suite of benchmarks and target models. Its results substantiate the importance of alleviating representational bottlenecks in block diffusion drafting and set the direction for further research at the intersection of efficient LLM inference and expressive architectural design.