Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge

Abstract: Diffusion LLMs (DLMs) offer substantial speed advantages through parallel decoding, but the lack of token dependencies limits generation quality compared to autoregressive (AR) models. Recent progress attempts to bridge the gap via importance sampling, with DLM being the proposal and AR being the target. However, due to the huge gap between their distributions, the sampling requires a large number of particles and is thus expensive to compute. In this paper, we introduce PoE-Bridge, a novel decoding framework that drastically improves generation speed and accuracy by introducing an intermediate distribution to bridge the gap. The distribution is constructed as a Product-of-Experts (PoE) of the DLM proposal and the AR target. With the intermediate distribution, we first use the DLM to draft multiple continuations in parallel, then apply rejection sampling to verify the drafted tokens and move the resulting candidates toward the PoE. We then use importance sampling to further correct the PoE-aligned candidates toward the AR target. We further propose several improved techniques, including mixed-temperature sampling for enhanced diversity and elastic rejection windows for reducing wasted verification. Empirically, PoE-Bridge achieves significantly improved accuracy with $5\times$ speedup over the standard DLM decoding approach, and recovers at least 95% of the target AR model's performance, efficiently advancing most of the quality gap on challenging mathematical reasoning and coding tasks. Our code is available at https://github.com/juntongshi48/poe-bridge.

• The paper proposes PoE-Bridge, an intermediate product-of-experts framework that bridges DLM proposals with AR targets to enhance text generation fidelity.
• It employs mixed-temperature sampling and elastic rejection, achieving at least 95% AR accuracy with up to 5× speedup and 2× higher throughput compared to standard methods.
• The approach decouples sampling efficiency from quality, enabling scalable, parallel decoding and motivating further study of PoE interpolants across diverse generative tasks.

Diffusion LLM Parallel Decoding via Product-of-Experts Bridge

Introduction and Motivation

Autoregressive LLMs (ARLMs) yield high-fidelity text generation but are fundamentally limited by sequential decoding, restricting inference-time parallelism and resulting in latency bottlenecks. Diffusion LLMs (DLMs), leveraging iterative refinement in parallel, have emerged as a compelling alternative, but typically suffer quality degradation due to independence assumptions between tokens, leading to inferior results on complex benchmarks. There is a strong practical need to decouple sampling efficiency from quality, marrying DLM parallelism with ARLM faithfulness.

Recent attempts to correct DLM generations via Monte Carlo techniques—such as rejection and importance sampling using an ARLM as target—are limited by severe proposal–target distribution mismatch. This results in excessive sample rejection or poor quality when the number of candidates is constrained.

The PoE-Bridge Framework

PoE-Bridge addresses this inefficiency by introducing an intermediate “bridge” distribution between the DLM proposal and AR target, instantiated as a Product-of-Experts (PoE). The bridge enables a two-stage correction:

1. Speculative rejection sampling transitions from DLM to the PoE bridge, maximizing batch acceptance by reducing the mismatch.
2. Importance sampling then corrects from the PoE-bridge to the AR target, using parallel candidate continuation selection weighted appropriately.

Given a DLM mean-field proposal $\tilde{p}_D$ and AR target $p_{\mathrm{AR}}$, the bridge is parameterized as

$$p_{\mathrm{PoE}}(x) \propto \tilde{p}_D(x)^w p_{\mathrm{AR}}(x)^{1-w},$$

with $w \in [0,1]$ controlling the interpolation—$w=1$ yields DLM, $w=0$ recovers AR, and intermediate values place the bridge between them.

The core algorithm is as follows: From the current prefix, the DLM proposes $K$ parallel drafts; speculative verification via PoE accepts runs of tokens; then importance weighting selects an AR-faithful candidate. This sequence is iterated with elastic adaptation—windowed verification for efficiency, and mixed-temperature candidate generation for improved diversity and importance resampling.

Figure 1: Illustration contrasting naive speculative sampling and PoE-Bridge; the latter achieves efficient, high-fidelity AR-aligned generation by interposing a bridge distribution.

Empirical Evaluation

Benchmarks include GSM8K, MATH, HumanEval, and MBPP, focusing on code and mathematical reasoning where DLM-AR accuracy gaps are prominent. The protocol uses Dream-7B-Instruct as the DLM, Qwen2.5-Math-7B-Instruct and Qwen2.5-Coder-7B-Instruct as AR experts, and controlled evaluation setups.

On all benchmarks, PoE-Bridge achieves at least 95% of AR accuracy while delivering up to 5× speedup in throughput compared to entropy-based DLM decoding and 2× higher throughput than standard AR decoding. Performance approaches ARLM reference lines on all challenging tasks.

Figure 2: Impact of increasing parallel candidate number $K$; mixed-temperature sampling yields consistent accuracy rises without early plateau, unlike uniform-temperature sampling.

Extensive ablation reveals:

• Mixed-temperature sampling is critical for scaling accuracy with $K$, preventing sample collapse and increasing effective AR coverage.
• The PoE weight $w$ governs a quality–efficiency tradeoff; $p_{\mathrm{AR}}$0 is empirically optimal, striking a balance between throughput and faithfulness.
• The Elastic Rejection Window adapts parallel token blocks verified at each step, efficiently utilizing hardware without excessive waste or serial bottlenecks.

  Figure 3: Additional statistics on MATH—generation length and acceptance rate increase with $p_{\mathrm{AR}}$1, confirming better AR approximation.

  

  Figure 4: Parallel results on MBPP, showing similar trends of improved parallelism and AR convergence with higher $p_{\mathrm{AR}}$2.

Theoretical and Practical Implications

The PoE-Bridge framework demonstrates that a tractable intermediate distribution—grounded in a token-level PoE construction—enables effective Monte Carlo correction between a parallel decoding proposal and an AR target. By controlling bridge proximity to DLM or AR via $p_{\mathrm{AR}}$3, the method facilitates longer accepted token blocks without significant AR mismatch, amortizing sequential verification costs.

The bridge design is compatible with batch hardware and can integrate with speculative decoding advances targeting overlapping or heterogeneous tokenization. Practical deployment can naturally extend to multi-query or distributed regimes, contingent on memory management and batching architecture.

Theoretically, PoE-Bridge motivates further study of PoE interpolants for bridging dissimilar generative processes beyond text, such as structured prediction and multimodal domains. More generally, it suggests that faithfulness–efficiency frontiers can be advanced by nontrivial proposal construction, extending beyond naive importance correction pipelines.

Limitations and Future Directions

Current limitations include reliance on shared tokenizer alignment and evaluation under batch-size-1, single-GPU setups. These restrictions can be circumvented through token alignment and distributed inference systems. PoE-Bridge does not offer strict AR exactness at finite $p_{\mathrm{AR}}$4, though empirical degradation is minor and decreases with scale.

Future work may explore integrating PoE-Bridge with advanced model serving strategies and extending to cross-modal or adaptive proposal–target conditioning. Efficient scaling to extremely large DLM–AR pairs, as well as formal bounds on bias and variance in the bridge regime, merit further analysis.

Conclusion

PoE-Bridge establishes an effective paradigm for AR-faithful, parallel text generation, unifying DLM and ARLM strengths through an explicit intermediate PoE bridge. It achieves near-AR accuracy with substantial throughput improvements, enabled by a principled, highly parallel two-stage correction. The framework advances the practical utility of discrete diffusion models and demonstrates that proposal–target mismatch can be systematically ameliorated with theoretical and algorithmic rigor (2606.08048).