I-DLM: Introspective Diffusion Language Model

• I-DLM is a parallel decoding framework that enforces introspective consistency to align token generation with causal evaluation, achieving AR-level sample quality.
• It employs the novel Introspective Strided Decoding (ISD) algorithm that validates proposal tokens via the Metropolis-Hastings criterion for accelerated, parallel generation.
• Specialized training objectives and system-level optimizations enable I-DLM to reach competitive benchmark scores while significantly boosting throughput.

The Introspective Diffusion LLM (I-DLM) is a parallel decoding framework for language generation that attains autoregressive (AR)-level quality while surpassing previous diffusion LLMs (DLMs) in efficiency and serving throughput. I-DLM achieves this by enforcing introspective consistency—the property that model-generated tokens are endorsed under causal re-evaluation—through specialized training objectives and the Introspective Strided Decoding (ISD) algorithm. For the first time, I-DLM delivers both the sample quality of same-scale AR transformers and the inference acceleration expected from parallel decoding, reaching competitive scores against established benchmarks and state-of-the-art models (Yu et al., 13 Apr 2026).

1. Introspective Consistency and Acceptance Rate

Introspective consistency is the structural property by which a model “accepts” its previously generated tokens when these are re-evaluated under the causal (next-token) distribution. By construction, AR models are perfectly introspectively consistent; the distribution used at step $k$ for next-token sampling, $q_k(x_k)$, always matches the actual conditional likelihood $p_\theta(x_k|x_{1:k-1})$, yielding acceptance probability $\alpha_k=1$ for each token.

Standard DLMs, however, generate tokens via diffusion-style masked proposals. This decoupling between generation and replayable probability creates introspective inconsistency: the generation pathway does not align with the causal “introspection” pathway, leading to sub-unity acceptance rates—typically $\alpha \approx 0.6$–$0.8$—under the Metropolis-Hastings criterion.

In I-DLM, for a generated token $x_k$:

• The decode (propose) distribution is $q_k(x_k)$.
• The introspective (causal) probability is $p_k(x_k) = p_\theta(x_k|x_{1:k-1})$.
• Per-token acceptance is defined by:

$$\alpha_k = \min\left(1, \frac{p_k(x_k)}{q_k(x_k)}\right)$$

• The average introspective acceptance rate is:

$q_k(x_k)$0

By jointly training the propose and introspect pathways using AR-style objectives with causally masked attention and logit shift, I-DLM drives $q_k(x_k)$1 toward $q_k(x_k)$2 (AR-level consistency), eliminating the fundamental mismatch that hampers earlier DLMs.

2. Introspective Strided Decoding (ISD)

ISD is the decoding procedure that enables I-DLM to maintain introspective consistency while performing parallel sequence extension. The algorithm operates in strided steps of size $q_k(x_k)$3:

1. Given a prefix of length $q_k(x_k)$4, append $q_k(x_k)$5 [MASK] tokens and compute a forward pass for proposal logits.
2. Sample the first new token $q_k(x_k)$6 using the AR-specific “free” slot for AR-exact quality; the next $q_k(x_k)$7 tokens $q_k(x_k)$8 are sampled as draft proposals.
3. In the same pass (by input concatenation), introspectively verify both previous and newly drafted tokens to obtain the anchor logits.
4. For each drafted token, accept or resample according to the Metropolis-Hastings acceptance criterion:

$q_k(x_k)$9

where $p_\theta(x_k|x_{1:k-1})$0.

1. If all proposals are accepted, sample a bonus token at $p_\theta(x_k|x_{1:k-1})$1.
2. Append up to $p_\theta(x_k|x_{1:k-1})$2 validated tokens to the prefix and update caches.

Each ISD iteration operates as two forward passes over $p_\theta(x_k|x_{1:k-1})$3 tokens but can be fused into a single “extend” operation for efficiency. The tokens-per-forward (TPF) metric scales with empirical acceptance $p_\theta(x_k|x_{1:k-1})$4 and stride $p_\theta(x_k|x_{1:k-1})$5: for uniform acceptance, $p_\theta(x_k|x_{1:k-1})$6 as $p_\theta(x_k|x_{1:k-1})$7 and degrades to $p_\theta(x_k|x_{1:k-1})$8 (fully sequential) as $p_\theta(x_k|x_{1:k-1})$9. In practice, with $\alpha_k=1$0 and $\alpha_k=1$1, I-DLM yields $\alpha_k=1$2 (Yu et al., 13 Apr 2026).

3. Training and Inference Methodology

Introspective-Consistency Training

I-DLM is initialized from a pretrained AR model (e.g., Qwen3-8B, Qwen3-32B) and fine-tuned for introspective consistency:

• For each clean token sequence $\alpha_k=1$3, construct a masked input $\alpha_k=1$4.
• Apply strict token-level causal attention throughout the concatenated [masked + clean] input, preserving AR causal kernels.
• Employ “logit shift”: at both masked and clean positions, the model predicts $\alpha_k=1$5 from position $\alpha_k=1$6 to align all hidden states to one-step-ahead prediction.
• Loss is the sum of masked and clean token cross-entropies, with auto-balanced per-step scaling so both pathways equally influence weight updates.

Notably, only 4–5B tokens of additional fine-tuning suffice to reach near-parity with the AR original, a significant reduction compared to prior DLM training which required over 50B tokens (Yu et al., 13 Apr 2026).

Inference Stack and Serving

Owing to strict causal attention and logit-shift alignment, I-DLM can be deployed atop the AR inference stack—including paged KV cache, continuous batch extension, fused attention kernels, and CUDA graph capture—without modification. A stationary-batch decode loop enables replay of pre-captured CUDA graphs at each stride, eliminating host-side scheduling overhead. Kernel fusion incorporates a paged-only attention kernel per layer and an online softmax with acceptance and Gumbel-max correction. The system supports LoRA gating at [MASK] positions only (“Residual ISD”), ensuring that anchor tokens remain bitwise identical to AR, guaranteeing lossless output equivalence.

4. Empirical Evaluation

I-DLM was evaluated on Qwen3-8B and Qwen3-32B bases, with denoising stride $\alpha_k=1$7, against benchmarks including AIME-24 and LiveCodeBench-v6.

Benchmark	Qwen3-8B (AR)	SDAR-8B	LLaDA₂.1-mini (16B)	I-DLM-8B
AIME-24	73.1	10.0	43.3	69.6
LiveCodeBench-v6	50.3	16.6	30.4	45.7

I-DLM-8B attains nearly AR-equivalent quality (<1 point difference) with full parallel decoding and outperforms the 16B diffusion model LLaDA₂.1-mini by 26 points on AIME-24 and 15 points on LiveCodeBench-v6. At the 32B model scale, I-DLM-32B surpasses even 100B-sized DLMs on challenging reasoning and coding tasks (Yu et al., 13 Apr 2026).

For throughput, measured on NVIDIA H100 at burst arrivals and output length 2048:

• At concurrency $\alpha_k=1$8, I-DLM-8B delivers 2.2–3.8× higher throughput than LLaDA₂.1-mini and 3.7–4.5× over SDAR-8B.
• Compared to speculative AR decoding with EAGLE-3, I-DLM achieves 1.3× higher throughput in lossless mode at low–medium concurrency ($\alpha_k=1$9), maintaining superiority up to $\alpha \approx 0.6$0.
• Throughput scales linearly with TPF, enabled by continuous batching, in contrast to SDAR’s synchronization overhead.

5. Model Architecture and System Integration

• Model sizes: I-DLM-8B (from Qwen3-8B), I-DLM-32B (from Qwen3-32B).
• Stride $\alpha \approx 0.6$1 is flexible, with training at $\alpha \approx 0.6$2, main evaluation at $\alpha \approx 0.6$3, and checkpoints at $\alpha \approx 0.6$4 achieving >4× TPF.
• ISD decode iterations process a batch of $\alpha \approx 0.6$5 tokens in one CUDA graph execution.
• Attention utilizes a paged-only causal FlashInfer kernel (one per layer), contrasting with the tripartite kernel cascade (ragged, prefix, merge) required in block diffusion approaches.
• KV cache is standard AR-style; no separate commit passes are necessary.
• Residual ISD for lossless decoding leverages rank-128 (for 8B) or rank-1024 (for 32B) LoRA adapters, applied selectively to [MASK] positions via CUDA graph streams, maintaining perfect match at introspect (anchor) positions.

6. Significance and Implications

I-DLM establishes the first instance of a diffusion-style LLM matching the quality of same-scale AR baselines while providing systematic acceleration in highly parallel serving environments. The enforcement of introspective consistency—quantified by the introspective acceptance rate—eliminates the core bottleneck of prior DLM techniques. Architectural alignment with AR inference stacks simplifies deployment, while system-level optimizations (such as stationary batching and kernel fusions) unlock efficient, high-throughput generation suited to real-world workloads (Yu et al., 13 Apr 2026).

A plausible implication is that introspective variants of DLM may further generalize AR-style consistency to multi-modal or multilingual joint models, provided causal training objectives and inference-time verification are preserved. The reduction in required fine-tuning tokens also suggests substantial gains in data and compute efficiency for future model conversions.