DiffusionVL: AR-to-Diffusion VLM

• DiffusionVL is a methodology that transforms AR vision-language models into diffusion-based models through fine-tuning and blockwise decoding.
• It leverages masked discrete diffusion with hybrid attention masks to achieve up to 2x inference speedup and improved data efficiency.
• Benchmark results indicate DiffusionVL outperforms traditional AR models, paving the way for scalable, efficient multimodal generation.

DiffusionVL refers to a class of models and a methodology enabling the transformation of powerful autoregressive (AR) vision-LLMs (VLMs) into diffusion-based vision-LLMs (dVLMs) via simple fine-tuning, with the goal of realizing the inherent decoding advantages of the diffusion paradigm in multimodal tasks. By integrating blockwise discrete diffusion with hybrid attention masks and leveraging established AR model initialization, DiffusionVL achieves strong performance in large-scale vision-language understanding and instruction following, surpassing existing open and closed-source diffusion VLMs while dramatically improving inference efficiency and data efficiency (Zeng et al., 17 Dec 2025).

1. DiffusionVL: Problem Statement and Motivation

The mainstream for multimodal generative modeling has historically been the autoregressive sequence-to-sequence transformer. However, autoregressive decoding is inherently sequential, limiting inference speed, parallelism, and flexibility. Diffusion models, especially their discrete variants implemented as iterative denoising or masked-prediction, offer an alternative capable of parallel generation within blocks, dynamic remasking, and higher flexibility in sequence completion and manipulation. Yet, performance of naïve dVLMs has previously lagged behind AR baselines due to weaker base architectures and insufficient scaling.

DiffusionVL addresses the core question: “Is it possible to construct dVLMs based on existing powerful AR models?” The approach demonstrates both the practical efficacy and feasibility of translating strong AR VLMs (as well as AR LLMs) into diffusion models through fine-tuning, producing models that are competitive with or surpass instruction-tuned AR VLMs, with notable improvements in efficiency and scalability (Zeng et al., 17 Dec 2025).

2. DiffusionVL: Formulation of Masked Discrete Diffusion

DiffusionVL adopts the masked discrete diffusion framework for generative modeling over token sequences, as follows:

Forward (noising) process: Each token $x_0^i$ from the clean sequence $x_0$ is replaced with uniform noise with probability $\alpha_t$ at time $t$:

$$q(x_t \mid x_0) = \prod_{i=1}^L \left[ (1 - \alpha_t) \delta(x_t^i = x_0^i) + \alpha_t U(x_t^i) \right]$$

where $U$ denotes the uniform distribution over the vocabulary, and $\alpha_t$ is a monotonic noising schedule with $\alpha_0 = 0$, $\alpha_1 = 1$.

Reverse (denoising) process: The model is trained to reconstruct original tokens from their noisy (masked) counterparts. The prediction at each step aims to reconstruct $x_0$ given noised $x_0$0 and optional conditioning $x_0$1:

$x_0$2

Training minimizes the cross-entropy loss over masked positions, scaled appropriately by the diffusion schedule:

$x_0$3

Blockwise decoding: To achieve both arbitrary-length sequence generation and efficient parallelism, DiffusionVL introduces a block-decoding strategy where sequences are partitioned into $x_0$4 blocks of size $x_0$5. Each block is denoised in parallel, while blocks are decoded in an autoregressive fashion:

$x_0$6

where $x_0$7 denotes the tokens in the same block as position $x_0$8, and $x_0$9 the clean tokens in earlier blocks. $\alpha_t$0 provides schedule-dependent weighting.

3. Methodology: AR-to-DiffusionVL Translation and Block-Decoding

The DiffusionVL procedure translates strong pretrained AR VLMs into dVLMs through an end-to-end fine-tuning process on block-diffusion objectives:

• Translation from AR-VLM to dVLM: Initialize from a vision-language AR transformer (e.g., Qwen2.5-VL), and fine-tune all parameters on the block-diffusion masked denoising objective.
• Translation from AR-LM to dVLM (modality + paradigm shift): First, learn a vision-text connector (small MLP) mapping image embeddings to text-space with the AR loss. Then, perform diffusion fine-tuning end-to-end, optimizing the block diffusion objective. No architectural changes or tokenizer adaptation is required; image tokens are simply prepended via the connector (Zeng et al., 17 Dec 2025).

Inference combines autoregressive block scheduling for arbitrary-length outputs with intra-block parallel diffusion denoising, each block initialized with all-noise and gradually denoised over $\alpha_t$1 steps. Key-value caches from previous blocks are reused, significantly speeding up decoding.

4. Benchmarking, Ablation, and Efficiency

Despite using <5% of the training data used by previous dVLMs, DiffusionVL achieves state-of-the-art results on several multimodal benchmarks:

Model	MMMU-Pro[vision]	Δ vs LLaDA-V	MME[cog.]	Δ vs LLaDA-V
LLaDA-V (8B,Diff)	18.6	–	491	–
DiffusionVL-7B	25.0	+34.4%	675	+37.5%

On large-scale tests (MMMBench, SeedBench, RealWorldQA, Muirbench, etc.), DiffusionVL-7B closes the gap to advanced AR VLMs, outperforming other open-source diffusion VLMs (Zeng et al., 17 Dec 2025).

Ablation studies:

• Optimal block size is $\alpha_t$2; smaller blocks increase quality at the expense of parallelism, larger blocks offer better speedup with small accuracy drop.
• Number of diffusion steps $\alpha_t$3 per block offers a tradeoff; more steps improve output quality (higher BERTScore), but reduce inference speed.
• Static versus dynamic remasking: dynamic schemes decode all tokens above a threshold confidence for further speedup at modest quality cost.

Efficiency:

• DiffusionVL achieves a $\alpha_t$4 inference speedup over auto-regressive and prior diffusion baselines (e.g., LLaDA-V-8B).
• Memory and compute cost are minimized via blockwise decoding and KV cache reuse.
• Code and models are openly released at https://github.com/hustvl/DiffusionVL.

5. Comparative Framework and Architectural Advances

DiffusionVL is related to advances in blockwise and hybrid-discrete diffusion models in VLU:

• SDAR-VL (Cheng et al., 16 Dec 2025) demonstrates complementary methods for stable and training-efficient blockwise diffusion, introducing asynchronous blockwise noise scheduling, mask ratio scaling, and progressive beta noise curriculum.
• Unified Diffusion VLA (UD-VLA) (Chen et al., 3 Nov 2025) explores joint discrete denoising for vision-language-action, but requires different tokenization and hybrid attention mechanisms.
• DiffusionVL is unique in providing a direct AR-to-diffusion translation path for any pretrained AR VLM, requiring no architectural changes and supporting arbitrary-length output via blockwise autoregressive scheduling (Zeng et al., 17 Dec 2025).
• Comparative analysis indicates any strong AR-LM is a better seed for dVLMs than diffusion-pretrained LLMs, even for strictly multimodal objectives.

6. Limitations, Open Problems, and Future Directions

Limitations of DiffusionVL include modest quality loss for very large blocks (due to lower autoregressive dependency) and possible room for further improvement by exploring custom noise schedules beyond uniform masking. The use of blockwise parallelism is naturally compatible with larger model scaling and could benefit from further research in hybrid AR–diffusion architectures, dynamic block sizing, and non-uniform noise scheduling.

Open questions identified in (Zeng et al., 17 Dec 2025) include:

• The potential gains from larger base AR-LMs (e.g., Qwen-15B or GPT-class models).
• The impact of advanced noise schedules and the extension of block diffusion to further modalities (audio, video) and reasoning tasks.
• Richer integration with multi-modal chain-of-thought and interpretable reasoning, analogous to the development in SDAR-VL and Unified Diffusion VLA (Cheng et al., 16 Dec 2025, Chen et al., 3 Nov 2025).

7. Impact and Significance

DiffusionVL represents a shift in vision-language modeling, showing that the paradigm migration from AR to diffusion is not only feasible but highly effective for strong multimodal instruction understanding and generation. The resulting dVLMs exhibit competitive or superior performance to AR models while offering up to $\alpha_t$5 inference speedup, arbitrarily long sequence support, and a data-efficient training pipeline. This suggests diffusion-based decoding is a practical, scalable route for next-generation VLMs and highlights the translational potential of mature AR model ecosystems to new generative paradigms (Zeng et al., 17 Dec 2025).

---

For additional details, methodologies, and code, refer to (Zeng et al., 17 Dec 2025).