--- title: 'DiffusionVL: AR-to-Diffusion VLM' url: https://www.emergentmind.com/topics/diffusionvl type: topic --- # DiffusionVL: AR-to-Diffusion VLM DiffusionVL refers to a class of models and a methodology enabling the transformation of powerful autoregressive (AR) vision-language models (VLMs) into diffusion-based vision-language models (dVLMs) via simple fine-tuning, with the goal of realizing the inherent decoding advantages of the diffusion paradigm in multimodal tasks. By integrating blockwise discrete diffusion with hybrid attention masks and leveraging established AR model initialization, DiffusionVL achieves strong performance in large-scale vision-language understanding and instruction following, surpassing existing open and closed-source diffusion VLMs while dramatically improving inference efficiency and data efficiency [2512.15713]. ## 1. DiffusionVL: Problem Statement and Motivation The mainstream for multimodal generative modeling has historically been the autoregressive sequence-to-sequence transformer. However, autoregressive decoding is inherently sequential, limiting inference speed, parallelism, and flexibility. Diffusion models, especially their discrete variants implemented as iterative denoising or masked-prediction, offer an alternative capable of parallel generation within blocks, dynamic remasking, and higher flexibility in sequence completion and manipulation. Yet, performance of naïve dVLMs has previously lagged behind AR baselines due to weaker base architectures and insufficient scaling. DiffusionVL addresses the core question: “Is it possible to construct dVLMs based on existing powerful AR models?” The approach demonstrates both the practical efficacy and feasibility of translating strong AR VLMs (as well as AR language models) into diffusion models through fine-tuning, producing models that are competitive with or surpass instruction-tuned AR VLMs, with notable improvements in efficiency and scalability [2512.15713]. ## 2. DiffusionVL: Formulation of Masked Discrete Diffusion DiffusionVL adopts the masked discrete diffusion framework for generative modeling over token sequences, as follows: **Forward (noising) process:** Each token \( x_0^i \) from the clean sequence \( x_0 \) is replaced with uniform noise with probability \( \alpha_t \) at time \( t \): \[ q(x_t \mid x_0) = \prod_{i=1}^L \left[ (1 - \alpha_t) \delta(x_t^i = x_0^i) + \alpha_t U(x_t^i) \right] \] where \( U \) denotes the uniform distribution over the vocabulary, and \( \alpha_t \) is a monotonic noising schedule with \( \alpha_0 = 0 \), \( \alpha_1 = 1 \). **Reverse (denoising) process:** The model is trained to reconstruct original tokens from their noisy (masked) counterparts. The prediction at each step aims to reconstruct \( x_0 \) given noised \( x_t \) and optional conditioning \( c \): \[ p_\theta(x_{t-1} \mid x_t, c) = \prod_{i=1}^L p_\theta(x_{t-1}^i \mid x_t, c) \] Training minimizes the cross-entropy loss over masked positions, scaled appropriately by the diffusion schedule: \[ \mathcal{L}_{\mathrm{DM}} = -\mathbb{E}_{t \sim U(0,1),\, x_0,\, x_t} \left[ \frac{1}{\alpha_t} \sum_{i=1}^L \log P_\theta(x_0^i \mid x_t, c) \right] \] **Blockwise decoding:** To achieve both arbitrary-length sequence generation and efficient parallelism, DiffusionVL introduces a block-decoding strategy where sequences are partitioned into \( B = L/D \) blocks of size \( D \). Each block is denoised in parallel, while blocks are decoded in an autoregressive fashion: \[ \mathcal{L}_{\mathrm{BDM}} = -\mathbb{E}\left[ \sum_{i=1}^L w_t \log P_\theta(x_0^i \mid x_{