Variational Diffusion Model

• Variational diffusion model is a likelihood-based framework formulated as an infinitely deep variational autoencoder with continuous-time latent diffusion.
• It employs a fixed Gaussian forward process and a reverse denoising chain to achieve tractable ELBO optimization and effective schedule learning.
• Extensions including learned encoders, conditional designs, and diffusion distillation broaden its applications to inverse problems and specialized probabilistic models.

Searching arXiv for the cited variational diffusion papers to ground the article in current literature. A variational diffusion model usually denotes a diffusion model formulated as a variational latent-variable model. In its canonical form, it is a likelihood-based diffusion generative model expressed as a variational autoencoder with infinitely many latent layers in the continuous-time limit, where a fixed Gaussian forward noising process plays the role of the variational posterior and the reverse denoising chain defines the generative model (Kingma et al., 2021). Later treatments recast the same construction as a particular Markovian hierarchical variational autoencoder, emphasizing that diffusion can be derived entirely from directed graphical modeling and variational inference rather than from non-equilibrium thermodynamics (Luo, 2022, Ribeiro et al., 2024).

1. Hierarchical variational interpretation

The variational interpretation starts from a hierarchical latent-variable model with a chain of latent variables and a reverse-time generative process. In the presentation that unifies diffusion with a Markovian hierarchical VAE, the generative model is written as

$$p(x,z_{1:T}) = p(z_T)\,p_\theta(x\mid z_1)\prod_{t=2}^{T} p_\theta(z_{t-1}\mid z_t),$$

with a corresponding variational posterior over the latent chain. A variational diffusion model is obtained by imposing three restrictions: the latent dimension equals the data dimension, the encoder transitions are fixed linear Gaussians rather than learned flexible posteriors, and the terminal latent is chosen so that it is standard Gaussian (Luo, 2022).

Under this specialization, the latent hierarchy is no longer a stack of progressively more abstract representations. Instead, each latent variable is a noisified version of the same datum. The forward marginal is

$$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$

and the signal-to-noise ratio is

$\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$

The schedule is chosen so that the noisiest latent is approximately or exactly standard Gaussian, which makes the aggregate posterior match the prior by construction (Kingma et al., 2021, Ribeiro et al., 2024).

This hierarchical view also clarifies why diffusion models fit naturally into ELBO-based likelihood training. The forward process serves as a fixed inference model, the reverse process serves as the decoder, and the hierarchy can be interpreted as an infinitely deep VAE in continuous time. A recurrent theme in expository work is that this interpretation removes much of the apparent conceptual gap between diffusion, VAEs, and score-based models (Ribeiro et al., 2024).

2. Forward diffusion, reverse denoising, and equivalent parameterizations

The forward process is Gaussian and Markovian. For arbitrary times $s<t$, the transition is

$$q(z_t \mid z_s) = \mathcal N\!\left(z_t;\alpha_{t\mid s}z_s,\sigma_{t\mid s}^2 I\right), \qquad \alpha_{t\mid s}=\frac{\alpha_t}{\alpha_s},\quad \sigma_{t\mid s}^2=\sigma_t^2-\alpha_{t\mid s}^2\sigma_s^2.$$

Because the model is linear-Gaussian in the forward direction, the top-down posterior is also Gaussian: $$q(z_s \mid z_t,x)=\mathcal N\!\left(z_s;\mu_Q(z_t,x;s,t),\sigma_Q^2(s,t)I\right).$$ This tractable posterior is the central algebraic fact behind variational diffusion training (Ribeiro et al., 2024).

The reverse generative transition is then defined by replacing the unknown clean datum with a learned predictor: $$p(z_s\mid z_t)= q\!\left(z_s\mid z_t,\; x=\hat{x}(z_t;t)\right).$$ The same model can be parameterized in three equivalent ways: by predicting the clean sample $\hat{x}(z_t;t)$, by predicting the source noise $\hat{\epsilon}(z_t;t)$, or by predicting the score function of the noisy marginal. The relation between denoising and noise prediction is

$$\hat{x}(z_t;t) = \frac{z_t - \sigma_t \hat{\epsilon}(z_t;t)}{\alpha_t},$$

and expository derivations explicitly connect the score parameterization to the denoising one through Tweedie’s Formula (Kingma et al., 2021, Luo, 2022).

One common misconception is that these are different model classes. In the variational account they are alternative parameterizations of the same Gaussian denoising problem, differing mainly in the target predicted by the network and in the form of the weighted loss (Luo, 2022).

3. ELBO structure, SNR simplification, and schedule learning

The variational lower bound decomposes into prior, reconstruction, and diffusion terms. In the formulation used for likelihood estimation,

$$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$0

where the diffusion term is a sum of KL divergences between the true reverse posterior and the learned reverse transition (Kingma et al., 2021).

A central result of the original VDM paper is that the diffusion term simplifies to a weighted denoising MSE whose weight is determined by SNR differences. In discrete time,

$$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$1

with $$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$2. In continuous time this becomes

$$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$3

After a change of variables from $$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$4 to $$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$5, the continuous-time objective depends only on the endpoint SNR values, not on the detailed shape of the schedule (Kingma et al., 2021).

This invariance is the basis for schedule learning. The variance-preserving parameterization used in VDM is

$$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$6

The schedule can therefore be optimized jointly with the denoiser to reduce estimator variance without changing the underlying continuous-time model class (Kingma et al., 2021).

Later conditional work qualifies the invariance claim in an important way. In “Conditional Variational Diffusion Models,” the schedule is learned as part of the conditional model and is regularized because discrete implementations can otherwise exploit degenerate, overly abrupt schedules. The paper introduces

$$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$7

to discourage pathological schedules, and explicitly argues that fine-tuning schedules by experimentation should be avoided because they can be learned during training in a stable way (Maggiora et al., 2023). This suggests that schedule learning has two distinct roles in the literature: variance control in the continuous-time VDM sense, and numerical regularization in discrete conditional implementations.

4. Learned encoders, finite-depth weighting, and modified VDM objectives

A major extension of the original VDM framework is to relax the assumption that the forward process must noise the raw datum directly. DiffEnc replaces $$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$8 with a learned, depth-dependent encoding $$q(z_t \mid x) = \mathcal N\!\left(\alpha_t x,\; \sigma_t^2 I\right),$$9 and defines

$\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$0

This changes the true reverse conditional and therefore changes the diffusion KL term. The paper compensates by modifying the generative mean, preserving the hierarchical VAE interpretation while adding flexibility to the forward process (Nielsen et al., 2023).

DiffEnc also studies the ratio between reverse-encoder variance and generative variance,

$\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$1

and shows that its role depends on depth. For finite depth, $\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$2 can be treated as a free weight and yields a weighted diffusion loss. For the infinite-depth hierarchy, the ELBO is only well-defined if $\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$3, equivalently $\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$4 (Nielsen et al., 2023). This is one of the clearest theoretical statements in the literature about when weighted diffusion objectives remain genuine ELBOs and when they cease to have a proper continuous-time limit.

Empirically, the learned encoder improves likelihood on CIFAR-10 in the reported larger-model comparison:

• VDMv-32: $\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$5 BPD
• DiffEnc-32-4: $\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$6 BPD

The paper reports $\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$7 in a t-test over seeds, and notes that the encoder is used only during training, so the likelihood gain does not impose a sampling-time penalty (Nielsen et al., 2023).

5. Conditional models, inverse problems, and diffusion as variational inference

The phrase “variational diffusion model” now covers a broader family of conditional and inference-oriented constructions. In CVDM, the unconditional VDM is extended to conditional inverse problems by modeling $\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$8 and learning a conditional schedule that may depend on the observation and even vary across pixels. The forward process is written as

$\mathrm{SNR}(t)=\alpha_t^2/\sigma_t^2.$9

with $s<t$0, and the continuous-time schedule is factorized through

$s<t$1

The model is trained by a VDM-style ELBO, regularized to avoid degenerate schedules, and is explicitly motivated by probabilistic conditioning for ill-posed inverse problems (Maggiora et al., 2023).

A different variational use of diffusion appears in RED-Diff. Rather than approximating the posterior score $s<t$2, RED-Diff introduces a variational posterior

$s<t$3

and minimizes $s<t$4. This yields an objective comprising a measurement-fitting term and a diffusion-based regularizer, with an SNR-based weighting rule

$s<t$5

The paper interprets the resulting method as “regularization by denoising diffusion process,” where denoisers at different timesteps impose different structural constraints over the image (Mardani et al., 2023).

Diffusion can also appear on the encoder side as an expressive approximate posterior. DDVI introduces auxiliary latent variables and a diffusion chain in latent space,

$s<t$6

and combines an ELBO with a wake-sleep-inspired denoising diffusion term. The diffusion posterior is presented as an expressive variational family that performs iterative refinement in latent space and is used as an approximate posterior rather than as a generative decoder (Piriyakulkij et al., 2024).

These lines of work correct another common misunderstanding: in the literature, “variational diffusion” no longer refers only to unconditional density estimation. It also denotes conditional inverse solvers, schedule-learning conditional models, and diffusion-based approximate posteriors.

6. Distillation, domain-specific variants, and present limitations

Variational constructions have also been used directly in diffusion-space distillation. VarDiU introduces a variational upper bound for one-step diffusion distillation. In the standard setting, the student is an implicit one-step generator

$s<t$7

and prior methods minimize the diffusive reverse KL

$s<t$8

VarDiU replaces this with a variational upper bound

$s<t$9

which is tight when the variational posterior matches the true posterior. Its main practical claim is that the resulting gradient estimator is unbiased with respect to the optimized objective because it uses the teacher score with stop-gradient and does not require a DSM-trained student score model (Wang et al., 28 Aug 2025).

Domain-specific papers have adopted variational diffusion terminology in more task-specialized forms. AT-VarDiff combines a DDPM-style restoration model with a VAE-like latent prior $$q(z_t \mid z_s) = \mathcal N\!\left(z_t;\alpha_{t\mid s}z_s,\sigma_{t\mid s}^2 I\right), \qquad \alpha_{t\mid s}=\frac{\alpha_t}{\alpha_s},\quad \sigma_{t\mid s}^2=\sigma_t^2-\alpha_{t\mid s}^2\sigma_s^2.$$0 for atmospheric turbulence correction, using a synthetic dataset with 1,000,000 training pairs and 2,500 test images. The reported perceptual metrics are LPIPS = 0.1094 and FID = 32.69, compared with LPIPS = 0.1923, FID = 60.87 for a conditional-diffusion baseline and LPIPS = 0.2150, FID = 80.05 for AT-DDPM (Wang et al., 2023).

In communication, VCDC uses the VDM-style Gaussian family

$$q(z_t \mid z_s) = \mathcal N\!\left(z_t;\alpha_{t\mid s}z_s,\sigma_{t\mid s}^2 I\right), \qquad \alpha_{t\mid s}=\frac{\alpha_t}{\alpha_s},\quad \sigma_{t\mid s}^2=\sigma_t^2-\alpha_{t\mid s}^2\sigma_s^2.$$1

to match AWGN corruption in LLR space and integrates belief propagation into the reverse denoising step. For LDPC $$q(z_t \mid z_s) = \mathcal N\!\left(z_t;\alpha_{t\mid s}z_s,\sigma_{t\mid s}^2 I\right), \qquad \alpha_{t\mid s}=\frac{\alpha_t}{\alpha_s},\quad \sigma_{t\mid s}^2=\sigma_t^2-\alpha_{t\mid s}^2\sigma_s^2.$$2, the reported complexity numbers are 316.4K FLOPs for BP, 1.6G FLOPs for HGN, 140.3G FLOPs for DDECC-Max, and 377.6K FLOPs with 264 B model size for Ours-20; the same comparison reports BER at 6 dB of $$q(z_t \mid z_s) = \mathcal N\!\left(z_t;\alpha_{t\mid s}z_s,\sigma_{t\mid s}^2 I\right), \qquad \alpha_{t\mid s}=\frac{\alpha_t}{\alpha_s},\quad \sigma_{t\mid s}^2=\sigma_t^2-\alpha_{t\mid s}^2\sigma_s^2.$$3 for BP and $$q(z_t \mid z_s) = \mathcal N\!\left(z_t;\alpha_{t\mid s}z_s,\sigma_{t\mid s}^2 I\right), \qquad \alpha_{t\mid s}=\frac{\alpha_t}{\alpha_s},\quad \sigma_{t\mid s}^2=\sigma_t^2-\alpha_{t\mid s}^2\sigma_s^2.$$4 for Ours-20 (Zhang et al., 17 May 2026).

The main limitations identified across the literature remain heterogeneous but consistent. Original VDMs optimize likelihood effectively, but later work shows that discrete schedules still require careful regularization (Kingma et al., 2021, Maggiora et al., 2023). DiffEnc gains flexibility but also shows that unequal reverse and encoder variances are problematic in the infinite-depth limit (Nielsen et al., 2023). VarDiU removes one source of gradient bias in distillation, yet its experiments are limited to a 2D 40-component Gaussian mixture and still assume access to a teacher score or a good approximation (Wang et al., 28 Aug 2025). Application-specific models such as AT-VarDiff rely on synthetic data or simulator-side degradation parameters (Wang et al., 2023). Taken together, these results define the present scope of variational diffusion modeling: a general variational framework for diffusion processes, coupled with a growing set of specialized objectives whose theoretical cleanliness and empirical advantages depend on the exact choice of latent hierarchy, schedule parameterization, and inference target.