---
title: Conditional Variational Inference Overview
url: https://www.emergentmind.com/topics/conditional-variational-inference-cvi
type: topic
---

# Conditional Variational Inference Overview

Searching arXiv for the cited papers and closely related CVI terminology to ground the article.
Conditional variational inference is not a single universally standardized algorithmic object. In current usage, the phrase denotes variational methods in which the approximation targets a conditional distribution and explicitly conditions either the variational law, the generative model, or both on observed variables. The literature covered here contains two distinct but related meanings. In one meaning, conditional variational inference refers to amortized conditional posterior estimation with conditional latent-variable models such as conditional variational autoencoders (CVAEs), where the goal is to approximate distributions such as \(p(\mathbf{x}\mid \mathbf{y})\) or \(p(x\mid c)\) by sampling a latent variable and decoding conditionally on the observation [2310.15850], [2605.17377]. In another meaning, “CVI” is an established acronym for “Conjugate-Computation Variational Inference,” a specific method for partly non-conjugate probabilistic models; that algorithm is not the same as conditional variational inference, and several recent papers explicitly warn against conflating the two usages [1703.04265], [2507.10945].

## 1. Terminological scope and competing uses of “CVI”

The most immediate source of confusion is terminological. The paper "Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models" [1703.04265] defines CVI as **Conjugate-Computation Variational Inference**, not conditional variational inference. Its subject is variational inference in models containing both conjugate and non-conjugate terms, using stochastic mirror descent in mean-parameter space together with conjugate updates [1703.04265].

By contrast, several later papers use language that is conditional in a different sense. The dynamic PET paper formulates posterior estimation as approximation of the conditional posterior \(p(\mathbf{x}\mid \mathbf{y})\) with a CVAE and describes the trained decoder as an amortized conditional sampler [2310.15850]. The kilonova paper likewise treats the target as a conditional distribution over physical parameters given a light curve and uses a CVAE for rapid parameter inference [2605.17377]. The multinomial probit paper uses “conditional variational inference” to mean an amortized neural approximation to the conditional posterior of latent utilities given the observed choice and covariates, \(q_{\boldsymbol{\xi}}(\boldsymbol{u}_i\mid \boldsymbol{y}_i,X_i)\) [2507.10945]. CATVI uses “conditional” to describe a structured variational family in which some variational factors remain conditional on adjacent latent variables rather than being fully mean-field [2001.04508].

These usages are related by a common idea: the variational approximation is enriched by preserving or learning dependence on observed variables or neighboring latent variables. A plausible implication is that “conditional variational inference” is best understood as a family of design patterns rather than a single canonical algorithm. The specific mechanism can be amortized latent-variable modeling, structured variational factorization, or a conditional approximation to a latent posterior.

## 2. Core probabilistic idea: approximating conditional posteriors

Across the cited works, the central target is a conditional posterior rather than a marginal latent distribution. In dynamic PET, the hidden variables are kinetic parameters
\[
\mathbf{x}=\{DVR,k_2,R_1\},
\]
the observed variable is the measured time-activity curve \(\mathbf{y}\), and the target posterior is
\[
p(\mathbf{x}\mid \mathbf{y}) \propto p(\mathbf{y}\mid \mathbf{x})\,p(\mathbf{x}).
\]
The paper emphasizes that the inverse map from \(\mathbf{y}\) to \(\mathbf{x}\) is ambiguous because the forward process loses information, so posterior estimation is preferred to deterministic regression [2310.15850].

The kilonova work uses analogous notation. Let \(x\) denote the physical parameter vector and \(c\) the observed light-curve features. The learned object is a conditional generative model for \(x\) given \(c\), mediated by a latent variable \(z\), written as
\[
p_\theta(x\mid c)=\int p_\theta(x\mid z,c)\,p(z\mid c)\,dz,
\]
or, in the simpler CVAE variant reconstructed in the paper,
\[
p_\theta(x\mid c)=\int p_\theta(x\mid z,c)\,p(z)\,dz.
\]
The paper’s deployed model is therefore best interpreted as amortized approximate posterior inference for \(p(x\mid c)\) [2605.17377].

The multinomial probit formulation is different in target but similar in structure. There the latent variables are utility vectors \(\boldsymbol{u}_i\), the observations are \((X_i,\boldsymbol y_i)\), and the central object is the conditional posterior
\[
p(\boldsymbol{u}_i\mid \boldsymbol{y}_i,X_i,\boldsymbol{\theta}),
\]
which is approximated by a shared amortized variational family
\[
q_{\boldsymbol{\xi}}(\boldsymbol{u}_i\mid \boldsymbol{y}_i,X_i).
\]
The conditioning is explicit on the observed choice and covariates, and the structural parameters are estimated jointly with the variational approximation [2507.10945].

CATVI generalizes the same conditional logic to hierarchical Bayesian nonparametric models. Instead of a fully factorized variational family, it uses
\[
q(\beta, \bz, G_0, \bG_{[J]}) = q(\beta)q( G_0)\prod_{j=1}^{J}q(G_j \mid G_0, \bz_j)\prod_{i=1}^{N_j}q(z_{ji}),
\]
so the middle-layer random measures remain conditionally coupled to their parent and child variables [2001.04508].

## 3. CVAE-based conditional variational inference

In the CVAE-based line of work, the defining move is to introduce a latent variable that captures variability remaining after conditioning on the observation. In dynamic PET, the paper motivates \(\mathbf{z}\) as capturing “the loss of information in the forward process,” with prior
\[
p(\mathbf{z})=\mathcal{N}(0,I),
\]
encoder
\[
q_\phi(\mathbf{z}\mid \mathbf{x},\mathbf{y}) \sim \mathcal{N}(\mathbf{z};\mu,\sigma^2),
\]
and decoder
\[
\mathbf{x} = \mathrm{Decoder}(\mathbf{y},\mathbf{z}).
\]
The trained decoder is interpreted as representing the posterior \(p(\mathbf{x}\mid \mathbf{y})\) by pushing forward latent samples through a \(\mathbf{y}\)-dependent deterministic map [2310.15850].

The corresponding conditional ELBO is written as
\[
\mathcal{L}(\mathbf{x},\mathbf{y};\phi,\psi) = \mathbb{E}_{q_\phi(\mathbf{z}\mid \mathbf{x},\mathbf{y})} \big[\log p_\psi(\mathbf{x}\mid \mathbf{y},\mathbf{z})\big] - \mathrm{KL}\!\left(q_\phi(\mathbf{z}\mid \mathbf{x},\mathbf{y})\,\|\,p(\mathbf{z})\right),
\]
and implemented as
\[
\mathcal{L}=\mathcal{L}_{MSE}+\mathcal{L}_{KL}.
\]
The encoder and decoder are simple multilayer perceptrons with three fully connected layers each, the latent dimension is \(M_z=10\), and inference consists of fixing \(\mathbf{y}^\ast\), sampling \(\mathbf{z}\sim\mathcal N(0,I)\), and decoding repeatedly to obtain an empirical approximate posterior [2310.15850].

The kilonova paper follows the same CVAE pattern. Its encoder maps \((x,c)\) to a Gaussian latent distribution,
\[
q_\phi(z\mid x,c)=\mathcal N\!\bigl(z;\mu_\phi(x,c),\operatorname{diag}(\sigma_\phi^2(x,c))\bigr),
\]
with two-dimensional latent variable \(z\in\mathbb R^2\). The decoder defines \(p_\theta(x\mid z,c)\), and the reconstructed training loss is the familiar decomposition
\[
\mathcal L = \mathcal L_{\rm recon} + \mathcal L_{\rm KL},
\]
where the reconstruction term corresponds to MSE on the four-dimensional parameter vector [2605.17377].

Both papers use the reparameterization trick,
\[
\mathbf{z}=\mu+\sigma\odot\epsilon,\qquad \epsilon\sim \mathcal{N}(0,I),
\]
or the equivalent \(z = \mu + \sigma \odot \epsilon\), to enable gradient-based optimization through stochastic latent samples [2310.15850], [2605.17377]. In both cases, uncertainty is represented implicitly by latent sampling rather than by an explicitly flexible closed-form posterior density.

A recurring interpretation in this line of work is that the decoder acts as an amortized conditional sampler. This suggests a transport view of conditional variational inference: a simple latent prior is pushed through a condition-dependent nonlinear map to induce an implicit distribution on the target variable.

## 4. Conditional variational approximations beyond CVAEs

Conditional variational inference also appears outside the CVAE framework. The multinomial probit paper is the clearest example. Its latent utility model is
\[
\boldsymbol{u_i} = X_i \boldsymbol{a}  + \boldsymbol{\varepsilon_i}, \qquad \boldsymbol{\varepsilon_i} \sim \mathrm{MVN}\left(\mathbf{0}, \Sigma\right),
\]
and the observed choice is determined by the argmax of utilities [2507.10945]. Direct posterior inference is difficult because conditioning on the observed choice produces high-dimensional truncated Gaussian distributions. The proposed remedy is to approximate the conditional posterior of utilities by a neural Gaussian encoder:
\[
\boldsymbol{u}_i|\boldsymbol{y}_i, X_i \sim MVN \left( \boldsymbol{\mu}_{\boldsymbol \xi}(\boldsymbol{y}_i, X_i; \boldsymbol{\xi}), \Sigma_{\boldsymbol \xi}(\boldsymbol{y}_i, X_i; \boldsymbol{\xi}) \right).
\]

Its objective is the negative ELBO
\[
\mathcal{L}(\boldsymbol{\xi},\boldsymbol{\theta})=
- \mathbb{E}_{q_{\boldsymbol{\xi}}(\boldsymbol{u}_i \vert \boldsymbol{y}_i,X_i)}\left[\log p(\boldsymbol{y}_i \vert \boldsymbol{u}_i)\right]
+  D_{KL}\left[q_{\boldsymbol{\xi}}(\boldsymbol{u}_i \vert \boldsymbol{y}_i,X_i)\, \|\, p(\boldsymbol{u}_i \vert \boldsymbol{\theta}, X_i)\right],
\]
with an identified version computed in differenced utility space [2507.10945]. The method’s distinctive ingredients are a full correlated Gaussian variational family, an \(LDL^\top\) covariance parameterization to guarantee positive definiteness, trace-normalized covariance regularization for identification, and straight-through estimation with Gumbel-Softmax to handle the argmax choice map without truncated Gaussian sampling [2507.10945].

CATVI provides a different non-amortized conditional construction. Its key variational factor
\[
q(G_j \mid G_0,\bz_j)
\]
restores dependence between group-level random measures, the top-level random measure, and local assignments. The paper shows that the optimal conditional factor is exactly
\[
q(G_j \mid G_0, \bz_j)=p(G_j \mid G_0, \bz_j),
\]
which is a structurally conditional variational move rather than a learned amortized one [2001.04508]. The corresponding objective is a nonparametric evidence lower bound satisfying
\[
\KL \big(Q (Z) \parallel P(Z | X)\big) + \text{NPELBO} = \log p(X).
\]

A nearby but distinct methodology appears in "Gradient-free variational learning with conditional mixture networks" [2408.16429]. That paper is not about the specific CVI algorithm and is described instead as a conditionally conjugate CAVI method. Its relevance lies in the fact that the approximate posterior is structured rather than fully mean-field,
\[
p(\mathbf{X}_1,\mathbf{Z}_1,\mathbf{\Theta}\mid Y,\mathbf{X}_0) \approx q(\mathbf{\Theta}) \prod_{n=1}^N q(z_1^n)\,q(\mathbf{x}_1^n\mid z_1^n),
\]
and Pólya–Gamma augmentation restores conjugacy for multinomial logistic blocks, enabling analytic coordinate-ascent updates [2408.16429]. A plausible implication is that conditional variational structure and conditional conjugacy are often complementary rather than competing ideas.

## 5. Objectives, architectures, and optimization mechanisms

The concrete implementations differ, but their objectives share a recognizable template: a reconstruction or likelihood-matching term plus a regularizer that ties the variational distribution to a prior or structural model.

The table below summarizes the main instantiations discussed in the cited papers.

| Setting | Conditional target | Main variational mechanism |
|---|---|---|
| Dynamic PET [2310.15850] | \(p(\mathbf{x}\mid \mathbf{y})\) | CVAE with Gaussian latent prior and MSE + KL loss |
| Kilonova inference [2605.17377] | \(p(x\mid c)\) | CVAE with Gaussian latent space and MSE-based reconstruction |
| Multinomial probit [2507.10945] | \(p(\boldsymbol{u}_i\mid \boldsymbol{y}_i,X_i,\boldsymbol{\theta})\) | Amortized neural Gaussian encoder with analytic Gaussian KL |
| HBNP topic models [2001.04508] | Structured posterior over \(G_0,G_j,z_{ji}\) | Conditional variational factorization and adaptive truncation |
| Conditional mixture networks [2408.16429] | Structured posterior over local latents and parameters | Conditionallly conjugate CAVI with Pólya–Gamma augmentation |

In the CVAE papers, the optimization is standard stochastic gradient training with the reparameterization trick [2310.15850], [2605.17377]. In the multinomial probit paper, the encoder is amortized but the structural parameters are also learned jointly, with minibatch stochastic optimization, ADAM, GPU-based parallelism, and annealed Gumbel temperature [2507.10945]. In CATVI, Monte Carlo Gibbs sampling of local assignments is embedded inside stochastic variational updates of global factors, and adaptive truncation expands posterior dimension when sampled assignments enter the residual component \(\phi_0\) [2001.04508]. In the conditional mixture network paper, coordinate ascent replaces gradient-based ELBO optimization once augmentation furnishes conditionally conjugate Gaussian blocks [2408.16429].

One substantive difference concerns what receives a full posterior approximation. The PET and kilonova CVAE papers approximate conditional posteriors over target parameters and deploy the decoder as a sampler [2310.15850], [2605.17377]. The multinomial probit paper approximates the posterior of latent utilities but returns point estimates for structural parameters \(\boldsymbol\theta^\Delta\), with uncertainty quantification delegated to bootstrap rather than being native to the objective [2507.10945]. CATVI approximates a structured posterior process but still factorizes some components, so it is not a full posterior coupling [2001.04508].

## 6. Empirical roles, benefits, and limitations

The empirical motivation is almost always computational. In dynamic PET, MCMC is treated as an unbiased asymptotic reference posterior, but it is expensive. For the single-voxel SRTM setting, PyMC-based MCMC takes about 10 minutes to infer 90,000 samples, whereas the trained CVAE can infer 90,000 samples in about 30 seconds; the paper emphasizes that MCMC must be rerun for each new \(\mathbf y^\ast\), while the CVAE amortizes this cost across all future test examples once training is complete [2310.15850].

The kilonova paper makes the same amortization argument. Training completes in about \(\sim 1\) hour on CPU only, generating 1000 physical-parameter samples across models and filters takes about \(\sim 10\) ms, and the abstract states that the total time from training to parameter inference is under \(\approx 3\) h [2605.17377].

The multinomial probit results emphasize scale in both sample size and number of alternatives. The proposed method calibrates MNP parameters with 20 alternatives and one million observations in approximately 28 minutes, roughly 36 times faster and more accurate than the existing benchmarks in recovering model parameters [2507.10945]. Its main computational gain comes from eliminating truncated Gaussian sampling altogether by replacing exact conditioning on the argmax event with unconstrained Gaussian encoder samples and differentiable surrogate decoding [2507.10945].

CATVI’s empirical claim is that conditional structure and adaptive truncation improve fit in hierarchical Bayesian nonparametric models. On arXiv, NYT, and Wiki, CATVI applied to HDP and \(\Gamma\)DP yields lower perplexity than OVI, MOVI, SMVI, and Gibbs sampling, and the number of topics rises from the initial value and then stabilizes [2001.04508].

These gains are paired with recurrent caveats. The PET paper notes that the variational family is restricted by a Gaussian latent prior and a diagonal Gaussian encoder posterior, the decoder is trained with MSE rather than a richer explicit conditional density, the latent dimension \(M_z=10\) is an empirical choice, training and testing both use the same forward/noise model, and experiments are limited to low-dimensional posterior estimation for a single region or voxel [2310.15850]. The kilonova paper similarly identifies sparse and discrete training grids, a two-dimensional latent bottleneck, and lack of guaranteed calibrated posteriors as limiting factors [2605.17377]. The multinomial probit paper notes approximation error from the Gaussian variational family, possible bias from the Gumbel-Softmax relaxation and straight-through mismatch, point estimation rather than full posterior inference for structural parameters, and potential amortization gap [2507.10945]. CATVI explicitly states that estimation of the number of topics is not consistent [2001.04508].

A plausible implication is that conditional variational inference systematically exchanges some form of exactness for reuse: the posterior approximation cost is paid in optimization or training, then reused across observations or throughout a hierarchy.

## 7. Relationship to conjugate-computation CVI and broader variational inference

Because the acronym CVI is overloaded, the distinction from conjugate-computation CVI is essential. The 2017 CVI paper considers models decomposable as
\[
p(\mathbf{y},\mathbf{z}) \propto p_{nc}(\mathbf{y},\mathbf{z})\, p_c(\mathbf{y},\mathbf{z}),
\]
with an exponential-family variational approximation
\[
q(\mathbf{z}\mid\lambda) = h(\mathbf{z}) \exp\left\{\langle \phi(\mathbf{z}), \lambda\rangle - A(\lambda)\right\}.
\]
Its core update accumulates stochastic gradients of the expected non-conjugate log-term in mean-parameter space and combines them with conjugate natural parameters through
\[
\widetilde{\lambda}_t = (1-\beta_t)\widetilde{\lambda}_{t-1} + \beta_t \widehat{\nabla}_{\mu} \mathbb{E}_q[\log p_{nc}] \big|_{\mu=\mu_t},
\qquad
\lambda_{t+1} = \eta + \widetilde{\lambda}_t,
\]
so that each step can be interpreted as inference in a conjugate model [1703.04265].

This algorithm is methodologically different from the conditional approaches above. The dynamic PET and kilonova papers are standard CVAE constructions for supervised conditional generation or inverse inference, not mirror-descent CVI [2310.15850], [2605.17377]. The multinomial probit paper states explicitly that “CVI” there does not mean the conjugate-computation framework from the broader variational literature; instead it means a conditional variational approximation to the posterior of latent utilities [2507.10945]. CATVI likewise uses “conditional” in a model-specific sense of conditioning variational laws of middle-layer random measures on neighboring latent variables, not in the sense of the Khan and Lin algorithm [2001.04508]. The conditional mixture network paper is related to CVI-style conjugacy exploitation but is described as CAVI rather than the specific CVI algorithm [2408.16429].

The broader methodological connection is still clear. All these methods enlarge or restructure the variational family to better align with posterior dependence while maintaining tractability. In some cases, tractability comes from amortized neural sampling [2310.15850], [2605.17377]; in others, from structured conditional Gaussian families and differentiable relaxations [2507.10945]; in others, from exact conditional factors and adaptive truncation [2001.04508]; and in others, from conditional conjugacy restored through augmentation [2408.16429].

Taken together, the literature supports a precise but plural definition: conditional variational inference is a class of variational strategies in which the approximation is explicitly conditional—on observations, on neighboring latent variables, or on both—and is designed to represent ambiguity, residual uncertainty, or hierarchical dependence that would be lost under deterministic inversion or fully factorized mean-field approximations.

Source: https://www.emergentmind.com/topics/conditional-variational-inference-cvi