---
title: Variational Rejection Sampling (VRS)
url: https://www.emergentmind.com/topics/variational-rejection-sampling-vrs
type: topic
---

# Variational Rejection Sampling (VRS)

Variational Rejection Sampling (VRS) is a hybrid framework for approximate inference that refines a parametric variational proposal by integrating a rejection sampling mechanism. The approach systematically enhances the fidelity of variational approximations to complex target distributions, particularly in latent-variable models. By combining properties of traditional rejection sampling with variational inference objectives—often via α-divergence or ELBO minimization—VRS bridges the gap between efficient proposal construction and principled sample-based correction. Extensions such as Refined α-Divergence Rejection Sampling (α-DRS) and Reparameterized Variational Rejection Sampling (RVRS) further broaden the practical and theoretical impact of this scheme in probabilistic modeling [1804.01712, 1909.07627, 2309.14612].

## 1. Theoretical Foundations and Model Setup

VRS addresses inference for latent variable models, where the joint distribution takes the form $p_\theta(x, z) = p_\theta(z)p_\theta(x | z)$, with $x$ observed and $z$ latent. The key challenge is constructing an efficient, high-fidelity posterior approximation $q_\phi(z \mid x)$, especially when the true posterior $p_\theta(z \mid x)$ exhibits complex, multimodal, or heavy-tailed structure that standard variational families fail to capture [1804.01712, 2309.14612].

The essential idea is to build an improved variational family by "resampling" or "refining" $q_\phi$ via an accept-reject process. This yields a new density:
\[
r(z \mid x) \propto q_\phi(z \mid x) a(z \mid x),
\]
where the acceptance function $a(z \mid x)$ depends on both the model and proposal densities, ensuring that as the acceptance criterion is tightened, the refined $r(z\mid x)$ approaches $p_\theta(z\mid x)$ [1804.01712].

## 2. Core Algorithm and Mathematical Framework

The canonical VRS acceptance probability follows the (hard) rejection sampling prescription:
\[
a_M(z\mid x) = \min\left\{1, \frac{p_\theta(x, z)}{M q_\phi(z\mid x)}\right\},
\]
where $M$ is a normalization constant bounding $p_\theta(x, z)/q_\phi(z \mid x)$. In practice, to circumvent the need for a strict bound, VRS employs a "soft" acceptance threshold $T$, yielding
\[
a_{\theta,\phi}(z\mid x, T) = \exp(-[l_{\theta,\phi}(z\mid x, T)]^+),
\]
with $l_{\theta,\phi}(z\mid x, T) = -\log p_\theta(x, z) + \log q_\phi(z\mid x) - T$ and $[\cdot]^+ = \log(1+\exp(\cdot))$ (softplus). The resulting resampled proposal is then normalized as
\[
r_{\theta,\phi}(z\mid x, T) = \frac{q_\phi(z\mid x) a_{\theta,\phi}(z\mid x, T)}{\mathbb{E}_q[a_{\theta,\phi}(z\mid x, T)]}.
\]
This procedure ensures a continuous, tunable trade-off between computational cost and closeness to the true posterior [1804.01712, 2309.14612].

## 3. Relation to α-Divergence and the α-DRS Scheme

The "Refined α-Divergence Variational Inference via Rejection Sampling" framework (α-DRS) generalizes VRS by introducing Rényi α-divergence as an objective. For $\alpha > 0$, $\alpha \neq 1$, the Rényi divergence between target $p$ and proposal $q$ is
\[
D_\alpha(p \| q) = \frac{1}{\alpha-1} \log \int p(x)^\alpha q(x)^{1-\alpha} \, dx.
\]
The α-DRS algorithm proceeds in two stages [1909.07627]:
- **Stage 1:** Optimize $q_\theta$ by minimizing a Monte Carlo estimate of $D_\alpha(p\|q_\theta)$.
- **Stage 2:** Use learned $q_\theta$ and an (approximate) optimal RS constant (or quantile-based surrogate) to perform rejection sampling, generating a refined sample-based approximation.

The key theoretical link is that as $\alpha \to \infty$, $D_\infty(p\|q) = \log M(\theta)$, with $M(\theta)$ the tightest rejection sampling constant for $q_\theta$. Crucially, it is established that the rejection step cannot increase $D_\alpha$, i.e.,
\[
D_\alpha(p\|r_\theta) \leq D_\alpha(p\|q_\theta),
\]
which guarantees improvement (or at least non-degradation) in the variational approximation after rejection sampling [1909.07627].

## 4. Variational Objectives, Gradient Estimation, and Reparameterization

VRS can be cast within a variational inference framework, where the Evidence Lower Bound (ELBO) under the resampled proposal is
\[
\mathcal{L}(\phi, T) = \mathbb{E}_{r_{\phi,T}(z\mid x)}[\log p(x, z) - \log r_{\phi, T}(z\mid x)] \leq \log p(x).
\]
Taking advantage of the structure of $r_{\phi, T}$, Grover et al. derive low-variance gradient estimators, involving covariances under the resampled proposal. For reparameterizable base proposals (e.g., Gaussian), Jankowiak & Phan introduce a "pathwise" (low-variance) gradient for the parameters $\phi$ of $q_\phi$ via the identity:
\[
\nabla_\phi \mathcal{L} = \mathbb{E}_{r_{\phi, T}}\left[\left(2\tilde{a}_T(z)\partial_z \tilde{a}_T(z) + \tilde{a}_T(z)\partial_z A(z)\right) \cdot \nabla_\phi z\right],
\]
where $\tilde{a}_T$ is the (smooth) acceptance, $A(z)$ is a function of the log-density ratios, and $\nabla_\phi z$ is the Jacobian from reparameterization [2309.14612]. This estimator exhibits substantially reduced variance compared to REINFORCE-style alternatives, enabling scalable and robust training.

## 5. Cost–Fidelity Trade-offs and Algorithmic Structure

The expected cost per accepted sample is inversely proportional to the mean acceptance probability, $Z(\phi, T) = \mathbb{E}_{q_\phi}[\tilde{a}_T]$. Lowering the acceptance threshold tightens the approximation (reducing bias/KL-divergence) but concomitantly reduces $Z$, thus increasing computational cost. The variational gap $\Delta = \log p(x) - \mathcal{L}$ can be bounded by
\[
\Delta < \tfrac{3}{2}\xi e^T
\]
for sufficiently heavy-tailed $q_\phi$ and $T < -\log(2\xi)$, reinforcing that accuracy improves at the expense of sampling effort [2309.14612]. The full algorithm typically involves an inner sample-reject loop embedded within standard SGD updates, with threshold $T$ either dynamically tuned (e.g., to match a target quantile acceptance) or set via theoretical criteria.

### Pseudocode Structure for α-DRS

```pseudo
Stage 1:
  repeat
    sample x₁,...,x_S ∼ q_θ
    compute L(θ) = (1/S) Σₛ [π̃(xₛ)/q_θ(xₛ)]^α
    update θ ← θ - η ∇_θ L(θ)
  until convergence

Stage 2:
  set T ← Quantile_{γ}{log q_{θ*}(x) − log π̃(x)}, x ∼ q_{θ*}
  for i = 1 ... N_desired:
    repeat
      sample x ∼ q_{θ*}
      accept with probability a(x|T)
    until accepted
    collect x
```
[1909.07627]

## 6. Empirical Results and Applications

Empirical evidence demonstrates significant improvements using VRS-based methods:
- Grover et al. report that on sigmoid belief networks trained on MNIST, VRS yields average improvements of 3.71 nats (single-sample) and 0.21 nats (multi-sample) in marginal log-likelihood over state-of-the-art baselines [1804.01712].
- α-DRS provides substantial reductions in $D_\alpha(p\|q) \to D_\alpha(p\|r)$, with marked fidelity improvements (e.g., posterior mode recovery in mixture models, improvement in Bayesian neural network regression) [1909.07627].
- Jankowiak & Phan observe that RVRS achieves lower gradient variance and superior or competitive posterior fidelity compared to normalizing flows, importance weighting, and hybrid MCMC/variational schemes. For instance, RVRS with a simple Gaussian proposal outpaces IWAE and normalizing flow VI on several inference tasks, with empirical speedups and improved negative ELBOs [2309.14612].

## 7. Practical Considerations, Limitations, and Extensions

Principal algorithmic considerations include the tuning of acceptance thresholds (either through quantile-based rules or theoretical approximations), selection of divergence order $\alpha$ (α-DRS), and proposal family expressivity. Too aggressive rejection increases variance and computational cost, while soft acceptance thresholds facilitate efficient trade-offs.

Limitations include:
- Increased complexity from rejection loops, leading to variable per-sample compute.
- Added hyperparameter tuning (thresholds, quantiles, or $\alpha$).
- For non-reparameterizable proposals, standard VRS relies on higher-variance gradient estimators.

VRS variants extend to richer variational families (normalizing flows, hierarchical structures), per-layer factorized resampling, and potentially reinforcement learning/policy search [1804.01712, 2309.14612]. 

VRS and its descendants stand as a flexible, model-agnostic enhancement to variational inference pipelines—explicitly utilizing model densities to refine approximate inference and offering principled mechanisms to balance fidelity and cost in probabilistic modeling [1804.01712, 1909.07627, 2309.14612].

Source: https://www.emergentmind.com/topics/variational-rejection-sampling-vrs