Weighted Variational Bound Methods

• Weighted variational bounds are a class of objective functions that incorporate weightings through techniques like importance sampling, Rényi divergence, and Hölder’s inequality to construct tighter bounds on log-partition and marginal likelihoods.
• They employ diverse methodologies—such as doubly-reparameterized gradients and perturbative expansions—to improve optimization stability and trade off bias with variance in probabilistic inference.
• Applications span variational autoencoders, hierarchical models, and reinforcement learning, while addressing challenges like weight collapse and gradient variance in high-dimensional settings.

A weighted variational bound is a class of variational objective functions that introduce weightings—either via alternative divergence measures, importance sampling, exponentiation schemes, polynomial approximations, or groupwise factorizations—into the construction of lower or upper bounds on log-partition functions, marginal likelihoods, or other quantities central to Bayesian inference, statistical physics, or ergodic theory. The term encompasses methods in probabilistic machine learning (e.g., importance-weighted variational inference, Rényi variational bounds, perturbative bounds, Hölder bounds), ergodic theory (weighted variational estimates for averages), and harmonic analysis (weighted variational estimates for orthogonal expansions), as well as variants designed for high-dimensional, hierarchical, and structured models.

1. Formal Definitions and Main Families

Weighted variational bounds can be grouped by their mathematical structure and by the domain in which they are applied.

• Importance-Weighted Bounds: Given a base variational distribution $q(z|x)$ and unnormalized joint $p_\theta(x,z)$, the standard Evidence Lower Bound (ELBO) is

$$\text{ELBO}(\theta,\phi;x) = \mathbb{E}_{q_\phi(z|x)}[\log p_\theta(x,z) - \log q_\phi(z|x)],$$

whereas the importance-weighted lower bound for $N$ samples ($\mathrm{IWAE}_N$) is

$$\mathrm{IWAE}_N(\theta,\phi;x) = \mathbb{E}_{z_{1:N}\sim q_\phi}\left[ \log \left(\frac{1}{N}\sum_{i=1}^N w(z_i;x) \right) \right], \quad w(z_i;x) = p_\theta(x,z_i) / q_\phi(z_i|x).$$

This bound tightens with increasing $N$ (Daudel et al., 15 Oct 2024, Mattei et al., 2022, Domke et al., 2018).

• Rényi / Alpha-Divergence Variational Bounds: For any $\alpha\neq1$ and $N=1$, the Rényi (VR) bound is

$$\mathcal{L}^{(\alpha)}(\theta,\phi;x) = \frac{1}{1-\alpha} \log \mathbb{E}_{q_\phi}\left[w(z;x)^{1-\alpha}\right],$$

interpolating between the ELBO ($\alpha \rightarrow 1$), and the exact log-likelihood as $\alpha\to0$ (Daudel et al., 2022, Daudel et al., 15 Oct 2024).

• Unified Weighted Bounds (VR-IWAE): A two-parameter family combines importance weighting with an alpha-divergence,

$$\mathcal{L}^{(N,\alpha)}(\theta,\phi;x) = \frac{1}{1-\alpha} \mathbb{E}_{z_{1:N} \sim q_\phi} \left[ \log \left( \frac{1}{N} \sum_{i=1}^N w(z_i;x)^{1-\alpha} \right) \right].$$

This specializes to IWAE for $\alpha=0$ and to the standard Rényi bound for $N=1$ (Daudel et al., 15 Oct 2024, Daudel et al., 2022).

• Hölder / Weighted Mean Bounds: The variational Hölder (VH) bound modifies the classical variational objective by using Hölder's inequality with optimizable exponents,

$$Z = \int \prod_{j=1}^J f_j(x) dx \leq \prod_{j=1}^J \|f_j\|_{1/\alpha_j}^{\alpha_j}, \; \sum_j \alpha_j = 1, \, \alpha_j \geq 0,$$

This provides a convex optimization problem for upper bounds on the partition function (Bouchard et al., 2015, Chen et al., 2021).

• Perturbative / Taylor-Based Bounds: Perturbative Black-Box VI introduces a family of polynomially-weighted bounds via finite-order Taylor expansions of the exponential function,

$\mathcal{L}^{(K)}(\lambda, V_0) = e^{-V_0} \sum_{k=0}^K \frac{1}{k!}\mathbb{E}_{\bz\sim q}\left[(\log p(\bx,\bz)-\log q(\bz;\lambda)+V_0)^k\right],$

with $K=1$ corresponding to KL-VI and $K\to\infty$ to the true marginal likelihood (Bamler et al., 2017).

• Hierarchical and Local Weighted Bounds: In hierarchical models, importance-weighted bounds can be applied locally,

$$\mathcal{L}_{\text{local}}(q) = \mathbb{E}_{q(\theta)}\left[ \log p(\theta) - \log q(\theta) + \sum_{i=1}^G \mathrm{IWELBO}_{K_i}(q(z_i \mid \theta)) \right ],$$

enabling unbiased stochastic optimization via groupwise subsampling (Geffner et al., 2022, Sobolev et al., 2019).

2. Theoretical Properties and Tightness

Weighted variational bounds are characterized by monotonicity, convergence, and explicit trade-offs:

• Monotonicity: Increasing the number of samples $N$ in importance-weighted bounds (or tightening the order $K$ in Taylor expansions) always tightens the bound, i.e.,

$L_N(\phi) \leq L_{N+1}(\phi) \leq \log p(x),$

provided the weights are exchangeable (Mattei et al., 2022, Domke et al., 2018, Bamler et al., 2017).

• Convergence Rates: The gap between the bound and the exact log-likelihood often decays as $1/N$ or $1/K$ under regularity conditions,

$\log p(x) - L_N(\phi) = O\left(\frac{1}{N}\right).$

• Bias-Variance Trade-off: Weighted bounds can exhibit controllable bias and variance as functions of the weighting parameters ($N$, $\alpha$, $K$). For example, higher $N$ reduces the bias in the estimated log-marginal likelihood, but for typical "pathwise" gradient estimators, the signal-to-noise ratio (SNR) of the inference network's parameter gradients may collapse as $O(1/\sqrt{N})$ (M'Charrak et al., 2022, Daudel et al., 15 Oct 2024, Liévin et al., 2020).
• High-Dimensional and Exponential Collapse: For large latent dimension $d$, even exponentially large sample increases (in $N$, $K$) may be required to recover a non-trivial variational gap due to "weight collapse"—i.e., the dominance of a single importance weight in the estimate (Daudel et al., 15 Oct 2024, Daudel et al., 2022).

3. Gradient Estimators and Optimization

Efficient training with weighted variational bounds relies on sophisticated unbiased gradient estimators, e.g.:

• Reparameterization (REP) Gradients:

$$\nabla_\phi \mathcal{L}^{(N,\alpha)} = \mathbb{E}_{\varepsilon_{1:N}} \left[ \sum_{i=1}^N \frac{w(z_i)^{1-\alpha}}{\sum_{j=1}^N w(z_j)^{1-\alpha}} \nabla_\phi \log w(z_i) \right].$$

For standard IWAE ($\alpha=0$), the SNR for inference network gradients degrades as $O(1/\sqrt{N})$ (Daudel et al., 15 Oct 2024, M'Charrak et al., 2022).

• Doubly-Reparameterized (DREP) Gradients: These suppress high-variance score term components, and crucially, for $\alpha>0$, the SNR for inference parameter gradients scales favorably as $O(\sqrt{MN})$ (Daudel et al., 15 Oct 2024, Liévin et al., 2020).
• Score-Function with Control Variates: In the limit of large $N$, optimized control variates (e.g., OVIS) can convert the $O(1/\sqrt{N})$ SNR decay to $O(\sqrt{N})$ growth for score-function estimators, making the estimation as stable as pathwise methods (Liévin et al., 2020).
• Optimizing Hölder Weights: For upper bounds via Hölder's inequality (e.g., the variational Hölder bound), gradients w.r.t. the weights $\alpha$ and any pivot parameters can be computed analytically to maintain convexity of the optimization problem (Bouchard et al., 2015, Chen et al., 2021).

4. Applications and Methodological Implications

Weighted variational bounds are implemented widely across probabilistic modeling, inference, and RL:

• Variational Autoencoders (VAE) and Deep Latent Models: IWAE and VR-IWAE are commonly used to alleviate the limitations of standard mean-field VI, providing tighter likelihood lower bounds and richer implicit posteriors (Domke et al., 2018, Daudel et al., 2022, Cremer et al., 2017).
• Hierarchical and Structured Models: Local or groupwise applications of weighted bounds make amortized inference possible in hierarchical Bayesian models with large numbers of local variables, enabling stochastic optimization with mini-batches (Geffner et al., 2022, Sobolev et al., 2019).
• Reinforcement Learning: Weighted variational bounds, such as Q-weighted VLO losses, have been adopted for online training of diffusion-model RL policies where experiences must be value-weighted due to the lack of “good” actions in the replay buffer (Ding et al., 25 May 2024).
• Thermodynamic Integration and Beyond: Weighted mean (Hölder) paths in thermodynamic variational objectives provide flat thermodynamic curves, sharper lower bounds, higher effective sample size, and more stable gradient estimators in challenging inference regimes (Chen et al., 2021).
• Vector-Valued Ergodic and Harmonic Analysis: Weighted $r$-variation bounds are established for ergodic averages and Fourier/Walsh series in weighted $L^p(w)$ spaces, with rates depending on the Muckenhoupt characteristic of the weight (Krause et al., 2014, Do et al., 2012, Lacey et al., 2011).

5. Empirical and Theoretical Comparisons

A synopsis of findings across key weighted variational methodologies:

Bound Type	Bias-Tightness Order	SNR (Inference)	SNR (Model)	Empirical Observations
ELBO	Loosest, O(1)	$O(\sqrt{M})$	$O(\sqrt{M})$	Fast inference, loose log bound
IWAE ($N$)	Tighter, O(1/N)	$O(\sqrt{M/N})$	$O(\sqrt{MN})$	SNR issue for encoder, decoder improves with N
VR / Rényi ($\alpha$)	O(1), can interpolate	$O(\sqrt{M})$	$O(\sqrt{M})$	Tuning $\alpha$ controls bias/SNR
VR-IWAE ($N,\alpha$)	O(1/N), trade-off in $\alpha$	$\sim \sqrt{M/N^\beta}$	$\sim \sqrt{MN}$	SNR stabilizes with $\alpha>0$, DREP estimator preferred
Hölder/VH	Convex upper, controlled gap	—	—	Upper bound, convexity, robust for correlated factors
Perturbative (PBBVI, $K$)	O(1/K), trade-off	Poly. variance	—	Intermediate K (e.g., 3) optimal in practice
Local/groupwise IW	Per-block O(1/K)	Blocks independent	—	Subsampling possible, scalable in G

For the generative model (θ), increasing “tightness” (N, K, ... ) always helps. For inference networks (φ), excessive tightness can impair optimization unless an appropriate estimator or blend (e.g., doubly-reparameterized, mixture, or partitioned estimator) is used (M'Charrak et al., 2022, Daudel et al., 15 Oct 2024).

6. Limitations and Open Problems

• Weight Collapse in High Dimensions: In high-dimensional regimes, importance weights concentrate on a single sample unless the number of samples grows exponentially in the latent dimension; in such settings, both the bound and gradient estimator SNRs collapse to the level of single-sample VI (Daudel et al., 15 Oct 2024, Daudel et al., 2022).
• Computational Cost: Increasing the number of samples (N, K) or polynomial order (K in PBBVI) increases both estimator variance and computational overhead; optimal trade-offs need empirical tuning (Bamler et al., 2017, M'Charrak et al., 2022).
• Gradient Variance and Stability: Pathwise gradient SNR for encoder parameters decays with tighter bounds, requiring either alternative estimators (DREP, OVIS) or blending strategies (MIWAE, PIWAE, CIWAE) to maintain optimization efficiency (Liévin et al., 2020, M'Charrak et al., 2022, Daudel et al., 15 Oct 2024).
• Choice of Weights and Parameterization: Convex weighted bounds (VH, Hölder, etc.) require optimization over weight vectors (e.g., $\alpha_j$). The problem is convex under suitable assumptions, but computation of norms $\|f_j\|_{1/\alpha_j}$ and related gradients may be prohibitive in complex or high-dimensional models (Bouchard et al., 2015).
• Variance Reduction Techniques: Leave-one-out control variates, importance resampling, and antithetic sampling have been investigated but require further empirical and theoretical work in complex generative models (Liévin et al., 2020, Mattei et al., 2022).

7. Summary and Future Directions

Weighted variational bounds generalize classical variational inference by introducing sample weighting, polynomial expansion, or alternative divergence-based objectives to control the tightness-bias-variance trade-off in variational approximation. Their properties are now mathematically well-understood, with standardized definitions for the bound types, gradient estimators, and asymptotic behaviors. Key future challenges are extending these approaches to richer variational families, addressing weight collapse in high dimensions, and integrating advanced variance control techniques with scalable implementations. The theoretical frameworks unify a broad spectrum of contemporary variational inference and remain a fundamental tool for scalable, accurate posterior inference in complex probabilistic models.

References: (Bamler et al., 2017, Bouchard et al., 2015, Daudel et al., 2022, Daudel et al., 15 Oct 2024, Chen et al., 2021, Mattei et al., 2022, Do et al., 2012, Lacey et al., 2011, Domke et al., 2018, Geffner et al., 2022, M'Charrak et al., 2022, Liévin et al., 2020, Krause et al., 2014, Sobolev et al., 2019, Ding et al., 25 May 2024).