χ²-Divergence Variational Objective

• The χ²-divergence variational objective is defined via Pearson’s χ² divergence, offering a principled, mass-covering approach for robust variational inference.
• It employs variational representations such as Fenchel dual and Chapman–Robbins bounds to enable efficient algorithmic implementations in both classical and quantum domains.
• Empirical results indicate enhanced likelihood estimation and optimization stability, demonstrating clear practical benefits in statistical and generative modeling.

The $χ^2$-divergence variational objective defines a principled approach for inference and learning by leveraging the structure of the Pearson %%%%1%%%%-divergence within the broader class of $f$-divergence-based objectives. It appears in a wide spectrum of contexts—statistical variational inference, generative modeling, variational expressions for quantum states, and neural estimation—motivated by its distinctive mass-covering and bias–variance control properties. This article develops the mathematical foundations, variational representations, algorithmic implementations, and empirical implications of the $χ^2$-divergence variational objective across classical and quantum domains.

1. Mathematical Definition and Properties

The Pearson $χ^2$-divergence between probability densities or mass functions $p(z)$ and $q(z)$ ($q(z)>0$ on the support of $p$) is given by

$$D_{χ^2}(p\,\|\,q) = \int \frac{(p(z)-q(z))^2}{q(z)}\,dz = \int \frac{p(z)^2}{q(z)}\,dz - 1 = \mathbb{E}_{q}\!\left[\left(\frac{p(z)}{q(z)}\right)^2\right] - 1.$$

In the $f$-divergence framework, this corresponds to the generator $f(t) = (t-1)^2$ and takes the canonical form

$$D_f(p\|q) = \int q(z)\,f\left(\frac{p(z)}{q(z)}\right)\,dz,$$

with convex conjugate $f^*(t) = \frac{1}{4}\,(t+2)^2 - 1$, ensuring operator-convexity and variational dual admissibility in both classical and quantum regimes (Li et al., 2023, Fang et al., 11 Feb 2025).

The $χ^2$-divergence has key inequalities relating to other divergences: $$D_{\mathrm{KL}}(p\|q) \;\le\; \ln\left(1 + D_{χ^2}(p\|q)\right) \;\le\; D_{χ^2}(p\|q),$$ and, unlike the reverse KL, it is mass-covering, strongly penalizing $q$ wherever $p$ dominates (Li et al., 2023, Dieng et al., 2016).

2. Variational Representations and Dual Formulations

Multiple variational characterizations of $D_{χ^2}(p\,\|\,q)$ have been established:

• Legendre Transform (Fenchel) Bound:

$$D_{χ^2}(p\|q) = \sup_{T} \left\{ \mathbb{E}_p[T(z)] - \mathbb{E}_q\left[\frac{1}{4} T(z)^2 + T(z)\right] \right\}$$

with supremum achieved at $T^*(z) = 2\left( \frac{p(z)}{q(z)} - 1 \right)$ (Nowozin et al., 2016, Birrell et al., 2020).

• Chapman–Robbins Variational Representation:

$$D_{χ^2}(p\|q) = \sup_{g}\left[2\,(\mathbb{E}_p[g]-\mathbb{E}_q[g]) - \mathrm{Var}_q(g)\right].$$

The tight “affine Hammersley–Chapman–Robbins” form is (Birrell et al., 2020, Salazar, 13 Nov 2025):

$$D_{χ^2}(p\|q) = \sup_{g} \frac{(\mathbb{E}_p[g]-\mathbb{E}_q[g])^2}{\mathrm{Var}_q(g)}.$$

These tighten conditioning and accelerate learning when used for neural estimation.

• Measured and Quantum χ² Variational Formulas:

In the quantum setting, measured $χ^2$ admits a closed-form convex program (Fang et al., 11 Feb 2025):

$$χ^2_{\mathrm{Meas}}(\rho\|\sigma) = 1 + \sup_{\omega = \omega^\dagger} \left\{\mathrm{Tr}[\rho\,\omega] - \frac{1}{4}\mathrm{Tr}[\sigma\,(\omega+2I)^2]\right\}.$$

General Petz $f$-divergences decompose as mixtures of quantum $χ^2$ “atomic” kernels (Salazar, 13 Nov 2025).

3. Variational Inference and the $χ^2$ Objective

The $χ^2$-divergence variational objective (χ²-VI) appears as a special case of $f$-divergence variational inference frameworks (Wan et al., 2020, Dieng et al., 2016, Zhang et al., 2019, Regli et al., 2018): $$\mathcal{J}(q) = \int \frac{p(z, \mathcal{D})^2}{q(z)}\,dz, \quad \text{or equivalently} \quad D_{χ^2}(p(\cdot,\mathcal{D})\|q) + 1,$$ for $p(z, \mathcal{D})$ the joint model and $q(z)$ the variational approximation. This is used to form the “$\chi^2$-Upper Bound” (CUBO): $$\log p(\mathcal{D}) \leq \mathrm{CUBO}_2(q) := \frac{1}{2}\log\left( \mathcal{J}(q) \right),$$ giving a sandwich estimator: $$\mathrm{ELBO}(q) \leq \log p(\mathcal{D}) \leq \mathrm{CUBO}_2(q).$$ Gradient estimation may use either the reparameterization trick (for reparameterizable $q$) or score-function estimators, and multi-sample importance weighting can be employed for bias-variance tradeoff (Wan et al., 2020, Dieng et al., 2016).

Block coordinate (mean-field) updates are available: $$q_j^{\mathrm{new}}(z_j) \propto \sqrt{ \mathbb{E}_{q_{-j}} \left[ \left( \frac{p(z, \mathcal{D})}{q_{-j}(z_{-j})} \right)^2 \right] }$$ with normalization over $z_j$ (Wan et al., 2020).

$\chi^2$-VI has been implemented in “CHIVI” (Dieng et al., 2016) and “VIS” (Li et al., 2023), as well as in spread-divergence-regularized VAEs (Zhang et al., 2019). The theoretical guarantee is the monotonic improvement of $\mathrm{CUBO}_2$, converging to the true evidence as $q \to p(z|\mathcal{D})$.

4. Algorithmic Implementation Across Domains

Algorithmic realizations of the $χ^2$-VI objective feature:

Domain/Class	Stochastic gradient (reparam)	Dual form optimization	Adversarial (f-GAN) implementation
Classical VI	Yes (Dieng et al., 2016, Wan et al., 2020)	Yes (Birrell et al., 2020)	Yes (Nowozin et al., 2016)
Quantum/Measured	Yes (Fang et al., 11 Feb 2025)	Yes (Salazar, 13 Nov 2025)	Not typical
Generative Models	Yes (Zhang et al., 2019, Nowozin et al., 2016)	—	Yes

CHIVI minimizes $\mathrm{CUBO}_2$ stochastically using black-box gradients; VIS minimizes the forward $χ^2$-divergence to design improved proposal distributions for variational importance sampling, showing superior bias and variance control for log-likelihood estimation (Li et al., 2023).

In a generative adversarial context, $f$-GANs can be specialized to $χ^2$ by choosing $f(u) = (u-1)^2$, yielding a saddle-point problem over generator and critic networks with stable quadratic critic updates (Nowozin et al., 2016).

Affine and shift-only duals for neural estimation accelerate convergence in high dimensions and give better-conditioned optimization than standard Fenchel duals (Birrell et al., 2020). Implementation as a convex program (SDP) is available for measured $χ^2$ in the quantum setting (Fang et al., 11 Feb 2025).

5. Empirical Behavior, Practical Considerations, and Theoretical Insights

Empirical findings:

• Mass-covering: $χ^2$-VI, and closely related forward divergences, penalize $q$ for missing regions with high $p$, encouraging variance overestimation and avoiding mode collapse observed with exclusive KL (Dieng et al., 2016, Li et al., 2023).
• Bias–variance trade-off: Lower bias in log-likelihood estimation (IS context) with optimal $q$ as $D_{χ^2}$ penalizes heavy tails; variance reduction with multi-sample estimates at the cost of higher stochastic gradient variance (Li et al., 2023, Wan et al., 2020).
• Optimization stability: High importance weights $(p/q)^2$ can lead to numerically unstable gradients, requiring moderate learning rates, regularization, or clipping (Dieng et al., 2016, Birrell et al., 2020, Regli et al., 2018).
• Robustness: Pure $χ^2$-VI may lack robustness to outliers and exhibit high gradient variance; log-transformed sAB divergences (e.g., gamma divergences with $\alpha + \beta > 1$) are preferable for regression with outliers (Regli et al., 2018).

Theoretical properties:

• Convergence guarantees: Under mild regularity, $D_{χ^2}(p\|q)$ strictly decreases per update in iterative schemes, with convergence to the unique minimum (Daudel et al., 2019).
• Sandwich bounds: ELBO and CUBO provide computable lower and upper bounds on $\log p(\mathcal{D})$; their gap reflects the proximity of $q$ to $p$ (Wan et al., 2020, Dieng et al., 2016).
• Duality and conditioning: Affine improvements of the variational bound yield better conditioned functionals, promoting faster and more stable convergence (Birrell et al., 2020).
• Quantum generalizations: The $χ^2$ mixture forms provide atomic decompositions of general Petz $f$-divergences in the quantum setting, with associated thermodynamic uncertainty relations (Salazar, 13 Nov 2025).

6. Connections, Extensions, and Related Divergences

$χ^2$-divergence is a limit point in the $f$-divergence family, arising as a special case of:

• Scale-invariant alpha–beta (sAB) divergences for $(\alpha,\beta) = (2, -1)$ (Regli et al., 2018).
• Petz–Rényi divergences as $\alpha \to 2$ (Salazar, 13 Nov 2025).
• It interpolates between mean-matching ($\alpha\!+\!\beta\!\uparrow\!2$), mass covering ($0\!<\!\beta\!<\!1$), and mode-seeking ($\beta\!<\!0$) (Regli et al., 2018).

Algorithmic strategies for $χ^2$-VI can be viewed as subset cases of more flexible $f$-VI or $f$-EI algorithms, which include KL-VI, Rényi-VI, and Cramer–von Mises objectives (Wan et al., 2020, Daudel et al., 2019).

Extensions to non-likelihood training via spread divergences and neural estimators, adversarial variants, and measured versions for quantum models have been systematically developed (Zhang et al., 2019, Salazar, 13 Nov 2025, Fang et al., 11 Feb 2025), with domain-appropriate guarantees and computational structures.

7. Summary Table: Core Variational Forms

Objective	Variational Formulation	Reference
Standard (Fenchel dual)	$$\sup_T \mathbb{E}_p[T] - \mathbb{E}_q\left[\frac{1}{4}T^2 + T\right]$$	(Nowozin et al., 2016, Birrell et al., 2020)
Affine/Chapman–Robbins	$$\sup_g \ [\mathbb{E}_p g - \mathbb{E}_q g]^2 / \mathrm{Var}_q(g)$$	(Birrell et al., 2020, Salazar, 13 Nov 2025)
ELBO/CUBO sandwich	$$\mathrm{ELBO}(q) \leq \log p(\mathcal{D}) \leq \mathrm{CUBO}_2(q)$$	(Dieng et al., 2016, Wan et al., 2020)
Mean-field χ²-VI update	$$q_j^{\mathrm{new}}(z_j) \propto \sqrt{\mathbb{E}_{q_{-j}}[(p(z,\mathcal{D}) / q_{-j}(z_{-j}))^2]}$$	(Wan et al., 2020)
Quantum measured χ²	$$1 + \sup_{\omega} \mathrm{Tr}[\rho\,\omega] - \frac{1}{4} \mathrm{Tr}[\sigma\,(\omega+2I)^2]$$	(Fang et al., 11 Feb 2025)

Empirical work demonstrates accelerated convergence, improved likelihood bounds, and variational flexibility in both classical and quantum models when utilizing the $χ^2$-divergence variational objective with the appropriate estimator, dual form, and regularization scheme. The choice of $χ^2$-VI, mass-covering (forward), or mode-seeking (reverse) $f$-divergences directly shapes the inferential and generative properties of the resultant models.