Gaussian Tilts in Bayesian Inference

• Gaussian tilts of the prior are defined as the multiplication of a Gaussian density by an exponential linear or quadratic form, preserving the closed-form structure of the distribution.
• They ensure the preservation of linearity in Bayes estimators under Gaussian noise, thereby enabling efficient computation of posterior means and tractable sampling.
• Applications span Bayesian inverse problems and variational autoencoders, where tilted priors improve out-of-distribution detection and sampling efficiency.

A Gaussian tilt of the prior refers to the construction in which an initial Gaussian (or, more generally, log-concave) probability density is multiplied by an exponential of a linear or quadratic form—yielding a new density that, depending on the structure of the tilt, remains within the Gaussian family or is sharply concentrated in particular regions. This concept is central in Bayesian estimation, statistical inverse problems, and generative modeling, where the form and properties of tilted priors have important implications for the structure of posterior distributions, the linearity of Bayes estimators, and algorithmic tractability. The mathematics of Gaussian tilt leverages the closure of the Gaussian family under exponential linear and quadratic tilts, leading to exact preservation of key analytical and computational properties in the posterior.

1. Definition and Analytical Formulation

Given a multivariate Gaussian prior on $X \in \mathbb{R}^n$ with mean $\mu$ and covariance $\Sigma$,

$$p(x) = (2\pi)^{-n/2}|{\Sigma}|^{-1/2} \exp\left(-\frac{1}{2}(x-\mu)^\top {\Sigma}^{-1}(x-\mu)\right),$$

the exponential (linear) tilt by $t \in \mathbb{R}^n$ modifies the prior to

$p_t(x) \propto e^{t^\top x} p(x).$

The normalization (cumulant-generating function),

$$Z(t) = \int e^{t^\top x} p(x) dx = \exp\left(t^\top \mu + \frac{1}{2} t^\top \Sigma t\right),$$

ensures $p_t(x)$ is still Gaussian with mean $\mu_t = \mu + \Sigma t$ and unchanged covariance $\Sigma_t = \Sigma$. Completion of the square in the exponent shows that all moments and the normalizing constant can be written in closed form. Quadratic tilts, i.e., multiplying by $\exp(-\frac{1}{2}x^\top Q x + b^\top x)$ with $Q$ positive semidefinite, correspond to the likelihood in linear-Gaussian models and yield updated Gaussian posteriors (Barnes et al., 2024).

2. Linearity of Bayes Estimators and Gaussian Characterization

A fundamental result—Theorem 1 of (Barnes et al., 2024)—establishes that when observing $Y = X + Z$ with $Z \sim \mathcal{N}(0, I_n)$ and seeking to minimize $L^p$-loss $\ell_p(x) = \|x\|_2^p$ for $1 \leq p \leq 2$, a linear Bayesian estimator $g_p(y) = A y$ (with $0 \prec A \prec I$) exists if and only if the prior is a nondegenerate multivariate Gaussian. Under these conditions, exponential tilting preserves exact linearity due to the Gaussian family’s closure under such operations. For quadratic loss ($p=2$), the Bayes estimator is the posterior mean: $$g_2^t(y) = \mathbb{E}_t[X|Y=y] = A y + (I - A) \mu_t,\quad A = \Sigma(\Sigma+I)^{-1},$$ and for zero-mean priors, the estimator remains purely linear even after tilting (Barnes et al., 2024).

For $p > 2$, the set of priors admitting linear estimators under Gaussian noise is no longer unique to the Gaussians, but in the high-probability regime $1 \leq p \leq 2$, only Gaussian tilts admit this property.

3. Gaussian Tilts in Posterior Construction and Boosted Sampling

In Bayesian inverse problems, especially with linear measurement models $y = Ax + w$, the posterior can be viewed as a (possibly quadratic) exponential tilt of the prior: $$\nu(x) \propto \pi(x) \cdot \exp\left(-\frac{1}{2}(A x - y)^\top \Sigma^{-1}(A x - y)\right).$$ This motivates sampling and inference via "tilted transport" methods (Bruna et al., 2024), where one forms a family of boosted posteriors by iteratively tilting the (possibly non-Gaussian) prior and evolving the parameters $(Q_t, b_t)$ through an ODE system. When the prior is Gaussian (or quadratic-tilted log-concave), the resulting induced posterior samples remain tractable Gaussians, facilitating exact or approximate sampling with provable mixing guarantees. The strong log-concavity of the boosted posterior can be quantified by a susceptibility bound involving the prior’s covariance propagated under tilting transformations (Bruna et al., 2024).

4. Algorithmic and Practical Consequences

The computational tractability of Bayesian estimation and posterior sampling under Gaussian tilts is central to modern high-dimensional problems. In the "tilted transport" sampling paradigm, when the prior is Gaussian, each iteratively tilted density remains Gaussian, preserving efficient sampling and exact mixing times predicted by log-Sobolev inequalities. If the prior is non-Gaussian (e.g., a mixture or compound distribution), the tilt no longer preserves Gaussianity, and both the analytical form of posteriors and algorithmic guarantees may break down except in certain log-concave or bounded susceptibility regimes (Bruna et al., 2024).

In variational autoencoders (VAEs), exponentially-tilted Gaussian priors (e.g., via $e^{\tau\|z\|}$, where $\tau > 0$) force the latent representations to concentrate on a hypersphere, radically altering the geometry and statistical regularization of the model. Closed-form expressions for normalizing constants (involving Kummer’s confluent hypergeometric function) and analytic KL-divergence bounds enable tractable implementation with empirical improvements in out-of-distribution detection and sample quality (Floto et al., 2021).

5. Special Forms: Jeffreys Priors and Constraint Effects

When the likelihood is Gaussian, the Jeffreys prior—maximally uninformative in the Fisher information sense—is constant (flat) if all parameters vary freely. Imposing constraints, such as nonnegativity, truncates the prior but does not induce any “tilt” in the relative sense: the analytic form and boundary normalization remain exactly as for the unconstrained (flat) case. For location-type parameters, such as the sum of neutrino masses or tensor-to-scalar ratio, substituting the correctly truncated Jeffreys prior for a flat prior yields no measurable shift in credible intervals, confirming the absence of meaningful tilt in this context (Hannestad et al., 2017).

6. Exclusivity of Gaussian Tilts in Preserving Linearity

Exponential tilts of Gaussian priors yield new densities in the same parametric class—no other prior possesses this closure. For $1 \leq p \leq 2$ and Gaussian noise, only Gaussian tilts (i.e., priors of the form $x \mapsto e^{t^\top x}p(x)$ with $p$ Gaussian) preserve the linearity of optimal Bayes estimators. Tilting a non-Gaussian prior by $e^{t^\top x}$ produces a non-Gaussian density, breaking this property (Barnes et al., 2024). The set of linear-preserving priors is thus completely characterized by the Gaussian family and their exponential linear (or quadratic) tilts.

7. Empirical Examples and Applications

Empirical investigations use Gaussian tilts in various regimes:

• In VAEs, exponentially-tilted Gaussian priors lead to state-of-the-art out-of-distribution detection, tightly concentrating encodings on hyperspheres and yielding high AUC–ROC scores (Floto et al., 2021).
• In high-dimensional inverse problems, Gaussian mixture priors and double-well potentials provide concrete scenarios where tilting and the resulting boosted posteriors substantially reduce autocorrelation times and improve sampling efficiency across SNR regimes (Bruna et al., 2024).
• When flat or truncated-flat priors are used in cosmological parameter estimation, replacing them with the (almost identical) Jeffreys prior confirms the lack of meaningful tilt for location-type parameters (Hannestad et al., 2017).

In summary, Gaussian tilts of the prior are analytically tractable, preserve essential properties for linear estimation and efficient sampling, and are uniquely robust among all prior transformations for maintaining the closure and computability essential to Bayesian inference, posterior sampling, and generative modeling.