Likelihood-Informed Subspace Techniques

• LIS techniques are a dimension reduction framework that identifies a low-dimensional subspace where observed data significantly update the prior in high-dimensional Bayesian inverse problems.
• They decompose the parameter space into data-dominated and prior-dominated components using eigen-analysis of the preconditioned Hessian, thereby concentrating computational effort effectively.
• Adaptive algorithms iteratively update the LIS basis, controlling approximation errors from neglected eigenvalues to yield robust posterior uncertainty quantification.

Likelihood-informed subspace (LIS) techniques constitute a rigorous framework for dimension reduction in large-scale Bayesian inverse problems. The principal insight is that, although the parameter of interest is typically high- or infinite-dimensional, the data may only meaningfully constrain the parameter in a low-dimensional subspace. By identifying this subspace—which captures the dominant directions in which the likelihood updates the prior—one can reformulate the inference problem, concentrate computational effort where it is most needed, and achieve substantial gains in sampling efficiency and posterior uncertainty quantification.

1. Mathematical Formulation of the Likelihood-Informed Subspace

Consider a Bayesian inverse problem with observations

$$y = G(x) + e,\quad e \sim N(0, \Gamma_\text{noise}),\quad x \sim N(0, \Gamma_\text{prior}),$$

where $G(x)$ is (possibly nonlinear), and $\Gamma_\text{prior}$ and $\Gamma_\text{noise}$ are symmetric positive-definite.

The negative log-likelihood Hessian, approximated via Gauss–Newton at $x$, is

$H(x) = J(x)^T \Gamma_\text{noise}^{-1} J(x),$

with $J(x) = \nabla G(x)$.

For Gaussian priors, define the prior-preconditioned Hessian (ppGNH) as $L^T H(x) L$, where $LL^T = \Gamma_\text{prior}$ is any square root.

The local LIS at $x$ is spanned by the leading eigenvectors of

$G(x)$0

or equivalently,

$G(x)$1

Eigenvalues $G(x)$2 quantify the relative influence of the likelihood compared to the prior: directions with $G(x)$3 are data-dominated; those with $G(x)$4 are prior-dominated (Cui et al., 2014).

For nonlinear problems where $G(x)$5 varies, the global LIS is defined via the posterior expectation:

$G(x)$6

and its leading eigenpairs give an $G(x)$7-dimensional LIS basis $G(x)$8. Monte Carlo methods approximate this expectation over posterior samples.

The LIS formalizes the intrinsic low-dimensionality induced by the interplay of data, prior, and model smoothing properties.

2. Posterior Factorization and Dimension Reduction

Given the LIS basis $G(x)$9, the parameter $\Gamma_\text{prior}$0 is decomposed as $\Gamma_\text{prior}$1. The Gaussian prior then factorizes:

$\Gamma_\text{prior}$2

and the approximate posterior becomes

$\Gamma_\text{prior}$3

where $\Gamma_\text{prior}$4 is the oblique projector onto the LIS. Changing variables yields

$\Gamma_\text{prior}$5

Thus, the data-dependent Bayesian update is confined to a low-dimensional subspace. The complement remains prior-dominated and often analytically tractable (Cui et al., 2014, Cui et al., 2015).

3. Practical Construction and Algorithms

The LIS construction proceeds via adaptive algorithms:

1. Compute a MAP estimate $\Gamma_\text{prior}$6 and the LIS at $\Gamma_\text{prior}$7.
2. Project into $\Gamma_\text{prior}$8 and run a Markov chain (e.g., MALA) in $\Gamma_\text{prior}$9 targeting $\Gamma_\text{noise}$0.
3. Lift states back to $\Gamma_\text{noise}$1, update the estimated global ppGNH, enrich the LIS basis, and check for subspace convergence.
4. Repeat until change in the LIS is below threshold, as measured by a subspace distance criterion weighted by eigenvalues.

During subchain simulation, only $\Gamma_\text{noise}$2 is sampled. Posterior expectations are obtained via Rao–Blackwellization:

$\Gamma_\text{noise}$3

which strictly reduces estimator variance compared to non-factorized MCMC (Cui et al., 2014).

4. Error Analysis and Theoretical Guarantees

The LIS-based posterior approximation incurs bias controlled by neglected eigenvalues of the expected ppGNH. Under suitable regularity, for discarded subspace $\Gamma_\text{noise}$4 and residual trace $\Gamma_\text{noise}$5:

• $\Gamma_\text{noise}$6,
• $\Gamma_\text{noise}$7.

Variance in estimators decreases dramatically as the low-dimensional LIS captures all significant data-informed directions (Cui et al., 2021, Lamminpää et al., 2017).

Monte Carlo error in estimating subspace and projected marginal likelihood is rigorously controlled:

• Subspace error scales as $\Gamma_\text{noise}$8 in number of samples,
• Marginalization error scales as $\Gamma_\text{noise}$9 in auxiliary samples (Cui et al., 2021).

Dimension-independent error bounds hold for trace-class priors in the linear-Gaussian case.

5. Connections to Related Dimension Reduction Techniques

The LIS should be distinguished from prior-covariance-based methods (e.g., truncated KLE or spectral filtering):

• Prior-based reduction uses directions of maximal prior variance, regardless of data. LIS, by contrast, targets directions where data meaningfully reduce uncertainty.
• Empirical studies show LIS attains a given Hellinger or KL tolerance with dramatically fewer subspace dimensions.

Extensions of the LIS framework support:

• Data-free LIS (DF-LIS): Computing the informed subspace by averaging Fisher information over prior, yielding a data-agnostic basis usable across multiple experiments (Cui et al., 2021).
• $x$0-tempered LIS ($x$1-LIS): Generalizing LIS to tempered or intermediate posteriors (as in SMC, annealing), yielding robust dimension reduction when the data are scarce or gradients are noisy (Bouillon et al., 20 May 2026).
• Prior-normalized LIS: Transforming to Gaussianized coordinates for non-Gaussian (e.g., heavy-tailed) priors, enabling LIS construction for sparsity-promoting models (Cui et al., 2022).
• Covariance-Informed Subspace (CIS): A gradient-free dimension reduction approach that leverages posterior-to-prior covariance ratios, yielding LIS in the linear-Gaussian case and serving as a practical LIS surrogate for nonlinear settings where gradients are unavailable (Polette et al., 15 Apr 2026).

6. Numerical Performance and Application Domains

LIS-based methods demonstrate:

• Order-of-magnitude reductions in autocorrelation and computational cost for MCMC sampling, with LIS dimension often $x$2, invariant under mesh refinement or discretization (Cui et al., 2014, Cui et al., 2015).
• In remote sensing, LIS enables reduced-dimensional inference independent of discretization, outperforming both prior-based truncation and naive subspace sampling (Lamminpää et al., 2017).
• In structural engineering inverse problems and static PDE models, LIS projection yields nearly machine-precision posterior means/covariances at a $x$3–$x$4 reduction in parameter dimension (Scheffels et al., 9 Oct 2025).
• In nonlinear, highly concentrated posteriors, iterative or tempered LIS variants (e.g., CIS-SMC, $x$5-LIS) are necessary to avoid subspace degeneracy and maintain robustness (Bouillon et al., 20 May 2026, Polette et al., 15 Apr 2026).

7. Assumptions, Limitations, and Extensions

LIS methods rest on several key assumptions:

• Gaussian or Gaussianizable prior and additive noise for analytic subspace construction and factorization.
• Availability of Gauss–Newton Hessian or log-likelihood gradients; otherwise, gradient-free variants (CIS) or prior-normalization must be used (Polette et al., 15 Apr 2026, Cui et al., 2022).
• Smoothness/compactness of the forward model, ensuring rapid eigenvalue decay of the Hessian and hence low LIS dimension.

Limitations include:

• Approximate posterior bias, though dominated by variance reductions in practical problems.
• User selection of subspace thresholds and stopping criteria.
• For highly non-Gaussian or heavy-tailed problems, careful prior normalization and augmented sampling schemes are required.

LIS continues to be developed for non-Gaussian targets, hierarchical and multilevel models, and gradient-free simulation-based inference, with ongoing work in adaptive subspace updates and certified error control throughout the Bayesian computational pipeline (Cui et al., 2019, Scheffels et al., 9 Oct 2025, Bouillon et al., 20 May 2026, Polette et al., 15 Apr 2026).