---
title: Likelihood-Informed Subspace Techniques
url: https://www.emergentmind.com/topics/likelihood-informed-subspace-lis-techniques
type: topic
---

# Likelihood-Informed Subspace Techniques

Likelihood-informed subspace (LIS) techniques constitute a rigorous framework for dimension reduction in large-scale Bayesian inverse problems. The principal insight is that, although the parameter of interest is typically high- or infinite-dimensional, the data may only meaningfully constrain the parameter in a low-dimensional subspace. By identifying this subspace—which captures the dominant directions in which the likelihood updates the prior—one can reformulate the inference problem, concentrate computational effort where it is most needed, and achieve substantial gains in sampling efficiency and posterior uncertainty quantification.

## 1. Mathematical Formulation of the Likelihood-Informed Subspace

Consider a Bayesian inverse problem with observations 
$$
y = G(x) + e,\quad e \sim N(0, \Gamma_\text{noise}),\quad x \sim N(0, \Gamma_\text{prior}),
$$
where $G(x)$ is (possibly nonlinear), and $\Gamma_\text{prior}$ and $\Gamma_\text{noise}$ are symmetric positive-definite.

The negative log-likelihood Hessian, approximated via Gauss–Newton at $x$, is
$$
H(x) = J(x)^T \Gamma_\text{noise}^{-1} J(x),
$$
with $J(x) = \nabla G(x)$.

For Gaussian priors, define the prior-preconditioned Hessian (ppGNH) as $L^T H(x) L$, where $LL^T = \Gamma_\text{prior}$ is any square root.

The **local LIS** at $x$ is spanned by the leading eigenvectors of
$$
L^T H(x) L v = \lambda v,
$$
or equivalently,
$$
H(x) u = \lambda \Gamma_\text{prior}^{-1} u, \quad u = Lv.
$$
Eigenvalues $\lambda$ quantify the relative influence of the likelihood compared to the prior: directions with $\lambda \gg 1$ are data-dominated; those with $\lambda \ll 1$ are prior-dominated [1403.4680].

For nonlinear problems where $H(x)$ varies, the **global LIS** is defined via the posterior expectation:
$$
S = \mathbb{E}_{x \sim \pi(x|y)} [ L^T H(x) L ],
$$
and its leading eigenpairs give an $r$-dimensional LIS basis $U_r = L \Phi_r$. Monte Carlo methods approximate this expectation over posterior samples.

The LIS formalizes the intrinsic low-dimensionality induced by the interplay of data, prior, and model smoothing properties.

## 2. Posterior Factorization and Dimension Reduction

Given the LIS basis $U_r$, the parameter $x$ is decomposed as $x = U_r x_r + U_\perp x_\perp$. The Gaussian prior then factorizes:
$$
\pi_0(x) = \pi_r(x_r) \,\pi_\perp(x_\perp),
$$
and the **approximate posterior** becomes
$$
\tilde{\pi}(x \mid y) \propto \pi(y \mid \Pi_r x)\, \pi_0(x),
$$
where $\Pi_r = U_r \Xi_r^T$ is the oblique projector onto the LIS. Changing variables yields
$$
\tilde{\pi}(x_r, x_\perp \mid y) = \tilde{\pi}(x_r \mid y)\, \pi_\perp(x_\perp),\qquad
\tilde{\pi}(x_r \mid y) \propto \pi(y \mid U_r x_r)\, \pi_r(x_r).
$$
Thus, the data-dependent Bayesian update is confined to a low-dimensional subspace. The complement remains prior-dominated and often analytically tractable [1403.4680, 1510.06053].

## 3. Practical Construction and Algorithms

The LIS construction proceeds via adaptive algorithms:

1. Compute a MAP estimate $x_\text{MAP}$ and the LIS at $x_\text{MAP}$.
2. Project into $x_r$ and run a Markov chain (e.g., MALA) in $\mathbb{R}^r$ targeting $\tilde{\pi}(x_r|y)$.
3. Lift states back to $x$, update the estimated global ppGNH, enrich the LIS basis, and check for subspace convergence.
4. Repeat until change in the LIS is below threshold, as measured by a subspace distance criterion weighted by eigenvalues.

During subchain simulation, only $x_r$ is sampled. Posterior expectations are obtained via **Rao–Blackwellization**:
$$
\mathbb{E}[h(x)] = \mathbb{E}_{x_r}\left[ \int h(U_r x_r + U_\perp x_\perp)\, \pi_\perp(dx_\perp)\right ],
$$
which strictly reduces estimator variance compared to non-factorized MCMC [1403.4680].

## 4. Error Analysis and Theoretical Guarantees

The LIS-based posterior approximation incurs **bias controlled by neglected eigenvalues** of the expected ppGNH. Under suitable regularity, for discarded subspace $U_\perp$ and residual trace $R = \sum_{i>r} \gamma_i$:
- $\mathrm{KL}(\pi \| \tilde{\pi}) \leq (C/2) R$,
- $\text{Hellinger}^2(\pi, \tilde{\pi}) \leq (1/4) R$.

Variance in estimators decreases dramatically as the low-dimensional LIS captures all significant data-informed directions [2101.02417, 1709.02611].

Monte Carlo error in estimating subspace and projected marginal likelihood is rigorously controlled:
- Subspace error scales as $O(1/\sqrt{N})$ in number of samples,
- Marginalization error scales as $O(1/\sqrt{M})$ in auxiliary samples [2101.02417].

Dimension-independent error bounds hold for trace-class priors in the linear-Gaussian case.

## 5. Connections to Related Dimension Reduction Techniques

The LIS should be distinguished from prior-covariance-based methods (e.g., truncated KLE or spectral filtering):
- Prior-based reduction uses directions of maximal prior variance, regardless of data. LIS, by contrast, targets directions where data meaningfully reduce uncertainty.
- Empirical studies show LIS attains a given Hellinger or KL tolerance with dramatically fewer subspace dimensions.

Extensions of the LIS framework support:
- **Data-free LIS (DF-LIS)**: Computing the informed subspace by averaging Fisher information over prior, yielding a data-agnostic basis usable across multiple experiments [2102.13245].
- **$\alpha$-tempered LIS ($\alpha$-LIS)**: Generalizing LIS to tempered or intermediate posteriors (as in SMC, annealing), yielding robust dimension reduction when the data are scarce or gradients are noisy [2605.21717].
- **Prior-normalized LIS**: Transforming to Gaussianized coordinates for non-Gaussian (e.g., heavy-tailed) priors, enabling LIS construction for sparsity-promoting models [2202.00074].
- **Covariance-Informed Subspace (CIS)**: A gradient-free dimension reduction approach that leverages posterior-to-prior covariance ratios, yielding LIS in the linear-Gaussian case and serving as a practical LIS surrogate for nonlinear settings where gradients are unavailable [2604.13563].

## 6. Numerical Performance and Application Domains

LIS-based methods demonstrate:
- Order-of-magnitude reductions in autocorrelation and computational cost for MCMC sampling, with LIS dimension often $r \ll n$, invariant under mesh refinement or discretization [1403.4680, 1510.06053].
- In remote sensing, LIS enables reduced-dimensional inference independent of discretization, outperforming both prior-based truncation and naive subspace sampling [1709.02611].
- In structural engineering inverse problems and static PDE models, LIS projection yields nearly machine-precision posterior means/covariances at a $10$–$100\times$ reduction in parameter dimension [2510.07950].
- In nonlinear, highly concentrated posteriors, iterative or tempered LIS variants (e.g., CIS-SMC, $\alpha$-LIS) are necessary to avoid subspace degeneracy and maintain robustness [2605.21717, 2604.13563].

## 7. Assumptions, Limitations, and Extensions

LIS methods rest on several key assumptions:
- Gaussian or Gaussianizable prior and additive noise for analytic subspace construction and factorization.
- Availability of Gauss–Newton Hessian or log-likelihood gradients; otherwise, gradient-free variants (CIS) or prior-normalization must be used [2604.13563, 2202.00074].
- Smoothness/compactness of the forward model, ensuring rapid eigenvalue decay of the Hessian and hence low LIS dimension.

Limitations include:
- Approximate posterior bias, though dominated by variance reductions in practical problems.
- User selection of subspace thresholds and stopping criteria.
- For highly non-Gaussian or heavy-tailed problems, careful prior normalization and augmented sampling schemes are required.

LIS continues to be developed for non-Gaussian targets, hierarchical and multilevel models, and gradient-free simulation-based inference, with ongoing work in adaptive subspace updates and certified error control throughout the Bayesian computational pipeline [1910.12431, 2510.07950, 2605.21717, 2604.13563].

Source: https://www.emergentmind.com/topics/likelihood-informed-subspace-lis-techniques