Bayesian Inverse Problems

• Bayesian inverse problems are a statistical framework that recovers unknown parameters or fields from indirect, noisy observations using forward models and prior information.
• They systematically integrate noise models and prior distributions to generate posterior measures, enabling rigorous uncertainty quantification and regularization.
• Efficient computational methods such as MCMC, variational inference, and neural surrogates are employed to tackle high-dimensional, ill-posed inverse problems in various applied domains.

Bayesian inverse problems are a class of statistical inference tasks in which an unknown parameter, function, or field is to be recovered from indirect and noisy observations via the solution of a (possibly ill-posed) forward model. The Bayesian approach systematically incorporates prior knowledge, noise models, and the structure of the forward map to yield a posterior probability measure over unknowns, facilitating uncertainty quantification, nonuniqueness, and regularization. This framework is central in contemporary applied mathematics, statistics, scientific computing, and engineering.

1. Mathematical Formulation and General Structure

A Bayesian inverse problem entails a measurable space for the unknown $u$ (parameter, function, or measure), a data space $Y$, and a forward map $G:X\rightarrow Y$ linking $u$ to ideal observations. The measurement model augments $G(u)$ with a stochastic noise variable $\eta$,

$y = G(u) + \eta.$

A prior probability measure $\mu_0$ on $u$ encodes physical, mathematical, or empirical knowledge; a likelihood arises from the noise model, e.g., for $\eta\sim\mathcal{N}(0,\Sigma)$,

$$p(y|u) \propto \exp\left(-\frac{1}{2}\|y - G(u)\|^2_{\Sigma^{-1}}\right).$$

Bayes' theorem yields the posterior,

$$\frac{d\mu^y}{d\mu_0}(u) \propto \exp(-\Phi(u;y)), \quad \Phi(u;y)=\frac{1}{2}\|y-G(u)\|^2_{\Sigma^{-1}}.$$

For function or field inference, $X$ is often a Hilbert or Banach space, and $G$ may be a solution operator to an ODE, PDE, or integral equation. The Bayesian framework persists in infinite-dimensions under appropriate measurability, integrability, and regularity conditions (Dashti et al., 2013, Bui-Thanh et al., 2013, Huynh, 30 Apr 2025).

2. Prior Modeling and Regularization

Priors govern both well-posedness and the informativeness of the solution. Canonical choices include:

• Gaussian Random Field Priors: Specified via a mean (often zero) and covariance operator $C$, e.g., Whittle–Matérn fields defined via fractional elliptic SPDEs,

$$(\kappa_0^2 - \nabla\cdot(a\nabla))^\alpha u = \mathcal{W},$$

with sample regularity depending on $\alpha$ (Bolin et al., 25 Jul 2025, Dashti et al., 2013, Knapik et al., 2011).

• Sparse or Measure-Valued Priors: To promote sparsity or atomicity, e.g., compound Poisson processes in spaces of Radon measures,

$u = \sum_{k=1}^K \gamma_k Q_k \delta_{Y_k}, \quad K\sim{\rm Poi}(\lambda), \ Q_k \sim \text{distribution in %%%%13%%%%}, \ Y_k \sim \text{Uniform}(D),$

leading to a posterior on measure-valued parameters (Huynh, 30 Apr 2025).

• Nonparametric and Hierarchical Priors: Series priors using eigenbasis expansions, hierarchical scale mixtures, and empirical Bayes schemes are employed to allow data-driven adaptivity (Knapik et al., 2011, Agapiou et al., 2021, Bochkina et al., 2019, Gugushvili et al., 2018).
• Non-Gaussian and Deep Generative Priors: When the prior must encapsulate empirical or data-driven features, deep generative models (e.g., GAN priors) or neural operator-based surrogates may be used, projecting the high-dimensional prior into an efficient latent space (Patel et al., 2021, Kaltenbach et al., 2022, Mohammad-Djafari et al., 2 Dec 2025).

The chosen prior regularity and structure fundamentally determine the contraction rates, identifiability, and credible set properties of the posterior.

3. Well-Posedness and Stability of the Bayesian Posterior

The posterior $\mu^y$ is said to be well-posed if it is a probability measure that depends stably on the data. Sufficient conditions include:

• Regularity of the forward map $G$, typically locally Lipschitz and boundedness on level sets,
• Coercivity, ellipticity, or stability properties of differential and integral operators,
• Sufficient integrability of the prior (e.g., $u\in H^{3/2}(\Gamma)$ almost surely for Gaussian Whittle–Matérn priors with $\alpha > 1$ ensures a unique solution for the fractional elliptic forward problem on a metric graph (Bolin et al., 25 Jul 2025)).

Stability is quantified via the Hellinger distance: For fixed data $y, y'$, if

$d_H(\mu^y, \mu^{y'}) \lesssim \|y-y'\|,$

then the Bayesian posterior is robust to data perturbations. These results are rigorously established for Bayesian inverse problems on infinite-dimensional spaces (Dashti et al., 2013, Huynh, 30 Apr 2025, Bolin et al., 25 Jul 2025).

4. Posterior Computation: Inference Algorithms and Numerical Methods

The solution of Bayesian inverse problems in practice demands efficient computational strategies:

• Markov chain Monte Carlo (MCMC): The preconditioned Crank–Nicolson (pCN) algorithm and variants (e.g., MALA, HMC) are dimension-robust samplers suited for Gaussian priors and function spaces. The proposal

$$u' = \sqrt{1-\tau^2}\, u + \tau \xi, \quad \xi\sim \mu_0,$$

together with Metropolis–Hastings acceptance, preserves the prior measure and achieves mesh-independence (Dashti et al., 2013, Bolin et al., 25 Jul 2025).

• Variational Inference: Approximates the posterior within parametric families by maximizing the evidence lower bound (ELBO),

$$\mathcal{L}(q) = \mathbb{E}_q[\ln p(y|u)] - \operatorname{KL}(q(u)||p(u)),$$

either in the mean-field, mixture, or exponential family (with optimization via coordinate ascent or homotopy ODEs) (Tsilifis et al., 2014, Yang et al., 2022).

• Gaussian/Laplace Approximations: For nearly linear or locally Gaussian problems, the posterior is approximated by a Gaussian centered at the MAP estimate with covariance the inverse Hessian of the negative log-posterior at the MAP,

$$\mu^{\text{Lap}} = \mathcal{N}(u_{\rm MAP}, [D^2 I(u_{\rm MAP})]^{-1}),$$

with bounds on the Hellinger error quantified by higher-order Taylor remainders (Wacker, 2017).

• Reversible-Jump MCMC: For measures (e.g., sparse Dirac mixtures), RJMCMC navigates varying parameter spaces associated with random support/weights (Huynh, 30 Apr 2025).
• Deep Operator and Neural Approaches: Neural surrogates (e.g., invertible DeepONets, BPINN) enable both forward solves and approximate Bayesian inference via learned latent representations or MC dropout posteriors (Kaltenbach et al., 2022, Mohammad-Djafari et al., 2 Dec 2025).

Efficient sampling often depends on combining low-rank parameterizations, latent-variable reductions, or emulators to alleviate the computational burden of repeated PDE solves in high dimensions (Bui-Thanh et al., 2013, Patel et al., 2021).

5. Posterior Consistency, Contraction Rates, and Uncertainty Quantification

Given true unknown $u_0$ and growing data, consistency of the posterior demands that, as noise decreases or sample size increases, posterior mass concentrates around $u_0$ at the optimal rate (often matching the minimax error of regularization theory):

• For Gaussian priors in mildly ill-posed linear inverse problems (singular values decay $\asymp i^{-p}$), contraction rate is

$\varepsilon_n \asymp n^{-\beta/(1+2\beta+2p)},$

for a $\beta$-Sobolev truth and prior regularity $\alpha\approx\beta$ (Knapik et al., 2011, Gugushvili et al., 2018, Bochkina et al., 2019).

• For nonlinear inverse PDEs (e.g., elliptic coefficient field inference, Robin boundary problems), rates may be logarithmic (Sobolev regularity) or algebraic (analytic class), with rescaling of the prior's RKHS-norm crucial for adaptation (Rasmussen et al., 2023).
• Uncertainty quantification is provided by the posterior covariance, marginal credible intervals, or full trajectories of solution fields. The choice of prior (rougher, matched, or smoother than the truth) directly impacts credible set coverage: matched or slightly undersmoothed priors yield well-calibrated inference; oversmoothed priors lead to suboptimal rates and miscalibrated bands (Knapik et al., 2011, Agapiou et al., 2021).
• Multimodality and nonuniqueness are addressed: the posterior can be multimodal (reflecting nonidentifiability), and local MAP (LMAP) and local conditional mean (LCM) estimators can extract all plausible solutions (Sun, 2021).

6. Advances, Extensions, and Domain-Specific Applications

Bayesian inverse problems have advanced in several directions:

• Metric Graphs: Extension to networks (e.g., power grids, traffic, quantum graphs), with Whittle–Matérn priors and adapted well-posedness/stability theory (Bolin et al., 25 Jul 2025).
• Spatiotemporal Modeling: Incorporation of spatiotemporal Gaussian process priors in inverse settings improves Fisher information, parameter recovery, and uncertainty quantification for subsystems governed by spatiotemporal PDEs or chaotic ODEs (Lan et al., 2022).
• Unknown Operators: Bayesian analysis extends to inverse problems where the forward map itself is uncertain or only partially observed, leveraging product priors with empirical Bayes adaptation (Trabs, 2018).
• High-dimensional/Deep Surrogates: Neural operator surrogates—either as generative priors or as learned push-forward/posterior maps—dramatically reduce inference costs for large-scale and high-dimensional inverse problems, especially in image deconvolution, super-resolution, or parametric PDE identification (Patel et al., 2021, Mohammad-Djafari et al., 2 Dec 2025).
• Measure-valued Unknowns/Sparsity: Poisson or compound point-process priors on measure spaces model sources or fields with sparse or atomic structure, crucial for applications such as acoustics, medical imaging, or sparse signal recovery (Huynh, 30 Apr 2025).

These extensions rely on rigorous regularity, stability, and computational frameworks developed for classical infinite-dimensional settings and extended to new data types, priors, or operators.

7. Practical Considerations and Best Practices

To ensure rigorous Bayesian inversion:

• Choose prior regularity to (at least) match or slightly undersmooth the expected unknowns. Use empirical Bayes techniques, priors with built-in adaptivity/hyperpriors, or posterior checks to guard against oversmoothing.
• Employ dimension-robust samplers (e.g., pCN, ∞-MALA, SMC), scalable representations (e.g., low-rank covariance approximations, neural operator surrogates), and mesh-independent algorithms for large parameter spaces (Bui-Thanh et al., 2013, Dashti et al., 2013).
• Monitor convergence and coverage of credible sets, especially in multimodal or nonunique settings. Use local estimators or geometric set summaries where appropriate (Sun, 2021).
• Validate via simulation: compare Bayesian posteriors with frequentist regularization, assess coverage and contraction, and, in applied contexts, check against ground truth or established benchmarks (Mohammad-Djafari et al., 2 Dec 2025, Rasmussen et al., 2023).

The Bayesian approach offers a principled, interpretable, and extensible methodology for inversion, allowing for robust quantification of uncertainty, optimal estimation, and principled incorporation of prior knowledge in complex, noisy, and high-dimensional inference tasks.