---
title: Bayesian Approximation Error (BAE)
url: https://www.emergentmind.com/topics/bayesian-approximation-error-bae
type: topic
---

# Bayesian Approximation Error (BAE)

The Bayesian Approximation Error (BAE) methodology is a statistical framework for quantifying and propagating errors that result from substituting an expensive or high-fidelity forward model with a computationally cheaper but approximate surrogate in Bayesian inverse problems and uncertainty quantification. Rather than neglecting model or emulator errors, BAE systematically models the approximation error as a random variable, estimates its distribution empirically, and rigorously corrects the statistical inference to account for the resulting bias and loss of coverage. This approach underlies a broad range of techniques in PDE-constrained inversion, surrogate-based inference, optimal experimental design, and neural network emulation, providing both feasibility and increased interpretability for high-dimensional and computationally intensive scientific problems.

## 1. Formulation and Statistical Structure

The BAE framework starts from a hierarchical Bayesian formulation in which the observed data $y \in \mathbb{R}^p$ are related to unknown parameters $\theta \in \mathbb{R}^n$ and a latent process $y_p = f(\theta)$ via measurement noise $\delta \sim \mathcal{N}(0, \Gamma)$, with prior $\pi(\theta)$. The measurement model is:
$$
y = f(\theta) + \delta\,, \quad \delta \sim \mathcal{N}(0, \Gamma)
$$
where $f(\theta)$ is the "fine" or "high-fidelity" forward model. To accelerate inference, this is replaced by a surrogate $g(\theta)$, introducing a model discrepancy:
$$
\varepsilon(\theta) = f(\theta) - g(\theta)
$$
In the augmented statistical model, the surrogate error $\varepsilon(\theta)$ is treated as a random variable, generally approximated (in the basic form) as $\varepsilon \sim \mathcal{N}(\mu_\varepsilon, \Sigma_\varepsilon)$, possibly estimated from empirical samples. The total error is the sum of surrogate and measurement errors: $\nu = \varepsilon + \delta$, leading to a joint likelihood:
$$
p(y \mid \theta) = \mathcal{N}(y; g(\theta) + \mu_\varepsilon,\; \Gamma + \Sigma_\varepsilon)
$$
This formalism generalizes to scenarios with nuisance or secondary parameters, black-box forward maps, and also underpins corrective treatment for neural-network-based emulators [1810.04350, 2205.11587, 2512.05723, 2405.07412].

## 2. Construction and Estimation of Approximation Error Distributions

The crucial step in BAE is the estimation of the mean and covariance of the approximation error $\varepsilon$:
- **Offline estimation** involves drawing samples $\theta^{(\ell)}$ (either from the prior, marginal, or a naive posterior), computing both $f(\theta^{(\ell)})$ and $g(\theta^{(\ell)})$, and calculating
  $$
  \mu_\varepsilon \approx \frac{1}{q} \sum_{\ell=1}^q \varepsilon^{(\ell)},\qquad 
  \Sigma_\varepsilon \approx \frac{1}{q-1} \sum_{\ell=1}^q (\varepsilon^{(\ell)} - \mu_\varepsilon)(\varepsilon^{(\ell)} - \mu_\varepsilon)^T
  $$
- **Posterior-informed error sampling** improves estimation by focusing sampling on regions of highest posterior density under the naive surrogate, especially when the surrogate is accurate far from the bulk of the prior [1810.04350].
- **Adaptive refinement** iteratively adds new training samples in parameter regions of high posterior probability, which allows surrogate error to be adaptively reduced over multiple rounds [1807.05187].
- **Taylor approximation variance reduction** leverages Taylor expansions (first or second order) of both $f$ and $g$ to provide low-variance control variates for Monte Carlo estimation of error statistics in very high-dimensional or PDE-governed settings [2512.05723].

## 3. Impact on Likelihoods, Posterior Distributions, and Uncertainty Quantification

The effect of the BAE correction is two-fold:
- **Shift in the mean:** The data-model misfit is re-centered by the estimated $\mu_\varepsilon$, correcting for systematic bias in the surrogate.
- **Inflation of variance:** The uncertainty is inflated by $\Sigma_\varepsilon$, avoiding the underestimation of uncertainty and producing credible intervals that recover appropriate frequentist coverage.

The resulting "BAE-corrected" posterior has the form:
$$
\pi(\theta|y) \propto \pi(\theta)\,\mathcal{N}(y; g(\theta) + \mu_\varepsilon,\,\Gamma + \Sigma_\varepsilon)
$$
This structure allows the use of standard inference algorithms (MCMC, optimization, variational methods) with minimal or no modification to the computational pipeline [1810.04350, 2205.11587, 1801.03592, 2405.07412].

Empirical and theoretical results demonstrate that:
- **Naive surrogate-based inference** yields biased and overconfident posteriors whose credible intervals often exclude the true parameter [1810.04350, 2205.11587, 1807.05187].
- **BAE correction** restores the accuracy and reliability of uncertainty quantification and aligns approximate inference with the behavior of the full-model posterior, often at vastly reduced computational cost.

## 4. Methodological Variants and Applications

The generic structure of BAE admits several methodological variants tailored to specific applications:

| Variant            | Surrogate      | Error Modeling       | Key Use Case                                      |
|--------------------|---------------|---------------------|---------------------------------------------------|
| Standard BAE       | Coarse model  | Gaussian $\varepsilon$ | PDE-constrained inversion, MCMC acceleration    |
| Posterior-informed | Coarse model  | Conditioned on naive posterior | Geothermal and hydrological inference      |
| Surrogate BAE      | Emulator (NN, PCE, GP) | Empirical error from validation/test set | Cosmology, surrogate-based UQ                 |
| Error-corrected    | PCE+GP, GP+PCE| Secondary surrogate for residual | High-dimensional, multimodal posteriors        |
| Taylor-variance    | Linear/Quadratic approximations | Control variate corrections | Large-scale PDEs, variance reduction           |

Applications include Bayesian inverse modeling in geothermal and hydrological systems, estimation of boundary conditions and parameters in PDEs with model uncertainty, cosmological parameter inference with neural nets, optimal design of experiments in high-dimensional settings, and linearized or surrogate-based infinite-dimensional Bayesian inversion [1810.04350, 1807.05187, 2205.11587, 2512.05723, 2211.03952, 2405.07412].

## 5. Error Bounds, Theoretical Properties, and Limitations

Recent work has established non-asymptotic and asymptotic bounds on the effect of surrogate/model error in Bayesian inference:
- **Hellinger and Wasserstein metrics** quantify the distance between exact and approximate posteriors, with explicit bounds in terms of the $L^p$-norms of the surrogate/model errors and integrability assumptions [1911.05669, 1809.09505].
- **Strong convexity and smoothness assumptions** enable tractable control of the posterior approximation error using gradients (Fisher distance), facilitating practical diagnostics and control of error in Laplace approximation, variational Bayes, and Hilbert coreset settings [1809.09505].
- **Algebraic-geometric analysis** in singular or non-regular models, such as non-negative matrix factorization, shows that the separation in negative log-evidence (free energy) between variational and fully Bayesian posteriors scales as the gap between their respective real log canonical thresholds (RLCT), with explicit lower bounds on BAE as a function of model rank and hyperparameters [1809.02963].
- Corollaries guarantee that with sufficient sampling, the bias and loss of coverage induced by surrogates can be made arbitrarily small, but in practice, the Gaussianity and independence assumptions on $\varepsilon$ are limiting in extreme regimes [1911.05669, 1810.04350].

## 6. Integration with Optimal Experimental Design (OED)

The BAE methodology has become integral to modern OED for PDE-based inverse problems and sensor placement:
- The inclusion of the BAE-corrected error covariance in the design criterion (e.g., A-optimality) ensures that designs do not exploit spurious confidence in an inaccurate model and are robust to model uncertainty [2211.03952].
- Recent advances prove that, under the BAE correction, the design objective is independent of the choice of surrogate or linearization, permitting sensor selection using only sample-based error statistics and a trivial surrogate (the zero map), which is practical for black-box forward models [2405.07412].
- Marginal designs accounting for both primary and nuisance parameters can be constructed, again using the BAE-corrected moments, to yield designs that maintain feasibility and accuracy across model misspecification regimes.

## 7. Practical Implementation Considerations

- **Offline computational cost**: The dominant cost lies in evaluating the high-fidelity model at sufficiently many sample points to estimate $\mu_\varepsilon$ and $\Sigma_\varepsilon$. Techniques such as Taylor expansion control variates, local corrections, adaptive sampling, and low-rank matrix approximations dramatically reduce sample requirements in high-dimensional settings [2512.05723].
- **Online inference**: Once error statistics are estimated, BAE-corrected inference incurs minimal additional computational cost over standard surrogate-based inference.
- **Surrogate selection**: The quality of the BAE correction depends strongly on the global fidelity of the surrogate. If the surrogate fails to reproduce core physical features, the correction may be too pessimistic or unreliable.
- **Diagnostics**: Feasibility and coverage of credible intervals should always be assessed by validation against ground truth or additional high-fidelity samples from the posterior region [1810.04350, 1807.05187].
- **Limitations**: The basic BAE approach assumes approximately Gaussian surrogate errors, stationarity, and (for computational simplicity) independence from parameters. Substantial non-Gaussian or strongly parameter-dependent errors require methodological extensions, such as nonparametric error models or local Gaussianizations [1810.04350, 2512.05723].

### References

- [1810.04350] Incorporating Posterior-Informed Approximation Errors into a Hierarchical Framework to Facilitate Out-of-the-Box MCMC Sampling for Geothermal Inverse Problems and Uncertainty Quantification
- [2205.11587] Bayesian error propagation for neural-net based parameter inference
- [2512.05723] Taylor Approximation Variance Reduction for Approximation Errors in PDE-constrained Bayesian Inverse Problems
- [1807.05187] Surrogate-Based Bayesian Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Approximation Error
- [1809.09505] Practical bounds on the error of Bayesian posterior approximations: A nonasymptotic approach
- [2211.03952] Optimal design of large-scale nonlinear Bayesian inverse problems under model uncertainty
- [2405.07412] Non-intrusive optimal experimental design for large-scale nonlinear Bayesian inverse problems using a Bayesian approximation error approach
- [1809.02963] Variational Approximation Error in Bayesian Non-negative Matrix Factorization
- [1911.05669] Error bounds for some approximate posterior measures in Bayesian inference
- [1801.03592] Estimation of the Robin coefficient field in a Poisson problem with uncertain conductivity field
- [1901.00909] A Bayesian approach to improving the Born approximation for inverse scattering with high contrast materials

Source: https://www.emergentmind.com/topics/bayesian-approximation-error-bae