Bayesian Approximation Error (BAE) Approach

• BAE is a statistical method that quantifies and propagates surrogate model errors in Bayesian inverse problems.
• It integrates error estimates into the likelihood by adjusting the mean and inflating the covariance to debias posteriors and capture uncertainty.
• BAE is applied in fields like cosmology, hydrology, and PDE inversions to accelerate computations while ensuring robust parameter estimates.

The Bayesian Approximation Error (BAE) approach is a systematic statistical framework for quantifying and propagating surrogate or model approximation errors within Bayesian inverse problems and uncertainty quantification tasks. It enables the use of computationally inexpensive surrogate models while rigorously correcting for their bias and uncertainty, ensuring unbiased posteriors and feasible uncertainty estimates. The methodology is widely applicable: from neural-net–accelerated cosmological inference, hydrological inversion, inverse acoustic scattering, PDE-constrained inference, optimal experimental design, to quantification of discretization error in finite element models.

1. Formal Definition and Core Principles

Given a true (often computationally expensive or unimplementable) forward model $f(\theta)$ and a surrogate or approximative model $g(\theta)$, the BAE approach introduces the pointwise model error

$\varepsilon(\theta) = f(\theta) - g(\theta)$

and treats $\varepsilon$ as an additional random variable in the highest-level Bayesian hierarchy (Grandón et al., 2022, Maclaren et al., 2018, Kaipio et al., 2019). Bayesian inference thus involves marginalizing over all sources of uncertainty: parameters, data noise, and model approximation error. This is encoded in a hierarchical joint posterior

$$p(\theta, \varepsilon | d) \propto p(d | \theta, \varepsilon) p(\varepsilon) p(\theta)$$

with choice of prior on $\theta$, data noise $\eta= d - f(\theta)$, and approximation error model $$\varepsilon | \theta \sim N(\mu_\varepsilon, \Sigma_\varepsilon)$$.

The central step is to modify the likelihood by integrating over unobserved error: $$p(d | \theta) = \int p(d | \theta, \varepsilon) p(\varepsilon) d\varepsilon$$ For Gaussian assumptions (standard in high-dimensional problems), the convolution yields a closed-form effective likelihood: $$p(d | \theta) = N(d; g(\theta) + \mu_\varepsilon, \Sigma_d + \Sigma_\varepsilon)$$ This shifts the surrogate’s mean by $\mu_\varepsilon$ (debiasing) and inflates the covariance by $\Sigma_\varepsilon$ (uncertainty quantification).

2. Workflow: Error Estimation and Posterior Correction

Following the formal structure, the BAE workflow consists of:

• Surrogate training: Learn $g(\theta)$ using a training set $\{\theta_i, f(\theta_i)\}$.
• Validation set acquisition: Obtain a large, independent set $\{\theta_v\}$ and compute errors $\varepsilon_v = f(\theta_v) - g(\theta_v)$.
• Moment estimation: Estimate error mean and covariance:

$$\mu_\varepsilon = \frac{1}{V} \sum_{v=1}^V \varepsilon_v, \quad \Sigma_\varepsilon = \frac{1}{V-1} \sum_{v=1}^V (\varepsilon_v - \mu_\varepsilon)(\varepsilon_v - \mu_\varepsilon)^T$$

• Posterior evaluation: Employ the adapted likelihood and run Bayesian samplers (e.g., MCMC, nested sampling) with negligible additional cost—only the single pass error evaluation incurs extra computation.

Empirical validation, such as in cosmological parameter inference, shows the BAE-modified posterior correctly recovers unbiased intervals and accurate uncertainty quantification even for surrogates with substantive systematic errors, whereas naïve surrogate posteriors are strongly biased and overconfident (Grandón et al., 2022).

3. Extension to Surrogates, Hybrid Models, and Adaptive Refinement

The methodology applies broadly to various surrogates, including neural networks, polynomial chaos expansions (PCE), Gaussian processes, and combinations thereof. For hydrological and PDE-constrained inversions, two canonical strategies are established (Zhang et al., 2018, Nicholson et al., 5 Dec 2025):

• Strategy A (Surrogate-prediction uncertainty): Use Bayesian predictive variance (e.g., from GP surrogates) as error covariance in the BAE likelihood.
• Strategy B (Secondary surrogate for error): Train an explicit error-surrogate $\delta_s(\theta)$ to directly model $f(\theta) - g_s(\theta)$, optionally hybridized (PCE+GP).

Adaptive refinement iteratively enhances the surrogate and error models in the posterior’s support, drawing new training points where posterior density is high, continuously improving accuracy where most needed.

4. Specialized BAE Formulations: PDEs, OED, Marginalization, and Discretization Errors

PDE-constrained problems: The BAE approach quantifies model error from neglected unknowns (e.g., uncertain conductivity in boundary value problems (Nicholson et al., 2018)). Offline Monte Carlo, or control-variates based on Taylor expansions, efficiently estimate error moments, substantially reducing the required number of high-fidelity PDE solves (Nicholson et al., 5 Dec 2025). This enables tractable, scalable Bayesian inference in high-dimensional settings.

Optimal Experimental Design (OED): The BAE-corrected likelihood and local posterior covariance are integrated into the A-optimal design criterion. Sensor selection is performed via trace minimization of BAE-aware posterior covariance, using greedy algorithms and scalable approximations (low-rank Hessians, trace estimation) (Alexanderian et al., 2022, Koval et al., 13 May 2024). The BAE framework is proven invariant to the choice of linearization in the OED objective, allowing fully non-intrusive implementations.

Marginalization: Uncertain or nuisance parameters (e.g., unknown auxiliary fields) are premarginalized via BAE, yielding a corrected likelihood and feasible uncertainty estimates for primary inversion parameters (Nicholson et al., 2018, Koval et al., 13 May 2024).

Finite Element Discretization Error: BAE is used to statistically represent the uncertainty due to discretization. Gaussian process priors are conditioned on coarse mesh results, with posterior covariance rescaled via eigenmode projections to recover load-dependent error structure (Poot et al., 2023).

5. Rigorous Error Bounds and Posterior Proximity

Theoretical guarantees are provided for BAE approximations. Error bounds for poste-rior measures arising from random/deterministic surrogate errors establish that, under exponential integrability and bounded $L^p$-norm surplus error, the approximate Bayesian posterior remains $O(\text{error})$-close to the true posterior (in Hellinger distance) (Lie et al., 2019). The Bayes update map is locally Lipschitz, quantifying how errors in surrogate models propagate to errors in posteriors.

6. Algorithmic Approaches and Computational Aspects

Algorithmic highlights include:

• Offline error sampling and estimation: High-fidelity model evaluations for error computation are performed in parallel; only a one-time cost.
• BAE-corrected likelihood: Integrates seamlessly with standard sampling and optimization algorithms (Metropolis–Hastings, Newton–CG, Laplace approximation).
• Efficiency: Posterior sampling is accelerated by orders of magnitude versus fine-model posteriors, particularly in settings leveraging neural networks and reduced-order models (Grandón et al., 2022, Maclaren et al., 2018).
• Adaptive refinement: Posterior-aware error modeling and surrogate updating concentrate computational resources in high-probability regions (Zhang et al., 2018).
• Variance reduction: Taylor-based control variates reduce the number of required expensive model evaluations by factors of 5–10× (Nicholson et al., 5 Dec 2025).

7. Impact, Limitations, and Recommended Practices

The BAE approach has demonstrated efficacy across disciplines—from cosmology to hydrology, geothermal systems, inverse scattering, optimal design, and numerical PDE uncertainty. By correcting for both bias and uncertainty from surrogates, BAE ensures feasible and reliable inference where naïve surrogate posteriors systematically fail (Grandón et al., 2022, Maclaren et al., 2018, Kaipio et al., 2019, Nicholson et al., 2018). Its flexibility and computational efficiency are substantiated in complex, high-dimensional, multiparameter, and multimodal cases.

Key recommendations include: empirical Gaussianity/independence checks for error samples, posterior-informed error estimation (sampling error statistics in high-posterior regions), and careful construction of the training prior for offline error model estimation. Surrogate model linearity or affine structure simplifies posterior computation, but for strongly non-Gaussian or nonlinear errors, further methodological development (e.g., non-Gaussian BAE, advanced MCMC schemes) may be necessary.

Theoretical limitations arise from reliance on the Gaussian error model and independence assumptions. In cases of pronounced nonstationary, non-Gaussian, or parameter-dependent errors, these require explicit non-Gaussian generalization, with increased computational and modeling complexity.

---

Summary Table: Key Elements of BAE Approach

Aspect	Description	Reference
Error Representation	$\varepsilon = f(\theta) - g(\theta)$, modeled as Gaussian	(Grandón et al., 2022, Maclaren et al., 2018)
Likelihood Correction	Marginalization over $\varepsilon$; covariance inflation, debiasing	(Grandón et al., 2022, Zhang et al., 2018, Kaipio et al., 2019)
Error Estimation	Empirical moments from validation/training set	(Grandón et al., 2022, Nicholson et al., 5 Dec 2025)
Adaptive Refinement	Posterior-aware surrogate/error enhancement	(Zhang et al., 2018)
Theoretical Bounds	Rigorous error bounds (Hellinger distance)	(Lie et al., 2019)

The BAE framework thus provides a rigorous, efficient, and widely adaptable mechanism for uncertainty quantification and bias correction in Bayesian inference with surrogate models, ensuring posterior credibility and computational tractability across a spectrum of challenging inverse problems.