Approximate Bayesian Computation (ABC)

• ABC is a simulation-based method that replaces explicit likelihood evaluation with summary statistic matching to approximate Bayesian posteriors.
• Advanced strategies like regression adjustments, kernel methods, and emulation enhance accuracy and efficiency in inferential tasks.
• ABC has diverse applications in population genetics, ecology, finance, and engineering, enabling robust inference in complex models.

Approximate Bayesian Computation (ABC) is a class of simulation-based methods for Bayesian inference under models where the likelihood is computationally expensive or analytically intractable, but for which forward simulation is feasible. Instead of directly evaluating the likelihood, ABC algorithms perform parameter inference and model selection by simulating datasets from the generative model, reducing both observed and simulated data to summary statistics, and then accepting parameters whose simulated summaries are sufficiently close to those of the real data—effectively approximating the posterior without explicit likelihood computation.

1. Foundations and Core Principles

ABC replaces explicit likelihood evaluation with a simulation–based accept/reject or weighting scheme, using summary statistics to compare observed and simulated data. The distinctive workflow comprises the following fundamental steps:

1. Simulation: Draw a parameter $\theta$ from the prior and simulate data $y$ under the model $p(y|\theta)$.
2. Summary Statistics: Reduce both simulated data $y$ and observed data $y_0$ to summary statistics $S(y)$ and $S(y_0)$.
3. Distance and Tolerance: Compute the distance $d(S(y), S(y_0))$ and accept the parameter $\theta$ if the distance is less than a tolerance $\epsilon$.
4. Posterior Approximation: The accepted $\theta$ values represent draws from an approximate posterior.

The ABC algorithm is not restricted to a specific domain; applications include demographic inference in population genetics, ecological models, and complex biological processes where exact likelihoods are prohibitive (Csilléry et al., 2011).

The approximation error stems from (1) the potential insufficiency of summary statistics; (2) the non-zero tolerance $\epsilon$; and (3) the finite number of simulations. In the ideal limit—with sufficient statistics and $\epsilon \to 0$—the ABC target converges to the exact posterior, but in practice, information loss and computational costs limit this convergence (Nakagome et al., 2012).

2. Algorithms and Adjustments

ABC methods have evolved from simple rejection schemes to sophisticated algorithms incorporating regression adjustments, kernel methods, and surrogate modeling to mitigate loss of efficiency and information.

Table 1: Major ABC Algorithmic Classes

Algorithm	Acceptance Rule	Adjustment/Enhancement
Rejection ABC	$d(S(y), S(y_0)) < \epsilon$	None
Regression ABC	As above	Local linear regression or neural net
Kernel ABC	Kernel weighting in summary space	RKHS mapping, automatic shrinkage
Emulation ABC	Use surrogate of simulator	Local regressions, Gaussian processes
Copula/Adaptive ABC	Regression + copula modeling	Gaussian copula, adaptive complexity
Robust-ABC	Partition summaries, adjust for misspecification	Additive correction $\Gamma$ to incompatible summaries

In regression-adjusted ABC, post-processing refines the posterior by modeling the relationship $\theta = m(S(y)) + \varepsilon$ and correcting simulated draws: $$\theta_i^* = \hat{m}(S(y_0)) + (\theta_i - \hat{m}(S(y_i)))$$, where $m(\cdot)$ can be estimated by local linear regression, neural networks, or nonlinear heteroscedastic regression (Csilléry et al., 2011). The “loclinear” and “neuralnet” methods specifically address nonlinearity and heteroscedasticity.

Kernel-based ABC extends regression adjustment by mapping summaries to a reproducing kernel Hilbert space (RKHS), allowing kernel ridge regression and enabling the use of high-dimensional summaries without handcrafted selection and without suffering variance explosion typical in naïve rejection ABC (Nakagome et al., 2012). The kernel ABC posterior mean for a test statistic $s$ is estimated by

$$E[\theta \mid s] \approx \sum_{i=1}^n W_i \theta_i,$$

with weights $W_i$ derived from the kernel Gram matrix.

Surrogate (emulator-based) ABC substitutes costly model simulations with statistical emulators—local regressions or Gaussian processes—trained on a budgeted design, with stochasticity preserved via local residual estimation (Jabot et al., 2014).

Recent developments include the robust-ABC method, which partitions the summary vector into “well-matched” and “misspecified” blocks, introducing an explicit adjustment parameter to the latter to control for model misspecification and provide diagnostics (Weerasinghe et al., 7 Apr 2025).

3. Choice and Role of Summary Statistics

Selection of informative, low-dimensional summary statistics is a central and often limiting aspect of ABC. Inefficient summaries inflate variance, while non-sufficient summaries bias inference. Several strategies have evolved:

• Handcrafted summaries: Chosen via domain knowledge to capture features informative of the parameters.
• Regression and model-based adjustments: Summary–parameter relationships are regressed (possibly using neural networks or non-linear fits) to further exploit the information content of summaries or to correct for low acceptance rates.
• Nonparametric discrepancy measures: Parzen kernel embeddings (Zuluaga et al., 2015), energy statistics (Nguyen et al., 2019), or sliced-Wasserstein distances (Nadjahi et al., 2019) allow bypassing explicit summary design, leveraging comparisons of full empirical distributions.
• Predictive sufficiency: For ABC predictive inference, specific attention must be paid to ensure the selected statistic is “predictive sufficient,” i.e., it contains all relevant information for forecasting future or missing observations, not just parameter identification (Järvenpää et al., 2022).

Active learning approaches now permit human-in-the-loop summary statistic selection by sequentially querying a domain expert for binary feedback, balancing dimensionality reduction and retention of relevant information (Bharti et al., 2022).

4. Theoretical Properties and Convergence

ABC’s theoretical infrastructure includes analyses of the bias–variance trade-off, convergence rates, and asymptotic properties:

• Bias: Under mild regularity, the bias of expected functionals computed via ABC is $O(\epsilon^2)$ as the tolerance decreases (Barber et al., 2013).
• Variance: Accept/reject ABC sampling variance increases as the tolerance shrinks; for a summary of dimension $q$, the computational cost scales as $\mathrm{cost} \sim n\,\epsilon^{-q}$.
• Optimal tolerance: The root mean squared error is minimized by taking $\epsilon \propto n^{-1/4}$, leading to RMSE decay as $\mathrm{cost}^{-2/(q+4)}$ (Barber et al., 2013).
• Nonparametric viewpoint: ABC can be framed as a $k$–nearest neighbor conditional density estimator, with rigorous results on pointwise and integrated mean squared error as well as explicit asymptotic expansions for bias and variance (Biau et al., 2012).
• Posterior consistency: Under Bayesian consistency and suitable summaries, ABC posteriors (and ABC-based predictives) merge with exact posteriors/predictives as sample size increases (Frazier et al., 2017).
• Kernel and energy metrics: Consistency theorems hold for kernel-ABC-like procedures and for importance-sampling ABC using energy statistics, including convergence of the pseudo-posterior to an indicator-truncated prior (Nguyen et al., 2019).

5. Model Misspecification and Robustification

A critical vulnerability of standard ABC is poor behavior under model misspecification, especially when regression adjustments amplify mismatches in the summary distribution. The robust-ABC methodology partitions the summaries into blocks: well-matched summaries ($\psi(y)$) guide initial localization of parameters, while misspecified summaries ($\phi(y)$) are incorporated with an additive adjustment parameter $\Gamma$. The R-ABC posterior is

$$\pi(\theta, \Gamma | \eta(y)) \propto \pi(\theta)\,\pi(\Gamma) \int p(z | \theta) [d\{\psi(z), \psi(y)\} \leq \epsilon_1][d\{\phi(z) + \Gamma, \phi(y)\} \leq \epsilon_2] dz$$

(Weerasinghe et al., 7 Apr 2025). Posterior concentration of $\Gamma$ away from zero reveals summary components not reproducible by the model, providing automatic diagnostics. Simulation studies (e.g., under a misspecified $g$–and–$k$ model or $\alpha$–stable stochastic volatility) demonstrate that R-ABC achieves nearly unbiased estimation and proper uncertainty quantification, contrary to standard regression-adjusted ABC or synthetic likelihood approaches.

6. Computational Enhancements and Software Platforms

The computational burden of ABC, driven by the exponential growth in required simulations with summary dimension and shrinking $\epsilon$, motivates algorithmic innovations:

• Ensemble Kalman Inversion (IEnKI)–ABC: Implements a sequence of Kalman-like ensemble updates in summary space, yielding variance-stabilized ABC likelihood estimates. In Gaussian settings, recursive formulas for the marginal likelihood (e.g., Eq. (3): $\hat{Z}_T^\text{d}$) provide low-variance, consistent estimators. Application to the stochastic Lotka–Volterra model demonstrates superior effective sample size and stable variance at lower tolerances compared to particle filter or synthetic likelihood ABC (Everitt, 26 Jul 2024).
• Emulation: Local regression and Gaussian process surrogates accelerate ABC by substituting the full model with a fitted emulator, especially effective within sequential ABC frameworks (Jabot et al., 2014).
• Parallelization and efficient SMC/Pseudo-marginal MCMC: State-of-the-art software platforms (e.g., EasyABC, ELFI, sbi) implement parallel simulation scheduling, population Monte Carlo algorithms with adaptive tolerances, and advanced sampling strategies for practitioners (Pesonen et al., 2021).

7. Applications and Impact

ABC has become a central tool for inference in models characterized solely by simulation, including:

• Population genetics: Demographic inference, model selection among evolutionary scenarios, and estimation of ancestral parameters using summary statistics such as Tajima’s $D$ or heterozygosity. The abc R package exemplifies this use case, supporting local linear and neural network adjustments, cross-validation, and model selection diagnostics (Csilléry et al., 2011).
• Ecology and public health: Parameter estimation and model comparison in ecological community dynamics, infectious disease transmission, and epidemiological predictions.
• Financial modeling: Predicting latent volatility and calibrating stochastic volatility models where the likelihood is intractable, employing ABC methods based on auxiliary models and emulators (Martin et al., 2014, Frazier et al., 2017).
• Engineering and insurance: Model calibration for complex loss models in insurance using aggregated data, exploratory model selection, and parameter inference, with implementations leveraging Wasserstein distance metrics and SMC sampling (Goffard et al., 2020).
• Forecasting: Generating probabilistic forecasts via ABC using ABC–based predictive densities that, under regularity, approach the performance of exact Bayesian predictives (Frazier et al., 2017, Järvenpää et al., 2022).

Likelihood-free inference via ABC continues to expand in scope, accommodating advances in kernel embeddings, classifier-based discrepancy measures, active expert-in-the-loop statistic selection, and robustification for uncertainty quantification under model misspecification. The continuing integration of ABC with machine learning surrogates and scalable parallel implementations is anticipated to further broaden application domains (Pesonen et al., 2021).