ABC-Pipeline Overview

• ABC-Pipeline is a structured framework for likelihood-free Bayesian inference that leverages simulation-based methods and summary statistics comparisons.
• It consists of sequential stages that include selecting informative summaries, implementing samplers (e.g., rejection, MCMC, SMC), and post-processing adjustments to refine posterior approximations.
• Advanced implementations like Hamiltonian ABC and Adaptive Gaussian Copula ABC address high-dimensional challenges using techniques such as synthetic gradients, adaptive proposals, and copula modeling.

Approximate Bayesian Computation (ABC) pipelines are structured methodologies for likelihood-free Bayesian inference in simulation-based models. ABC involves replacing intractable or expensive likelihood evaluations with posterior approximations based on measuring the similarity between observed and simulated summary statistics. The ABC pipeline consists of three main stages: specification of summaries and discrepancy measures, sampling-based likelihood-free inference (e.g., rejection, MCMC, or SMC samplers), and post-processing or adjustment of the resulting approximations. Recent advances have introduced novel pipeline implementations, each targeting specific computational or statistical challenges within ABC, such as high-dimensional parameter spaces or improved posterior approximation fidelity.

1. Foundational Structure of ABC Inference Pipelines

ABC frameworks operate by positioning the observed data $y_{\text{obs}}$ and its summary $s_{\text{obs}}$ alongside summary statistics $S(\cdot) : y \mapsto s$ drawn from simulated data $y \sim p(y|\theta)$. A kernel $K_\epsilon$ and distance metric $\rho$ quantify the discrepancy between observer and simulated summaries. The canonical ABC posterior approximation is

$$p_\epsilon(\theta | y_{\text{obs}}) \propto \int K_\epsilon(\rho(s, s_{\text{obs}})) p(s|\theta) \pi(\theta) ds.$$

The construction of an ABC pipeline thus entails:

• Selecting informative summary statistics and a suitable distance metric.
• Choosing a kernel and tolerance schedule $\epsilon$.
• Specifying and executing an ABC sampler (rejection, importance, MCMC, SMC).
• Adjusting and interpreting posterior approximations via post-processing such as regression adjustment or copula constructions (Fan et al., 2018).

2. Main Sampling Paradigms and Workflow

ABC samplers are implemented through three principal paradigms:

1. Rejection and Importance Sampling: Samples $\theta$ from a prior (or proposal), simulates $y' \sim p(y|\theta)$, computes $s' = S(y')$, and accepts $\theta$ if $\rho(s', s_{\text{obs}}) \leq \epsilon$. The importance sampler variant assigns weights

   $$\widetilde{w}^{(i)} = K_\epsilon(\rho(s^{(i)}, s_{\text{obs}})) \frac{\pi(\theta^{(i)})}{g(\theta^{(i)})}$$

and normalizes. Rejection ABC is unbiased but rapidly declines in efficiency as $\epsilon \to 0$ or dimension increases.

1. ABC–MCMC: The kernel-ABC Metropolis–Hastings algorithm proposes local moves in $\theta$-space and accepts them according to an acceptance probability that incorporates the ABC kernel, prior, and proposal densities. It avoids catastrophic rejection rates but is prone to poor mixing for inappropriately chosen $\epsilon$ or proposals.
2. ABC–SMC: Constructs a sequence of approximations to the posterior with decreasing $\epsilon_t$, propagating particles through resampling, mutation (often via MCMC moves), and weighting. Particle weights are updated as

   $$W_i^{(t)} \propto \frac{\pi(\theta_i^{(t)})}{\sum_{j=1}^N W_j^{(t-1)} K_{\text{trans}}(\theta_i^{(t)}|\theta_j^{(t-1)})}$$

and tolerances $\epsilon_t$ are adaptively determined to preserve effective sample size (ESS) (Fan et al., 2018).

Practical recommendations emphasize starting with ABC–SMC to localize high posterior regions, using local MCMC moves, monitoring ESS, and iteratively refining $\epsilon$.

3. Hamiltonian ABC Pipeline

Hamiltonian ABC (HABC) introduces stochastic-gradient Hamiltonian dynamics (SGHD) into the ABC pipeline to enable scalable inference in high-dimensional spaces, drastically improving efficiency over standard ABC rejection or MCMC workflows (Meeds et al., 2015). The HABC workflow consists of:

• Initialization: Choose initial $\theta_0$, draw momentum $p_0 \sim N(0,M)$, sample $S$ random seeds $\omega_1, ..., \omega_S$, set hyperparameters for dynamics (step-size $\eta$, mass matrix $M$, friction or thermostat parameters, SPSA perturbation scale $c$, repetitions $R$, and seed refresh probability $\gamma$).
• Simulation and Synthetic Likelihood: For each seed $s$, deterministically simulate $x_s = f(\theta_t, \omega_s)$, compute statistics, and fit a synthetic likelihood (sample mean $\mu_\theta$, covariance $\Sigma_\theta$), modeling

  $$p_\epsilon(y|\theta) = N(y | \mu_\theta, \Sigma_\theta + \epsilon^2 I)$$

• Potential and Gradient Estimation: The Hamiltonian potential is

  $$U(\theta) = -\log p_\epsilon(y|\theta) - \log p(\theta)$$

Gradients $\nabla_\theta U$ are computed via finite differences or Simultaneous Perturbation Stochastic Approximation (SPSA), evaluated with common random numbers for variance control.

• Hamiltonian Updates: Leapfrog or other SGHD updates (e.g., SGHMC, SGLD, SGNHT) are performed on $(\theta, p)$, often bypassing full-data Metropolis corrections by appropriate noise/friction injection. Seed refreshes ("sticky seeds") employ a pseudo-marginal MH mechanism: for each $s$, propose $\omega'_s$ with probability $\gamma$ and accept using the synthetic likelihood.

A key innovation is retaining seeds $\{\omega_s\}$ in the chain state, enabling common random numbers, persistent pseudo-randomness, and reduced gradient estimator variance.

Computationally, HABC requires $O(S(2R+1))$ simulator calls per step, independent of the parameter dimension $D$, contrasting sharply with traditional ABC acceptance complexity (Meeds et al., 2015).

4. Adaptive Gaussian Copula ABC Pipeline

Adaptive Gaussian Copula ABC (AGC-ABC) combines regression ABC, sequential proposal adaptation, and Gaussian copula modeling in a two-stage procedure (Chen et al., 2019). The pipeline proceeds as follows:

• Coarse-Grained Stage: A small simulation budget ($\lambda N$) is used to sample from the prior, simulate observations, fit a regression model $\theta|s = g(s) + \xi$, and generate regression-adjusted samples $\theta'^{(i)} = g(s^o) + [\theta^{(i)}-g(s^{(i)})]$. The top $m$ samples are retained, and a Gaussian auxiliary proposal distribution $$q_{\text{coarse}}(\theta|s^o) = \mathcal{N}(g(s^o), V)$$ with inflated covariance is constructed.
• Fine-Grained Stage: The remaining budget is used to sample from $q_{\text{coarse}}$, simulate, and perform regression adjustment on top $n$ samples. Marginals are estimated by KDE, correlations by transforming to latent Gaussian variables ($z_k^{(i)}$), and a semi-parametric Gaussian copula $p_{\text{aux}}(\theta|s^o)$ is constructed.
• Posterior Recovery: Importance reweighting corrects the proposal,

  $$p(\theta|s^o) \propto \frac{\pi(\theta)}{q_{\text{coarse}}(\theta|s^o)} p_{\text{aux}}(\theta|s^o),$$

yielding a consistent approximation as $N \to \infty$.

Key theoretical assumptions include the additive noise regression model and a homogeneity condition that residuals are approximately invariant within $\epsilon$-balls around $s^{o}$. AGC-ABC has demonstrated competitive or superior Jensen–Shannon divergence to ground-truth posteriors compared to comparable ABC methods—particularly in the presence of residual heterogeneity or small simulation budgets (Chen et al., 2019).

5. Mathematical Components and Pseudocode Representation

ABC pipelines formalize their procedures in labeled pseudocode and LaTeX formulas, ensuring replication and rigorous analysis.

Key Equations

Component	Equation / Formula
ABC posterior	$$p_\epsilon(\theta|y_{\text{obs}}) \propto \int K_\epsilon(\rho(s, s_{\text{obs}})) p(s|\theta) \pi(\theta) ds$$
Synthetic-likelihood ABC	$\mu_\theta = \frac{1}{S} \sum_{s=1}^S x^{(s)}$; $$\Sigma_\theta = \frac{1}{S-1}\sum_{s}(x^{(s)}-\mu_\theta)(x^{(s)}-\mu_\theta)^T$$; $$p_\epsilon(y|\theta) = N(y|\mu_\theta, \Sigma_\theta+\epsilon^2I)$$
Hamiltonian potential	$$U(\theta) = -\log p_\epsilon(y|\theta) - \log p(\theta)$$
SPSA gradient	$$ĝ^{(r)} = \frac{U(\theta+c\Delta^{(r)})-U(\theta-c\Delta^{(r)})}{2c} (\Delta^{(r)})^{-1}$$; average over $R$ reps
Copula density	$$p_{\mathrm{aux}}(\theta|s^o) = c_{GC}(u_1,\ldots,u_K; \Lambda) \prod_{k=1}^K f_k(\theta_k)$$

Pseudocode Archetypes

Both (Fan et al., 2018, Meeds et al., 2015), and (Chen et al., 2019) present explicit stepwise pseudocode covering:

• Initialization of sampling budgets, seeds/proposals
• Iterative parameter sampling, simulation, statistics calculation, and acceptance/reweighting
• Sequential or adaptive proposal refinement

6. Empirical Performance and Practical Considerations

Empirical studies illustrate varying performance orientation. HABC achieves dimension-independent simulator call complexity and effective posterior exploration by leveraging synthetic gradients, "sticky seeds," and Hamiltonian trajectories, matching or exceeding standard ABC-MCMC/SMC in high dimensions (in particular, $D=1568$ logistic regression with only $10$ SPSA reps per step) (Meeds et al., 2015). AGC-ABC outperforms neural-based and classical ABC competitors on problems exhibiting residual heterogeneity (e.g., M/G/1 queue, Lotka–Volterra); its utility is modulated by the validity of regression adjustment and sufficiency of homogeneity assumptions (Chen et al., 2019). Classical rejection and MCMC ABC remain effective only in low dimensions or for loose tolerances, while SMC ABC demonstrates flexibility through sequential adaptation but incurs increased implementation complexity (Fan et al., 2018).

Summary statistic selection, simulation budget allocation, parameter normalization, and tuning (e.g., kernel bandwidth, proposal covariance inflation) are all essential implementation considerations, detailed with suggested values in AGC-ABC and HABC documentation.

7. Comparative Assessment and Pipeline Selection

Method	Key Features	Computational Cost	High-dimensional Handling	Empirical Performance
Rejection	Direct, independent, unbiased	$O(S)$ per accepted sample	Poor	Inefficient for low $\epsilon$, $D$
MCMC-ABC	Local proposals, pseudo-marginal	$O(S)$ per move	Moderate	Sensitive to proposal/tuning
SMC-ABC	Population-based, adaptive	$O(\ell N S)$	Moderate (adaptive)	Robust, flexible, complex
HABC	SGHD, sticky seeds, synthetic grad.	$O(S(2R+1))$ per HMC step	Excellent	High fidelity, scalable
AGC-ABC	Regression + copula + adaptivity	$N$ simulator calls in total	Good (with copula)	JSD-minimal in heterogeneity

Choice of ABC pipeline is strongly problem-dependent. HABC is preferable for high-dimensional, simulation-expensive inference with smooth statistics. AGC-ABC is advantageous in the presence of strong residual heterogeneity and limited computational budget, provided regression assumptions are met. Classical SMC-ABC is indicated where complex or multimodal posteriors necessitate sequential adaptation.

• Hamiltonian ABC (Meeds et al., 2015)
• ABC Samplers (Fan et al., 2018)
• Adaptive Gaussian Copula ABC (Chen et al., 2019)