Sequential Neural Likelihood (SNL)

• Sequential Neural Likelihood (SNL) is a simulation-based Bayesian inference framework that employs neural conditional density estimation to approximate intractable likelihoods.
• It iteratively refines a surrogate likelihood model and adapts proposal distributions using neural flows such as Masked Autoregressive Flows or RealNVP to target complex posteriors.
• SNL achieves high sample efficiency and robust posterior estimation, forming a key component of likelihood-free inference pipelines in fields like astrophysics and econometrics.

Sequential Neural Likelihood (SNL) is a simulation-based Bayesian inference framework that enables parameter estimation in scientific models with intractable or computationally prohibitive likelihood functions. SNL operates by iteratively training a neural conditional density estimator—typically a powerful normalizing flow—to approximate the intractable likelihood, then using the learned surrogate to focus simulations and generate approximate posterior samples via standard MCMC. By sequentially refining both the likelihood model and the proposal distribution, SNL achieves high sample efficiency, robustness, and extensibility to high-dimensional and complex data, and forms the basis for modern likelihood-free inference pipelines in astrophysics, econometrics, and the natural sciences (Papamakarios et al., 2018, Durkan et al., 2018, Kelly et al., 2023, Vílchez et al., 2024, Vílchez et al., 17 Sep 2025, Bastide et al., 11 Jul 2025).

1. Statistical Foundations and Problem Formulation

SNL targets Bayesian inference in contexts where the data-generating process can be simulated but the likelihood $p(x\,|\,\theta)$ cannot be evaluated, i.e., in so-called simulator or likelihood-free models:

• Simulator: $x \sim p(x\,|\,\theta)$ with $\theta \sim p(\theta)$, $x$ observed.
• Objective: Compute or sample from posterior $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$.

The key observation is that while $p(x\,|\,\theta)$ may be intractable, conditional density estimation techniques enable flexible learning of a surrogate likelihood $q_\phi(x\,|\,\theta)$ via simulated $(\theta, x)$ pairs. The posterior is then approximated as:

$$q_\text{post}(\theta\,|\,x_0) \propto q_\phi(x_0\,|\,\theta)\,p(\theta).$$

Sampling is performed with MCMC targeting this density (Papamakarios et al., 2018, Durkan et al., 2018, Bastide et al., 11 Jul 2025). SNL thus recasts likelihood-free Bayesian inference as conditional density modeling, circumventing the need for Approximate Bayesian Computation (ABC) or synthetic likelihoods, and exploits advances in neural density estimation.

2. Core Algorithmic Workflow

The SNL procedure alternates rounds of simulation, likelihood surrogate refinement, and proposal adaptation:

1. Initialization: Proposal $q_0(\theta) = p(\theta)$ (the prior). Dataset $x \sim p(x\,|\,\theta)$0.
2. Simulation and Data Aggregation: At round $x \sim p(x\,|\,\theta)$1, sample $x \sim p(x\,|\,\theta)$2 parameters $x \sim p(x\,|\,\theta)$3, simulate $x \sim p(x\,|\,\theta)$4, and aggregate $x \sim p(x\,|\,\theta)$5 into $x \sim p(x\,|\,\theta)$6.
3. Likelihood Model Training: Optimize a neural conditional density estimator (e.g., Masked Autoregressive Flow or RealNVP) $x \sim p(x\,|\,\theta)$7 by maximizing

$x \sim p(x\,|\,\theta)$8

(Papamakarios et al., 2018, Vílchez et al., 2024).

1. Proposal Adaptation: Posterior is approximated by $x \sim p(x\,|\,\theta)$9; use MCMC (e.g. slice sampling or parallel-tempered MCMC) to target and sample from $\theta \sim p(\theta)$0 (Durkan et al., 2018, Bastide et al., 11 Jul 2025, Vílchez et al., 2024).
2. Repeat: Iterate until posterior estimates stabilize or a preset number of rounds is reached.

Pseudocode reflecting these elements appears in (Papamakarios et al., 2018, Durkan et al., 2018, Bastide et al., 11 Jul 2025).

3. Neural Likelihood Model Architecture and Training

SNL employs conditional normalizing flows—typified by Masked Autoregressive Flows (MAF) or RealNVP—to represent $\theta \sim p(\theta)$1:

• Each $\theta \sim p(\theta)$2 is mapped bijectively to a latent $\theta \sim p(\theta)$3 via $\theta \sim p(\theta)$4, modeled as a stack of conditional affine or autoregressive transformations parameterized by $\theta \sim p(\theta)$5 (Papamakarios et al., 2018, Vílchez et al., 2024).
• The model density is defined by the change of variables:

$\theta \sim p(\theta)$6

(Bastide et al., 11 Jul 2025, Vílchez et al., 17 Sep 2025).

• Training employs maximum log-likelihood over the growing dataset of simulated pairs, which is equivalent to minimizing the KL divergence between $\theta \sim p(\theta)$7 and $\theta \sim p(\theta)$8 on the support of the proposals (Papamakarios et al., 2018, Durkan et al., 2018).

For high-dimensional data, preprocessing such as PCA or autoencoder-based compression is critical. PCA to $\theta \sim p(\theta)$9 ($x$0–128) is used to facilitate tractable density estimation while retaining essential signal variance; autoencoders can handle nonlinear compression at the expense of potential information loss (Vílchez et al., 2024, Vílchez et al., 17 Sep 2025).

Key hyperparameters include number of SNL rounds ($x$1–100), simulations per round ($x$2–10^4), flow depth (e.g., 5–15 MADE blocks or coupling layers), and optimizer settings (Adam variants with learning rates $x$3–$x$4) (Bastide et al., 11 Jul 2025, Vílchez et al., 2024, Papamakarios et al., 2018).

4. Statistical Properties, Diagnostics, and Robustness

Sample Efficiency: SNL focuses simulation in regions of high posterior mass via sequential adaptation. Empirically, SNL achieves calibration and coverage comparable to, or exceeding, SNPE and ABC at $x$5 of the simulator calls required by standard MCMC in problems such as LISA MBHB inference (Vílchez et al., 2024, Vílchez et al., 17 Sep 2025).

Diagnostics: Statistical calibration is assessed via simulation-based calibration (SBC), convergence is monitored by the median distance between simulated and observed data, and goodness-of-fit is measured by Maximum Mean Discrepancy (MMD) between samples from $x$6 and $x$7 (Papamakarios et al., 2018). Effective sample size ($x$8) and Gelman–Rubin statistics are used for MCMC-based posterior draws (Vílchez et al., 17 Sep 2025).

Robustness: SNL is sensitive to model misspecification: under incompatibility, it tends toward overconfident, miscentered posteriors. The RSNL extension introduces an adjustment parameter $x$9 per summary statistic, with a shrinkage prior, allowing robust inference under misspecification while recovering SNL when the model is compatible (Kelly et al., 2023). Empirically, RSNL yields credible intervals with nominal or conservative coverage.

5. Extensions: Marginal Likelihood and Model Comparison

While SNL originally focused on posterior approximation, recent developments leverage the surrogate likelihood and posterior samples for marginal likelihood (evidence) estimation (Bastide et al., 11 Jul 2025):

• SIS-SNLE: Sequential importance sampling estimator, aggregating ratio estimates across rounds.
• IS-SNLE: Importance sampling with a flow-fit proposal $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$0 over final posterior samples; achieves lowest bias and variance in moderate dimensions.
• HM-SNLE: Harmonic mean estimator with an instrumental density $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$1, stabilized by temperature; sensitive to heavy tails.

These approaches enable Bayesian model comparison in likelihood-free inference, broadening the utility of SNL workflows to evidence-based model selection. IS-SNLE typically outperforms alternatives with lower bias and variance in benchmarks (Bastide et al., 11 Jul 2025).

6. Practical Applications and Performance

SNL has been extensively validated in astrophysical inference (e.g., LISA MBHB parameter estimation), biological simulation models, and mechanistic econometric models (Vílchez et al., 2024, Vílchez et al., 17 Sep 2025). In LISA MBHB analysis:

• SNL with PCA-compressed likelihood flows recovers posteriors comparable to MCMC at $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$2–$$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$3 of simulator calls.
• Forward simulation, whitening, and dimensionality reduction are required for tractability at $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$4.
• Posterior credible intervals produced by SNL are quantitatively accurate for intrinsic and extrinsic parameters, though autoencoder-based compression may broaden or bias posteriors if waveform features are lost (Vílchez et al., 2024, Vílchez et al., 17 Sep 2025).

The flexible modular design enables easy extension to richer waveform models, complex noise, and multi-source scenarios.

7. Comparisons, Limitations, and Recent Alternatives

Comparisons with Alternative SBI Methods:

• SNPE: Directly learns the posterior $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$5; SNL's surrogate likelihood approach is more stable with less correction/bias from proposal adaptation and better sample efficiency in tested regimes (Papamakarios et al., 2018, Durkan et al., 2018).
• SNPLA: Jointly trains both $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$6 and $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$7 via reverse KL, eliminating the need for MCMC, and enables rapid posterior sampling but requires joint optimization and can be more sensitive to training instabilities (Wiqvist et al., 2021).
• ABC/Synthetic Likelihood: Generally requires more simulations and yields less accurate posteriors; SNL demonstrates robust inference with orders-of-magnitude lower simulation cost (Papamakarios et al., 2018, Durkan et al., 2018).

Limitations:

• MCMC sampling at each round adds overhead, especially in higher dimensions.
• High input dimension $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$8 in $$p(\theta\,|\,x_0) \propto p(x_0\,|\,\theta)\,p(\theta)$$9 can strain conditional density estimation, requiring ad hoc compression.
• SNL may be overconfident in the presence of simulator misspecification without explicit corrections (e.g., RSNL) (Kelly et al., 2023).

SNL thus represents a principled, sample-efficient, and extensible paradigm for simulation-based inference, and remains at the core of modern likelihood-free inference pipelines across scientific domains (Papamakarios et al., 2018, Bastide et al., 11 Jul 2025, Vílchez et al., 2024).