Neural Likelihood Approximation

• Neural likelihood approximation is a family of methods that use neural surrogates to replace intractable likelihoods in Bayesian inference.
• It employs techniques like conditional density estimation, likelihood ratio estimation, and score matching to build accurate surrogate models.
• These methods enable efficient simulation-based inference across diverse applications such as ecology, high-energy physics, and Bayesian inverse problems.

Neural likelihood approximation denotes a family of methods that replace an intractable, expensive, or analytically inconvenient likelihood with a learned surrogate, most often a neural conditional density estimator, an implicit likelihood-ratio model, or an un-normalized potential. In the canonical simulator-based setting, one can generate samples $x \sim p(x \mid \theta)$ but cannot evaluate $p(x \mid \theta)$ pointwise, and the surrogate is used to construct an approximate posterior such as $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$. Related literatures use the same phrase more broadly for Gaussian surrogates of known non-Gaussian likelihoods and for convex potential-based formulations in Bayesian inverse problems, so the term spans several adjacent but not identical research programs (Durkan et al., 2018, Lueckmann et al., 2018, Bui, 2024, Schneider et al., 7 Jul 2026).

1. Conceptual scope and formal setting

In simulation-based inference, the standard setup consists of a stochastic simulator

$p(x \mid \theta),$

a prior $p(\theta)$, and one observed dataset $x_0$. The inferential goal is Bayesian recovery of

$p(\theta \mid x_0),$

despite the fact that $p(x_0 \mid \theta)$ is unavailable for direct evaluation. Within this setting, a central distinction is whether one learns the inverse conditional map $x \to \theta$, i.e. an approximate posterior $q_\phi(\theta \mid x_0)$, or the forward conditional map $p(x \mid \theta)$0, i.e. a surrogate likelihood $p(x \mid \theta)$1 or emulator (Durkan et al., 2018).

Likelihood-learning approaches are often described as neural synthetic likelihood or neural likelihood estimation. Their defining step is to fit a tractable surrogate $p(x \mid \theta)$2 to simulated parameter–data pairs and then combine it with the prior. Posterior-learning approaches instead fit $p(x \mid \theta)$3 directly. This distinction matters because the conditional target $p(x \mid \theta)$4 is unchanged by the proposal distribution over $p(x \mid \theta)$5, whereas direct posterior learning is more sensitive to proposal mismatch in sequential settings (Durkan et al., 2018).

The phrase “likelihood approximation” is not uniform across subfields. In simulation-based inference it usually refers to learning an unknown or intractable likelihood from simulator output. In Bayesian supervised learning, however, it can mean replacing a known non-Gaussian likelihood such as softmax or logistic by a Gaussian pseudo-likelihood in transformed logit coordinates, so that Gaussian-regression machinery becomes available (Bui, 2024). A plausible implication is that the field is better understood as a cluster of methods organized by what is being approximated—unknown simulator likelihoods, likelihood ratios, or analytically known but computationally awkward observation models—rather than by a single canonical algorithmic template.

2. Direct neural likelihood models

A foundational line of work treats the surrogate explicitly as a conditional density model $p(x \mid \theta)$6. Emulator-network methods formulate likelihood-free inference as learning a tractable probabilistic emulator of the simulator likelihood, then using that emulator as a synthetic likelihood inside standard Bayesian inference. These methods distinguish local emulators, trained for a specific observation $p(x \mid \theta)$7, from global emulators, intended to approximate $p(x \mid \theta)$8 over a broad parameter and data domain and therefore support amortized reuse across observations (Lueckmann et al., 2018).

Sequential Neural Likelihood (SNL) is a particularly influential formulation. It learns a surrogate conditional likelihood $p(x \mid \theta)$9, combines it with the prior, and samples from

$$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$0

In a controlled comparison using the same Bayesian mixture density network for all methods—a single hidden layer with 50 tanh units and a mixture of 5 full-covariance Gaussian components, trained by stochastic variational inference with Adam for 1000 epochs per round over 10 rounds—likelihood-learning methods SNL and ASNL outperformed the posterior-learning baseline SNPE-B on the Gaussian toy problem and on Lotka–Volterra, while all methods were similar on M/G/1. The same study concluded that “learning the likelihood seems preferable to learning the posterior, all other model choices equal,” and also found that ASNL’s uncertainty-driven active selection produced no meaningful gains over SNL’s simpler heuristic, while incurring much greater wall-clock cost (Durkan et al., 2018).

Other direct likelihood models impose more structure. Score-matched neural exponential families define

$$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$1

where $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$2 functions as learned sufficient statistics and $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$3 as learned natural parameters. Because $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$4 is intractable, training uses conditional score matching or sliced score matching rather than maximum likelihood. The resulting surrogate is amortized across observations and can be used both as a summary-statistic generator for ABC and as an unnormalized likelihood inside ExchangeMCMC (Pacchiardi et al., 2020).

Neural likelihood models have also been specialized to discrete partially observed stochastic processes. For integer-valued time series generated by partially observed continuous-time Markov chains, an autoregressive surrogate

$$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$5

is parameterized by a causal CNN with discretized mixture-of-logistics outputs and trained entirely from unconditional simulations. In that formulation, Sequential Neural Likelihood is used to focus simulations in posterior-relevant regions, while posterior sampling is performed with NUTS on the surrogate posterior (O'Loughlin et al., 2023).

A further synthesis is Sequential Neural Posterior and Likelihood Approximation (SNPLA), which jointly learns a likelihood flow $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$6 and a posterior flow $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$7. The likelihood is trained by conditional maximum likelihood on simulations from a proposal mixture of prior and current posterior approximation, while the posterior is trained by reverse KL against the target induced by the learned likelihood and prior. This design avoids both MCMC and SNPE-style proposal correction, and in the reported experiments posterior draws were generated about $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$8 times faster than in MCMC-based SNL (Wiqvist et al., 2021).

3. Ratio estimation, likelihood surfaces, and pseudo-likelihoods

A second major branch does not learn a normalized $$\hat p(\theta \mid x_0) \propto q_\phi(x_0 \mid \theta)\,p(\theta)$$9 at all. Instead it learns a likelihood ratio, or a quantity proportional to the likelihood, through classification. With balanced classes, a classifier output $p(x \mid \theta),$0 trained to distinguish samples from $p(x \mid \theta),$1 and $p(x \mid \theta),$2 yields the familiar odds identity

$p(x \mid \theta),$3

This principle underlies a large family of neural ratio estimators for hypothesis testing, reweighting, unfolding, and anomaly detection (Acosta et al., 26 Mar 2025, Rizvi et al., 2023).

The ratio-estimation literature has emphasized that asymptotically valid losses can behave very differently in finite neural-network settings. A systematic comparison of binary cross-entropy, mean squared error, maximum-likelihood classifier, and square-root losses found that output parameterization is critical; among standard defaults, “MLC with exponential output is typically the best default,” and generalized $p(x \mid \theta),$4-MSE or $p(x \mid \theta),$5-SQR losses can outperform the defaults in some settings (Rizvi et al., 2023). This suggests that neural likelihood approximation through ratios is governed as much by optimization geometry and output parameterization as by the underlying ratio identity.

Classifier-based likelihood approximation can also target a full likelihood surface up to proportionality. For spatial processes on regular grids, one can train a probabilistic classifier $p(x \mid \theta),$6 to distinguish dependent pairs drawn from $p(x \mid \theta),$7 from independence-coupled pairs drawn from $p(x \mid \theta),$8. Under balanced classes,

$p(x \mid \theta),$9

This construction yields “neural likelihood surfaces” that support grid-based maximum likelihood estimation and likelihood-ratio confidence regions. In the reported Gaussian-process and Brown–Resnick examples, post-hoc Platt scaling was central because the odds transform amplifies calibration error in $p(\theta)$0 (Walchessen et al., 2023).

Pseudo-Likelihood Inference (PLI) occupies an intermediate position between ABC and neural posterior approximation. It defines

$p(\theta)$1

with $p(\theta)$2 instantiated by integral probability metrics such as MMD or squared $p(\theta)$3-Wasserstein distance. A neural posterior model $p(\theta)$4 is then trained by weighted maximum likelihood under a KL trust-region update. PLI is explicitly not a standard neural likelihood estimator of the $p(\theta)$5 type; it is an ABC-derived pseudo-likelihood framework with neural posterior fitting (Gruner et al., 2023).

In iterative ratio-based pipelines, stability becomes part of the inference problem itself. For OmniFold-style unbinned unfolding, even small stochastic fluctuations in neural ratio estimates propagate through repeated reweighting. Empirical work on stabilization found that ensembling at intermediate steps, before weights are pushed downstream, improves final bias and variance relative to naïvely propagating a single noisy model, while parallel ensembling gave the strongest overall variance reduction in the reported benchmark (Acosta et al., 26 Mar 2025).

4. Statistical theory and identifiability

For many years neural likelihood methods were primarily justified empirically. A recent theoretical analysis places neural likelihood estimation on a footing comparable to ABC and Bayesian synthetic likelihood. In the summary-statistic setting, if $p(\theta)$6 denotes the true summary likelihood and $p(\theta)$7 its neural approximation, the neural-likelihood posterior is

$p(\theta)$8

Under regularity, compatibility, and approximation assumptions, the resulting posterior concentrates at rate

$p(\theta)$9

and if the exact summary-based posterior is locally asymptotically normal, then the neural-likelihood posterior is asymptotically Gaussian as well. The same work argues that NPE and NLE can be “just as accurate” as ABC and BSL in asymptotic statistical terms while often requiring fewer simulator calls, although NLE retains the operational burden of posterior sampling after likelihood learning (Frazier et al., 2024).

A different theoretical development reformulates neural likelihood approximation for Bayesian inverse problems in terms of un-normalized potentials rather than normalized conditional densities. If $x_0$0 is a surrogate negative log-likelihood and

$x_0$1

then the induced posterior satisfies

$x_0$2

The key objective becomes

$x_0$3

This functional is strictly convex in the potential $x_0$4, modulo the equivalence that adding a $x_0$5-only term does not change the posterior, and empirical approximate minimizers converge in probability to the true potential as sample size grows (Schneider et al., 7 Jul 2026).

These results sharpen two themes that recur across the literature. First, statistical validity depends on local approximation quality near the observed data or summaries, not only on global function-approximation capacity. Second, identifiability is often posterior-relative rather than likelihood-global: many surrogates are only required to be accurate on proposal-supported or observation-relevant regions. A plausible implication is that the strongest theory currently aligns most naturally with one-shot or potential-based formulations, while sequential and highly amortized neural likelihood methods remain more empirically grounded.

5. Application domains

Neural likelihood approximation has been developed in response to concrete scientific bottlenecks. In ecology, epidemiology, and systems biology, it has been used for Lotka–Volterra simulators, SIR and SEIAR epidemic models, predator–prey dynamics, Hodgkin–Huxley neuron models, and Lorenz96 time series. In these settings the central attraction is usually the same: inference can be performed from unconditional or targeted simulations without requiring analytic likelihood evaluation, and the learned surrogate can be reused for posterior sampling or posterior predictive checks (Durkan et al., 2018, Pacchiardi et al., 2020, O'Loughlin et al., 2023, Lueckmann et al., 2018).

In spatial statistics, classifier-based neural likelihood surfaces have been proposed for Gaussian processes with computationally intensive likelihoods and Brown–Resnick max-stable processes with intractable likelihoods. The method is formulated for spatial processes on a regular grid, uses a CNN to process the field, and yields a surrogate likelihood surface that supports maximum likelihood estimation and approximate confidence regions through likelihood-ratio inversion (Walchessen et al., 2023).

In Bayesian inverse problems, neural surrogates have been trained directly on joint parameter–observation data and then substituted for expensive forward PDE solves inside posterior samplers. In the reported nonlinear PDE imaging experiment, a single likelihood evaluation by the learned surrogate took $x_0$6 s on CPU and $x_0$7 s on GPU, compared with $x_0$8 s for the PDE solve on CPU; for 1000 evaluations the corresponding times were $x_0$9 s, $p(\theta \mid x_0),$0 s, and $p(\theta \mid x_0),$1 s, respectively. Posterior sampling that required $p(\theta \mid x_0),$2 hours with the PDE model was reported to take less than $p(\theta \mid x_0),$3 minutes with the neural surrogate on CPU and less than $p(\theta \mid x_0),$4 minutes on a GPU cluster (Schneider et al., 7 Jul 2026).

High-energy physics supplies two distinct application patterns. One is discriminative ratio estimation for unfolding and reweighting, where stability under retraining and iterative propagation is a first-order concern (Acosta et al., 26 Mar 2025). The other is interpolation and distribution of already specified experimental likelihoods. DNNLikelihood trains a deep network to regress the full log-likelihood as a function of both parameters of interest and nuisance parameters—95 inputs in the case study—and thereby preserves the complete nuisance-parameter dependence of the original analysis without Gaussian approximation, profiling, or marginalization. The best reported models achieved prediction times around $p(\theta \mid x_0),$5 on an Nvidia Tesla V100 GPU (Coccaro et al., 2019).

A broader but terminologically distinct application appears in Bayesian classification and Gaussian-process classification. There, transformed-basis Laplace, variational, and moment matching are used to replace softmax or logistic observation models with Gaussian pseudo-likelihoods over logits. The resulting approximation is especially valuable for GP regression machinery, Bayesian last-layer models, streaming updates, and active learning, where exact inference in the approximate Gaussianized model can outperform approximate inference in the exact non-Gaussian model under tight computational budgets (Bui, 2024).

6. Limitations, controversies, and recurring misconceptions

A recurring misconception is that neural likelihood approximation always means direct simulator-likelihood learning with a neural density estimator. In fact, the literature includes conditional density surrogates, classifier-based likelihood-ratio estimators, joint-versus-product likelihood surfaces, ABC-style pseudo-likelihoods, Gaussianized logit-space surrogates for known classifier likelihoods, and un-normalized potential learning for inverse problems (Bui, 2024, Gruner et al., 2023, Schneider et al., 7 Jul 2026).

Methodological trade-offs are pervasive. Sequential likelihood learning is often described as more robust to proposal adaptation because the target $p(\theta \mid x_0),$6 does not change across rounds, but this does not imply that more elaborate acquisition policies are always preferable. In the matched-architecture comparison of SNL and ASNL, uncertainty-driven active selection did not improve final inference quality over simple posterior-guided simulation, and ASNL often spent simulations near prior boundaries while incurring much greater wall-clock cost (Durkan et al., 2018).

The likelihood-versus-posterior distinction is also not absolute. In low-dimensional summary-statistic settings, learning $p(\theta \mid x_0),$7 can be easier or more reliable than learning $p(\theta \mid x_0),$8. But if observations become very high-dimensional while parameters remain low-dimensional, directly learning $p(\theta \mid x_0),$9 may become easier than modeling $p(x_0 \mid \theta)$0 (Durkan et al., 2018). This suggests that the most appropriate approximation target depends on the asymmetry between observation complexity and parameter complexity, not only on philosophical preference.

Ratio-based methods carry their own limitations. They usually estimate ratios or likelihoods only up to proportionality, not normalized densities, and their downstream performance can be dominated by calibration and run-to-run stability rather than asymptotic risk alone. In iterative pipelines, ensembling and pretraining alter a bias–variance trade-off rather than merely improving robustness “for free” (Acosta et al., 26 Mar 2025, Rizvi et al., 2023).

Several frameworks also retain unresolved normalization issues. PLI introduces a partition function $p(x_0 \mid \theta)$1 in the pseudo-likelihood but states that it is effectively treated as constant in experiments, even though this is not fully faithful to the formal derivation (Gruner et al., 2023). In the convex potential framework, strict convexity holds in function space, not in neural-network parameter space, so global optimization guarantees for neural training remain out of reach (Schneider et al., 7 Jul 2026). In score-matched neural exponential families, direct posterior sampling via ExchangeMCMC is suitable only for moderate-dimensional observations—“up to a few hundred”—because the inner chain over $p(x_0 \mid \theta)$2 becomes expensive as dimension grows (Pacchiardi et al., 2020).

Taken together, these limitations indicate that neural likelihood approximation is less a single method than a design space. Its persistent themes are surrogate construction, calibration, sequential data allocation, and the choice of which probabilistic object to approximate—likelihood, posterior, ratio, or potential—under the computational and statistical constraints of a given scientific problem.