Likelihood Annealing (LIKA)

Updated 16 April 2026

Likelihood Annealing (LIKA) is a family of Bayesian and likelihood-based methods that gradually integrates likelihood information via adaptive annealing schedules to enable robust sampling in complex models.
These techniques leverage thermodynamic concepts and controlled simulation or emulation to balance computational cost with precise inference across various challenging scenarios.
LIKA methods are applied in Bayesian inference, latent-variable models, deep regression, and expensive likelihood emulation, demonstrating faster convergence and improved uncertainty quantification.

Likelihood Annealing (LIKA) encompasses a family of Bayesian and likelihood-based computational methods in which inference is performed by gradually introducing or modulating likelihood information through an annealing or tempering schedule. This class of algorithms exploits analogies with thermodynamic and statistical-physics concepts—such as temperature, entropy production, and Gibbs distributions—to achieve robust sampling or optimization when the likelihood is intractable, computationally expensive, or yields poorly calibrated uncertainty. LIKA methods are deployed across Bayesian simulation-based inference, sequential Monte Carlo, scalable Bayesian computation for latent-variable models, uncertainty quantification for deep regression, and more. They include both purely classical and quantum–inspired formulations, and their variants often circumvent the need for closed-form likelihood calculations by leveraging forward simulation and adaptive annealing strategies.

1. Thermodynamic Foundation and Simulation-Based LIKA

The prototypical framework of Likelihood Annealing is embodied by ensemble-based simulated-annealing approaches for Bayesian inference, notably the Simulated Annealing Bayesian Computation (SABC) scheme of Albert (2015) (Albert, 2015). Here, the objective is to sample from the posterior

$f_{\mathrm{post}}(\theta \mid y) \propto L(y \mid \theta)\, f_{\mathrm{pri}}(\theta)$

without explicit computation of the likelihood $L(y \mid \theta)$ . Instead, model outputs $x$ are generated by simulating $x \sim L(\cdot \mid \theta)$ , and a distance or discrepancy function $u(x) = \rho(x, y)$ is used as an “energy” proxy $E(\theta)$ . An ensemble of $N$ particles inhabits the joint $(\theta, x)$ space and evolves via a Metropolis–Hastings update, where at annealing step $k$ :

Propose $\theta'$ with a symmetric kernel, simulate $L(y \mid \theta)$ 0, compute $L(y \mid \theta)$ 1.
Accept the new state with probability

$L(y \mid \theta)$ 2

where $L(y \mid \theta)$ 3 is the temperature at iteration $L(y \mid \theta)$ 4.

The temperature $L(y \mid \theta)$ 5 is adaptively decreased according to thermodynamic criteria:

Global energies (e.g., mean $L(y \mid \theta)$ 6), heat flux $L(y \mid \theta)$ 7, and specific heat $L(y \mid \theta)$ 8 are estimated on the fly.
The schedule $L(y \mid \theta)$ 9 is tuned—either to hold entropy production constant or for optimal “fast” annealing, minimizing total entropy produced while ensuring near-equilibrium (Gibbs state) behavior.

With a flat prior ( $x$ 0), $x$ 1 can be enforced by choice of $x$ 2, and the asymptotically optimal temperature schedule obeys $x$ 3. The ensemble’s $x$ 4 marginal converges to the true Bayesian posterior provided mixing is not hampered by over-fast annealing or insufficient proposal width. Notably, the algorithm never computes or forms $x$ 5 explicitly; it relies purely on forward simulation and adaptive acceptance, making it effective for models with high-dimensional, intractable likelihoods (Albert, 2015).

2. Annealed Likelihood and Importance Sampling for Latent Variable Models

In Bayesian settings where likelihoods are not analytically available but can be unbiasedly estimated, Likelihood Annealing Importance Sampling (alternatively, Annealed Importance Sampling for Estimated Likelihoods, AISEL) provides a scalable solution (Tran et al., 2014):

Constructs a sequence of annealed targets

$x$ 6

with $x$ 7.

The unbiased estimator $x$ 8 is used in place of the true $x$ 9; incremental importance weights

$x \sim L(\cdot \mid \theta)$ 0

are accumulated across stages.

Precision tuning of the likelihood estimator is critical: the optimal variance for $x \sim L(\cdot \mid \theta)$ 1 is

$x \sim L(\cdot \mid \theta)$ 2

for $x \sim L(\cdot \mid \theta)$ 3, so computational effort is minimized at optimal variance. The effect of likelihood noise is precisely quantified; for a linear scheduling of $x \sim L(\cdot \mid \theta)$ 4, the efficiency loss in effective sample size is only a mild exponential in $x \sim L(\cdot \mid \theta)$ 5. Annealed importance sampling in this manner enables robust posterior and marginal likelihood estimation in generalized (e.g., latent-variable) settings, with rigorous guidance for balancing computational cost and estimator variance (Tran et al., 2014).

3. Annealed Sequential Monte Carlo and Adaptive Control

In high-dimensional, non-linear state-space models with complex likelihood structure, LIKA methodologies have been extensively developed within the sequential Monte Carlo (SMC) paradigm, particularly via annealed controlled SMC (AC-SMC) (Fulop et al., 2022):

Define a path of intermediate targets with “inverse-temperature” $x \sim L(\cdot \mid \theta)$ 6:

$x \sim L(\cdot \mid \theta)$ 7

Gradually introduce observations by increasing $x \sim L(\cdot \mid \theta)$ 8 over a discrete schedule.
Construct globally optimal proposal distributions (policies) via backward recursion (analogous to dynamic programming), resulting in controlled proposals that yield exactly flat incremental weights—achieving zero-variance likelihood estimators in the ideal case.

The annealing schedule $x \sim L(\cdot \mid \theta)$ 9 is adaptively chosen, typically via effective sample size criteria. The methodology is incorporated into SMC $u(x) = \rho(x, y)$ 0, yielding theoretically stable, consistent, and asymptotically normal estimators of both parameters and state-trajectory posteriors, even for highly informative observations and high-dimensional latent spaces (Fulop et al., 2022).

4. Annealing and Emulator-Accelerated Inference for Expensive Likelihoods

When individual evaluations of the likelihood function are computationally expensive, as in parameter inference for complex forward models (e.g., in cosmology), LIKA is leveraged to minimize computational cost while preserving posterior accuracy (Paranjape, 2022):

Phase I: Anisotropic Simulated Annealing efficiently explores parameter space via Latin hypercube designs and energy-weighted local updates, rapidly localizing the posterior mode region.
Phase II: Gaussian Process regression is used to interpolate a surrogate cost function $u(x) = \rho(x, y)$ 1, trained on the samples from Phase I.
Phase III: Standard MCMC is run on the GP emulator, yielding accurate posterior samples at negligible computational cost.

Compared to brute-force MCMC or other emulator-based schemes, this approach achieves $u(x) = \rho(x, y)$ 2– $u(x) = \rho(x, y)$ 3 reduction in expensive likelihood calls for linear and highly non-linear examples, provided the cost of a single $u(x) = \rho(x, y)$ 4 exceeds $u(x) = \rho(x, y)$ 51 s (Paranjape, 2022).

5. Likelihood Annealing for Fast, Calibrated Uncertainty in Deep Regression

Likelihood Annealing has also been formulated as a fast, calibrated, uncertainty-quantifying training objective in deep regression tasks (Upadhyay et al., 2023):

The standard heteroscedastic negative log-likelihood is augmented by two temperature-annealed regularizers—pulling the point prediction $u(x) = \rho(x, y)$ 6 towards the observation and enforcing variance $u(x) = \rho(x, y)$ 7 to match the absolute error.
The augmented loss is

$u(x) = \rho(x, y)$ 8

with $u(x) = \rho(x, y)$ 9 under exponential annealing.

The practical effect is 2–6× faster convergence, calibration error reductions up to an order of magnitude, and improved uncertainty quantification for both low- and high-dimensional regression outputs. The method is general to any architecture and avoids post hoc calibration (Upadhyay et al., 2023).

6. Quantum Annealing and Deterministic Annealed Likelihood

Deterministic Quantum-Annealing EM (QAEM) exploits an annealed free-energy principle with quantum kinetic regularization for maximum likelihood estimation in nonconvex models (Miyahara et al., 2016):

The objective combines classical log-likelihood with a quantum-induced Laplacian term, controlled by a diminishing quantum parameter $E(\theta)$ 0.
The annealing schedule linearly interpolates inverse temperature $E(\theta)$ 1 from $E(\theta)$ 2 to $E(\theta)$ 3 and quantum strength $E(\theta)$ 4 from positive to $E(\theta)$ 5.
In the path-integral picture, this enables “tunneling” through sharp local minima, achieving higher global-optimum success rates and improved convergence relative to both classical EM and classical deterministic annealing EM.

Empirical results show that QAEM attains monotonic free-energy decrease, dramatic improvements in likelihood landscape navigation, and superior performance in mixture models and other highly multimodal scenarios (Miyahara et al., 2016).

7. Practical Guidance and Application Domains

Across the spectrum of LIKA methodologies, appropriate tuning of the annealing schedule, proposal width, and, where relevant, emulator fidelity is paramount:

Thermodynamic quantities, such as energy mean, heat capacity, and acceptance rates, should be monitored to assure equilibrium and robust mixing (Albert, 2015).
In latent-variable/intractable likelihood settings, precision of the likelihood estimator and schedule shape ( $E(\theta)$ 6 or $E(\theta)$ 7) must be judiciously selected for variance/cost trade-off (Tran et al., 2014, Fulop et al., 2022).
For expensive likelihoods, care should be taken to ensure that the emulator cross-validates to within a given fractional error threshold, and edge detection mechanisms should be employed to robustly enclose the relevant region of parameter space (Paranjape, 2022).
In deep learning for uncertainty quantification, annealing temperature should be exponentially decayed, and validation calibration metrics (e.g., Expected Calibration Error, Prediction Interval Coverage) should be monitored for early stopping (Upadhyay et al., 2023).

LIKA frameworks have demonstrated efficacy in stochastic dynamic systems, high-dimensional latent-variable models, structured macroeconomic models, regression tasks with aleatoric uncertainty, and cosmological parameter inference, among others. The unifying feature is the robust handling of complex, multimodal, and computationally demanding likelihood landscapes—accommodated by annealed, forward-simulation, and/or carefully regularized techniques that maintain Bayesian correctness and strong uncertainty quantification.