Multilevel Neural Simulation-Based Inference (MLMC-SBI)

Last updated: June 11, 2025

Advances in computational modeling ° across many scientific and engineering domains have cemented simulation-based inference ° (SBI °) as the go-to approach for Bayesian parameter estimation ° when the likelihood function ° is intractable but forward simulation is feasible. Modern neural SBI methods ° employ networks like normalizing flows ° to learn likelihood or posterior surrogates, enabling accurate uncertainty quantification and powerful amortized inference ° for complex simulators. However, a central obstacle remains: computational bottlenecks ° due to high-fidelity simulation ° costs. When each run of a high-fidelity simulator ° (e.g., in cosmology, biology, or engineering) is expensive, the size of the simulation dataset is limited, constraining the accuracy of neural inference.

"Multilevel neural simulation-based inference" (Hikida et al., 6 Jun 2025 ° ) presents a framework to address this challenge by incorporating Multilevel Monte Carlo ° (MLMC °) methods into neural SBI ° workflows. MLMC is a well-established variance-reduction technique ° in numerical simulation scenarios ° where simulators of different fidelity and computational cost are available. This paper extends MLMC from its origins in forward uncertainty quantification to the training of neural density estimators ° for SBI, enabling practitioners to combine fast, low-fidelity simulations with slower, high-fidelity ones for optimal efficiency and accuracy.

Problem Setup and Motivation

High-fidelity simulators °, while accurate, are typically so computationally intensive that only a limited number of simulations can be produced. Meanwhile, lower-fidelity versions—produced by reducing resolution, using approximate solvers, or dropping expensive sub-processes—are much less costly but yield less accurate results.

The core question: How can neural SBI harness both low- and high-fidelity simulators to achieve the most accurate posterior inference ° for a given total simulation cost?

MLMC-Enhanced Neural SBI: Method and Implementation

Multilevel Monte Carlo Principle

Let $G^l$ denote the simulator at fidelity level $l$ ($l=0$ is lowest cost, $l=L$ is highest fidelity/highest cost), with simulation cost $C_l$. MLMC uses a telescoping sum to express expectations under the highest-fidelity distribution:

$\mathbb{E}_{G^L}[f(z)] = \mathbb{E}_{G^0}[f(z)] + \sum_{l=1}^L \mathbb{E}_{G^l}\left[ f(z) - f^{l-1}(z) \right]$

where the bracketed terms represent corrections between successive fidelity levels. The estimator's variance is minimized by correlating $f^l$ and $f^{l-1}$ using shared simulation seeds (seed-matching), enabling accurate estimation of the highest-fidelity expectation with more cheap simulations and fewer costly ones.

Neural SBI Training with MLMC

The neural SBI objective (negative log-likelihood for NLE °, negative log posterior for NPE °) can always be written as an expectation over simulator outputs for sampled parameters $\theta$:

• For NLE (Neural Likelihood Estimation), the standard loss for parameter $\phi$ is

$f^{l, \text{NLE}}_\phi(u, \theta) = -\log q_\phi^{\text{NLE}}(G^l_\theta(u)\ |\ \theta)$

• For NPE (Neural Posterior Estimation), possibly using $m$ simulated outputs per $\theta$:

$f^{l, \text{NPE}}_\phi(u_{1:m}, \theta) = -\log q_\phi^{\text{NPE}}(\theta\ |\ G^l_\theta(u_1),\ldots,G^l_\theta(u_m))$

The MLMC estimator for the neural network loss becomes:

[ \ell_{\text{MLMC}}(\phi) = \frac{1}{n_0} \sum_{i=1}^{n_0} f^{0_\phi(u_{1:m,i}0,} \theta_i⁰⁾ + \sum_{l=1}^L \frac{1}{n_l} \sum_{i=1}^{n_l} \big[ f^{l_\phi(u_{1:m,i}l,} \theta_i^l) - f^{{l-1}\phi(u{1:m,i}l,} \theta_i^l) \big] ]

where:

• $n_l$ is the number of samples at level $l$ (with $n_0 \gg n_1 \gg \cdots \gg n_L$),
• Each $f^l$ is evaluated by drawing $\theta$ and random seeds $u$ and running $G^l$ and $G^{l-1}$ with the same $u$ (seed-matching, for variance reduction).

Variance Analysis and Sample Allocation

The variance of the MLMC loss estimator at each level depends on the smoothness (Sobolev norm) and similarity between adjacent simulator levels:

$\operatorname{Var}[h_l(\phi)] \leq \frac{K_l(\phi)}{n_l} \left( \|G^l - G^{l-1}\|_{W^{1,4}}^2 + 1 \right)$

More low-fidelity ($n_0$), fewer high-fidelity ($n_L$) samples are optimal; sample size at each level is set to minimize total variance under a fixed computational budget:

$n_l^\star \propto \frac{C_{\text{budget}}}{\sqrt{C_l + C_{l-1}}} \sqrt{ \|G^l - G^{l-1}\|_{W^{1,4}}^2 + 1 }$

Intuitive implication: Allocate most samples to the cheapest simulator, use more expensive levels only to correct for their inaccuracy.

Practical Implementation Considerations

• Seed matching (common randomness) across $G^l$ and $G^{l-1}$ is crucial for variance reduction. This typically requires simulators to accept the same input random seed ° and be strictly reproducible.
• Loss Optimization: The MLMC estimator is a sum of differences; naive stochastic gradient descent can be unstable. The implementation uses loss scaling and "gradient surgery" (gradient projection methods) to stabilize training.
• Multiple SBI variants: MLMC can be adapted for both NLE and NPE, for single or multi-output simulations.

Empirical Results

The paper demonstrates substantial gains across multiple scientific benchmarks:

• g-and-k Distribution (Synthetic): For a given simulation budget, MLMC-based NLE achieves tighter, more accurate posterior contours versus standard single-fidelity ° NLE.
• Ornstein-Uhlenbeck Process ° (Financial): MLMC-powered NPE is more robust than transfer learning baselines, avoiding the need for additional hyperparameter tuning.
• Synthetic Biology (Toggle-Switch): MLMC with more than two simulation fidelities robustly outperforms any single-level NLE, measured by maximum mean discrepancy °.
• Cosmological Inference ° (CAMELS): In this astrophysical simulation scenario—where high-fidelity runs are extremely costly—MLMC-SBI improves negative log posterior density ° and calibration at fixed cost compared to single-fidelity NPE.

Example empirical finding: MLMC-SBI can reach the same posterior accuracy as standard SBI at a fraction (often $<50\%$) of the high-fidelity simulation cost.

Limitations and Deployment Notes

• Requirements: Simulators must provide access to multiple fidelities and support seed-matching/randomness control.
• Engineering overhead: Minor increase (15–20%) in neural network training time, but negligible compared to simulator cost in real scientific applications.
• Not a replacement for all efficient SBI innovations: MLMC is compatible with other simulation-efficient SBI strategies, such as sequential training, truncated proposals, and adaptive simulation allocation.

Key Takeaways

• Simulation Efficiency: MLMC enables effective sharing and correction between fast/cheap and slow/expensive simulators, directly reducing the high-fidelity simulation burden.
• Quality Gains: MLMC-trained SBI achieves state-of-the-art inference accuracy under fixed budgets, backed by theoretical variance-reduction guarantees.
• Wide Applicability: The approach is tested across finance, biology, and cosmology but is generally applicable to any scenario with multi-fidelity ° simulators—making it a practical upgrade for neural SBI workflows in computational science °.

Example: Implementing MLMC-SBI (in Pseudocode)

for l in range(L+1):
    for i in range(n_l):
        theta = sample_prior()
        u = sample_seed()
        x_l = G_l(theta, u)
        if l == 0:
            # Base fidelity loss
            loss_l.append(-log_posterior(theta | x_l))
        else:
            x_lm1 = G_{l-1}(theta, u)
            loss_l.append(-log_posterior(theta | x_l) - (-log_posterior(theta | x_lm1)))
total_loss = mean(loss_0) + sum([mean(loss_l) for l in range(1, L+1)])

Conclusion

Multilevel neural simulation-based inference (Hikida et al., 6 Jun 2025 ° ) offers a robust, general, and simulation-efficient paradigm for neural SBI where expensive simulators are the bottleneck. MLMC-SBI achieves high-quality posterior inference with a carefully balanced mix of high- and low-fidelity simulations. Its flexibility allows direct integration with existing SBI toolkits and neural architectures, making it a compelling choice for modern Bayesian workflows in science and engineering.