---
title: Marginal Neural Ratio Estimation (MNRE)
url: https://www.emergentmind.com/topics/marginal-neural-ratio-estimation-mnre
type: topic
---

# Marginal Neural Ratio Estimation (MNRE)

Searching arXiv for recent and foundational papers on Marginal Neural Ratio Estimation to ground the article.
Marginal Neural Ratio Estimation (MNRE) is a simulation-based inference technique for Bayesian parameter estimation with intractable likelihoods, designed to learn marginal posteriors for low-dimensional subsets of parameters directly from simulations rather than by first approximating a full joint posterior and then numerically marginalizing nuisance variables. In the standard formulation, a neural network is trained to estimate a likelihood-to-evidence ratio, or equivalently a posterior-to-prior ratio, by distinguishing joint samples from product-of-marginals samples; when the network is fed only a subset of parameters, the resulting ratio is already marginalized over the remaining coordinates [2011.13951][2110.00449][2303.07339]. This makes MNRE particularly suited to scientific settings in which the parameter space is high-dimensional, the simulator is expensive, and the quantities ultimately reported are one- and two-dimensional marginals rather than the full posterior [2011.13951][2107.01214].

## 1. Conceptual foundations

The canonical setting for MNRE is simulation-based inference with a forward model
\[
\theta \mapsto x \sim p(x\mid \theta),
\]
where one can generate synthetic observations but cannot evaluate \(p(x\mid \theta)\) analytically or cheaply. In this setting, Bayesian inference still targets \(p(\theta\mid x_{\mathrm{obs}})\), but the likelihood must be bypassed rather than computed explicitly [2303.07339][2403.14750].

Within the neural SBI literature, the standard taxonomy distinguishes Neural Posterior Estimation, which models \(p(\theta\mid x)\) directly; Neural Likelihood Estimation, which models \(p(x\mid \theta)\); and Neural Ratio Estimation, which models a likelihood-to-evidence ratio with a classifier [2303.07339][2403.14750]. MNRE belongs to the ratio-estimation family, but differs from full-joint NRE by targeting marginal posteriors over selected parameters \(\theta_A\) rather than the full parameter vector \(\theta\) [2011.13951][2110.00449].

This emphasis on marginals is motivated by a recurring scientific pattern. In many physics and astronomy applications, the primary scientific object is not the full high-dimensional posterior, but low-dimensional marginals used for credible intervals, corner plots, and substantive interpretation [2011.13951]. From that perspective, learning the full joint distribution can be both statistically and computationally excessive. A concise characterization given in the arbitrary-subset formulation is that MNRE is standard NRE applied to \((x,\theta_A)\) instead of \((x,\theta)\), with nuisance parameters integrated out implicitly by the simulator [2110.00449].

A further extension is arbitrary-subset marginalization. In Arbitrary Marginal Neural Ratio Estimation, a binary mask \(a\in\{0,1\}^d\) specifies which coordinates are retained, allowing a single amortized network to provide marginal posteriors over arbitrary subsets without retraining and without numerical integration over complements [2110.00449]. This suggests a unifying interpretation of MNRE as a family of classifier-based posterior-to-prior estimators indexed by parameter subsets.

## 2. Mathematical formulation

The basic ratio used throughout MNRE is
\[
r(x,\theta) = \frac{p(x\mid \theta)}{p(x)} = \frac{p(\theta\mid x)}{p(\theta)}.
\]
Once \(r(x,\theta)\) is known, the posterior follows from
\[
p(\theta\mid x)\propto r(x,\theta)\,p(\theta)
\]
[2303.07339][2011.13951].

Neural ratio estimation obtains this quantity through binary classification. One generates positive samples from the joint distribution,
\[
(x,\theta)\sim p(x,\theta)=p(\theta)p(x\mid \theta),
\]
and negative samples from the product of marginals,
\[
(x,\theta)\sim p(x)p(\theta),
\]
often by shuffling \(x\) and \(\theta\) to destroy their dependence. A neural classifier with output \(\pi_\phi(x,\theta)\) is trained with binary cross-entropy loss; at the optimum,
\[
\frac{\pi^*(x,\theta)}{1-\pi^*(x,\theta)}=\frac{p(x\mid \theta)}{p(x)}=r(x,\theta),
\]
so the classifier logit approximates \(\log r(x,\theta)\) [2303.07339][2011.13951].

MNRE replaces \(\theta\) by a subset \(\theta_S\). The marginal posterior satisfies
\[
p(\theta_S\mid x)=\frac{p(x\mid \theta_S)p(\theta_S)}{p(x)},
\]
with
\[
p(x\mid \theta_S)=\int p(x\mid \theta_S,\theta_{\bar S})\,p(\theta_{\bar S}\mid \theta_S)\,d\theta_{\bar S}.
\]
The corresponding marginal likelihood-to-evidence ratio is
\[
r_S(x,\theta_S)=\frac{p(x\mid \theta_S)}{p(x)}=\frac{p(\theta_S\mid x)}{p(\theta_S)}.
\]
MNRE trains the same classifier construction on \((x,\theta_S)\), using simulations from the full prior but discarding \(\theta_{\bar S}\) at the network input. At optimum, the network approximates \(\log r_S(x,\theta_S)\), and the marginal posterior is recovered as
\[
p(\theta_S\mid x_{\mathrm{obs}})\propto \exp(d_{\phi,S}(x_{\mathrm{obs}},\theta_S))\,p(\theta_S)
\]
[2303.07339][2011.13951].

In the arbitrary-subset formulation, the subset is encoded by a mask \(a\), and the classifier is written as \(d_\phi(\theta_a,x,a)\), or equivalently \(d_\phi(x,\theta\cdot a,a)\) to maintain fixed-dimensional inputs. This yields
\[
r_a(\theta_a,x)=\frac{p(\theta_a\mid x)}{p(\theta_a)},
\]
for any non-empty subset specified by \(a\) [2110.00449]. This construction avoids numerical marginalization over nuisance coordinates entirely.

## 3. Truncation, nesting, and autoregressive extensions

Truncated Marginal Neural Ratio Estimation (TMNRE) extends MNRE by sequentially restricting the prior support to regions where the posterior for a target observation has non-negligible mass [2107.01214]. Instead of proposing simulations from the full prior in every round, TMNRE defines a truncated prior
\[
p_\Gamma(\theta)\propto p(\theta)\,\mathbb{I}(\theta\in\Gamma),
\]
where \(\Gamma\) is chosen from approximate posterior information. In the original truncation scheme, the region is updated by thresholding marginal ratios relative to their maxima with a cutoff \(\epsilon\), yielding a nested sequence of constrained supports [2011.13951][2107.01214].

A central feature of TMNRE is local amortization. Because the estimator is trained on a truncated prior rather than on a single fixed observation, it remains amortized over observations generated from that region, which in turn makes empirical posterior tests feasible [2107.01214][2111.08030]. This is presented as a distinctive advantage: TMNRE is simulation efficient and also enables fast empirical posterior testability [2107.01214].

The swyft line of work adds a second layer of efficiency through simulation reuse. An inhomogeneous Poisson point process cache stores previously simulated parameter points and their outputs, allowing later truncation rounds and even new inference tasks to recycle simulations rather than regenerate them [2011.13951]. In that framework, “nested ratio estimation” is conceptually the same as MNRE, with truncation and caching used to focus simulation effort on plausible regions while retaining all previous simulator calls [2011.13951].

Recent work generalizes TMNRE beyond axis-aligned truncation. Autoregressive Neural Ratio Estimation factorizes the joint posterior into one-dimensional conditionals and learns the corresponding conditional ratios, thereby addressing correlated high-dimensional posteriors more robustly than a single joint classifier [2308.08597]. The same paper replaces hyper-rectangular truncation by block-wise correlated truncation based on a high-dimensional ratio contour, and introduces a slice-based nested sampling algorithm that can draw both posterior samples and constrained prior samples from the learned ratio estimator [2308.08597]. This suggests a broader evolution of MNRE from direct low-dimensional marginals toward structured high-dimensional ratio models that remain compatible with sequential truncation.

A conceptually distinct but related generalization is “detection is truncation,” where detection itself is interpreted as a truncation in latent source space. In that setting, TMNRE is used to couple source detection, selection functions, and population inference by conditioning simulations on bright-source configurations compatible with the observed data [2211.04291].

## 4. Architectures, software, and workflow

MNRE is architecturally flexible. The foundational arbitrary-subset implementation uses fully connected MLPs for toy models and a residual network with 17 residual blocks for gravitational-wave parameter inference [2110.00449]. In application papers, image-like data are processed with CNN embeddings plus MLP heads for ratio estimation, as in strong-lensing warm-dark-matter inference [2205.09126] and halo-map reconstruction from \(N\)-body simulations [2206.11312]. In supernova analyses, separate per-object embedders and dataset summarizers are used to process irregular collections of light curves before feeding parameter-specific ratio heads [2209.06733][2403.07871].

The most widely used software framework is swyft, introduced as an open-source implementation of nested neural likelihood-to-evidence ratio estimation together with a Poisson point process cache for simulation reuse [2011.13951]. In swyft, a typical workflow is to define the simulator, specify a factorized prior, choose marginals of interest, configure the number of truncation rounds and samples per round, and train a shared featurizer with separate parameter heads [2011.13951]. The 21-cm reionization analysis explicitly uses swyft’s implementation of truncated MNRE to obtain all one- and two-dimensional marginals from a single simulation set [2303.07339].

At the opposite end of the software spectrum, a later 21-cm tutorial presents a beginner-friendly PyTorch pipeline that implements MNRE directly with a simple MLP, plain NumPy/pickle data handling, and binary cross-entropy training, deliberately avoiding specialized SBI frameworks [2509.06834]. This demonstrates that the core method does not depend on a particular software stack: what is essential is the classifier-based approximation of the posterior-to-prior ratio.

## 5. Scientific applications

MNRE and TMNRE have been deployed across a broad range of astrophysical and cosmological problems, typically in regimes with expensive simulators, many nuisance parameters, and interest in low-dimensional marginals.

| Domain | Representative paper | Reported outcome |
|---|---|---|
| 21-cm CD/EoR inference | [2303.07339] | Accurate six-parameter posteriors with \(2\times10^4\) simulations; \(\sim10^4\) sufficient in convergence tests |
| CMB and likelihood-free cosmology | [2111.08030] | Orders of magnitude fewer simulator calls than conventional MCMC |
| Stage IV 3×2pt LSS | [2403.14750] | Speedup factor of \(\sim 10\)–60 over classical MCMC methods |
| Strong lensing and WDM | [2205.09126] | Expected 95% credible lower limit on \(m_{\mathrm{WDM}}\) around \(6.5\,\mathrm{keV}\) in the scenario closest to CDM |
| Standard sirens and \(H_0\) | [2301.05241] | Corrects for \(\sim 90\%\) of \(H_0\) bias with extensive tiling of the localization region |
| SN Ia cosmology | [2209.06733] | Unbiased and precise cosmological posteriors for up to 100 000 SNe Ia |
| Real SN Ia dust inference | [2403.07871] | Implicitly marginalises over 4000 latent variables for \(\approx 100\) SNe Ia |

The 21-cm reionization study is a particularly explicit worked example. Using 21cmFAST, a six-dimensional astrophysical parameter vector, and SKA-like thermal noise, the method recovers the posterior distribution at significantly lower computational cost than conventional likelihood-based MCMC and shows a factor of 3–10 improvement in simulation efficiency relative to the comparison baseline [2303.07339]. The same training dataset is then reused to study redshift dependence and modeling uncertainty without rerunning 21cmFAST [2303.07339].

In strong gravitational lensing, TMNRE is used to target the half-mode mass \(M_{hm}\) while marginalizing over lens, source, and halo-population uncertainties. There the sequential truncation of nuisance parameters is central: six rounds of \(10^5\) simulations per round are used to constrain smooth lens and source parameters before training the final cutoff-mass estimator [2205.09126]. The paper combines multiple images through a product of ratios, underscoring that MNRE-style estimators can be composed across independent datasets [2205.09126].

Large-scale-structure and CMB applications emphasize nuisance-parameter scalability. In the Stage IV 3×2pt analysis, MNRE is applied to 17- and 19-dimensional cosmological models but reconstructs only one- and two-dimensional marginals, producing posteriors that match Metropolis–Hastings and MultiNest while achieving speedups of \(\sim 40\times\) in \(\Lambda\)CDM and \(\sim 60\times\) in decaying-dark-matter extensions [2403.14750]. The earlier CMB-focused TMNRE study similarly reports orders-of-magnitude reductions in simulator calls and stresses that the required number of samples is effectively independent of the number of nuisance parameters [2111.08030].

Supernova work has pushed the method into fully hierarchical settings. SICRET applies TMNRE to a BAHAMAS-style hierarchical model and verifies unbiased cosmological posteriors for up to 100 000 SNe Ia while also showing that local amortization can be used to convert approximate Bayesian posteriors into exact-coverage frequentist confidence regions [2209.06733]. SIDE-real then extends the methodology to real low-redshift SN Ia light curves with a hardware-accelerated forward model, inferring population-level dust parameters and individual-object parameters while marginalizing over thousands of latent variables [2403.07871].

## 6. Validation, strengths, and limitations

A recurrent theme in the MNRE literature is that approximate posteriors must be tested, not merely produced. Coverage diagnostics appear in the 21-cm reionization analysis, where empirical coverage for all one- and two-dimensional marginals agrees with nominal levels within uncertainties [2303.07339]; in the warm-dark-matter lensing analysis, where the expected-coverage curve lies close to the diagonal [2205.09126]; in the Stage IV LSS study, where 500 mock observations are used to verify empirical coverage of \(\Lambda\)CDM marginals [2403.14750]; and in the CMB, supernova, and SIDE-real TMNRE papers, where local amortization is explicitly presented as the mechanism that makes such tests practical [2111.08030][2209.06733][2403.07871].

Across these studies, several advantages recur. MNRE directly targets the one- and two-dimensional marginals actually used in scientific interpretation, automatically marginalizes nuisance parameters through simulation, and avoids explicit numerical integration over complements [2011.13951][2110.00449]. TMNRE further focuses simulation effort through prior truncation, often yielding order-of-magnitude gains in simulator efficiency while preserving local amortization [2011.13951][2107.01214]. Reuse of simulations across rounds, across marginals, and in some cases across related inference tasks is a distinctive operational benefit [2011.13951][2303.07339].

The limitations are equally consistent. Results are only as good as the forward model, so semi-numerical or approximate simulators can induce biased posteriors unless model discrepancy is explicitly treated [2303.07339][2205.09126][2403.14750]. Neural training can produce miscalibrated posteriors if architectures or hyperparameters are poor, which is why coverage tests are repeatedly emphasized [2303.07339][2011.13951]. Prior support matters: extrapolation beyond the training prior or into poorly sampled corners is unsafe [2303.07339]. Truncation itself can bias results if \(\epsilon\) is too aggressive or if strong correlations cause posterior mass to be chopped off, a failure mode discussed explicitly in the original TMNRE work and in the swyft literature [2107.01214][2011.13951].

A broader synthesis suggested by these papers is that MNRE is best understood not as a single fixed architecture but as a methodological pattern: direct estimation of marginal posterior-to-prior ratios from simulator output, optionally combined with truncation, caching, autoregressive factorizations, and calibration procedures. In that sense, MNRE has become a general tool for likelihood-free inference whenever the scientifically salient outputs are low-dimensional marginals and the dominant computational burden lies in simulating realistic data rather than evaluating tractable likelihoods [2110.00449][2308.08597][2403.07871].

Source: https://www.emergentmind.com/topics/marginal-neural-ratio-estimation-mnre