Bayesian Nested Sampling Retrievals

• Bayesian nested sampling retrievals are inference algorithms that transform multidimensional integrals into one-dimensional forms to simultaneously compute Bayesian evidence and posterior distributions.
• They utilize dynamic live point sampling and constrained methods such as MCMC and ellipsoidal approaches to efficiently navigate high-dimensional, multimodal parameter spaces in fields like astrophysics and cosmology.
• Recent innovations like neural flow proposals and field-theoretical post-processing reduce computational costs and uncertainty, enhancing performance in complex, high-dimensional inference tasks.

Bayesian Nested Sampling Retrievals

Bayesian nested sampling retrievals are a class of inference algorithms that enable simultaneous computation of the Bayesian evidence (marginal likelihood) and the posterior parameter distribution from either tractable likelihood models or complex forward simulators. Developed around the framework introduced by Skilling (2006), these methods transform the multidimensional evidence integral into a one-dimensional integral over the prior volume, facilitating robust parameter inference and model selection in high-dimensional, multimodal, or otherwise challenging posterior landscapes. They have become foundational in fields such as astrophysics, cosmology, exoplanet atmospheric retrieval, and phylogenetics due to their statistical rigor and broad algorithmic flexibility.

1. Mathematical Foundations and Core Algorithm

The core objective of Bayesian nested sampling is estimation of the evidence

$Z = \int_\Theta L(\theta)\,\pi(\theta)\,d\theta$

where $L(\theta)$ is the likelihood, $\pi(\theta)$ is the prior, and $\Theta$ denotes the parameter space. Nested sampling reformulates this integral by introducing the prior volume

$$X(\lambda) = \int_{L(\theta)>\lambda} \pi(\theta)\,d\theta,\quad 0 \leq X \leq 1$$

with the inverse mapping $L(X)$. This yields the one-dimensional form

$Z = \int_{0}^{1} L(X)\,dX$

Operationally, nested sampling maintains a set of $N$ “live points,” each independently sampled from the prior or a constrained prior $\pi(\theta)\,\mathbb{I}_{L(\theta) \geq L_{i}}$. At each iteration:

• Identify and remove the live point $\theta_\text{min}$ with the smallest likelihood $L_i = L(\theta_\text{min})$.
• Estimate the current prior volume as $X_i \approx \exp(-i/N)$.
• Record the incremental weight $w_i = X_{i-1} - X_i$ and update $Z \leftarrow Z + L_i w_i$.
• Replace the discarded point with a new draw from the prior restricted to $L(\theta) > L_i$, using appropriate sampling methodology (e.g., constrained MCMC, ellipsoidal rejection, normalizing flows).
• The process continues until a stopping criterion based on the estimated remaining evidence (e.g., $L^{\text{max}}_\text{live} X_i \ll Z$) is satisfied.

Posterior samples are accumulated as weighted sets $(\theta_i,\,w_i L_i)$, providing parameter inference “for free” alongside evidence estimation (Maturana et al., 2017).

2. Evidence Estimation, Posterior Sampling, and Uncertainty Quantification

In nested sampling, each discarded point $\theta_i$ is weighted proportionally to its contribution to $Z$, allowing construction of an approximate posterior by resampling with probabilities $p_i = (w_i L_i)/Z$. The effective sample size (ESS) of the posterior is

$\mathrm{ESS} = \exp(-\sum_i p_i \log p_i)$

The uncertainty in $\log Z$ is dominated by the stochasticity in the sequence of prior volumes $\{X_i\}$ due to order statistics. Standard deviation is estimated as

$\mathrm{SD}[\log Z] \simeq \sqrt{H/N}$

with “information” $$H = \int p(\theta) \log\left(p(\theta)/\pi(\theta)\right) d\theta \approx \sum_i (w_i L_i/Z) \log(L_i/Z)$$ (Maturana et al., 2017, Fowlie, 23 May 2025, Speagle, 2019). Empirical bootstrap methods or posterior reconstructive samplers (see below) enhance uncertainty estimation, especially in non-Gaussian or multimodal settings.

3. Sampling Strategies and Advanced Algorithms

Efficiently drawing new samples from the prior constrained by $L(\theta)>L_i$ is the central challenge. Algorithmic innovations include:

• Ellipsoidal sampling (MultiNest): Clusters live points into ellipsoidal bounds, inflating for coverage, and samples uniformly within them (Dittmann, 2024).
• Slice sampling and clustering (PolyChord, PolyStan): Employs slice sampling along randomly chosen directions in a whitened basis, periodic clustering to capture multimodal structure, and dynamic expansion (Fowlie, 23 May 2025).
• Hamiltonian/galilean constrained Monte Carlo: Integrates Hamiltonian/galilean dynamics with reflecting boundaries at the likelihood constraint, yielding efficient traversal in high dimensions and multimodal spaces (Betancourt, 2010, Feroz et al., 2013).
• Dynamic nested sampling (dynesty): Allocates live points adaptively according to posterior or evidence “importance,” focusing computational effort on “difficult” regions (Speagle, 2019).
• Neural and flow-based samplers: Normalizing flow-based models approximate highly nontrivial constraint surfaces, improving sample efficiency and scalability to $D\sim50-100$ (Villa et al., 3 Nov 2025).
• Importance Nested Sampling (INS, NAUTILUS): Recycles all samples (accepted or rejected), assigning importance weights that rigorously preserve unbiasedness and substantially increase efficiency (Lange, 2023).
• Phantom-powered NS: Reuses autocorrelated Markov chain proposals (“phantom points”) to further reduce likelihood calls with provable accuracy (Albert, 2023).

The optimal strategy depends on the geometry of the posterior, dimensionality, and computational constraints.

4. Developments in Uncertainty Reduction and Post-Processing

Recent work addresses the dominant “stochastic noise” in the $X$ sequence using nonparametric field-theory methods:

• Information Field Theory (IFT; (Westerkamp et al., 2023, Westerkamp et al., 2024)): Reconstructs the $L(X)$ curve by imposing smoothness and monotonicity, inferring the field $\tau(\ln X)$ via variational inference or HMC, and marginalizing over the latent $t_i$ (shrink factors). The evidence is then estimated as a distribution over possible $Z$ values, yielding quantifiably reduced uncertainties, particularly impactful for runs with modest $N$.
• These field-based methods diagnose pathologies, improve evidence accuracy for small live sets, and slot naturally as a post-processing step in Bayesian retrieval pipelines (Westerkamp et al., 2024).

5. Diagnostic Techniques, Tuning, and Best Practices

Robust Bayesian nested-sampling retrievals require careful calibration:

• Convergence diagnostics: Perform hyperparameter scans over $N_\text{live}$ and accuracy/volume parameters (e.g., ellipsoid inflation $\epsilon$ in MultiNest). Test for stability in both $Z$ and marginal posterior quantities (Dittmann, 2024).
• Posterior width analysis: Monitor credible-interval widths as a function of tuning parameters. Systematic trends (e.g., width shrinking with increasing $\epsilon$) signal bias.
• Effective sample size (ESS) and diagnostic tests: Report and monitor ESS for posterior stability. Use insertion-index or KS tests for uniformity of constrained sampling draws (Fowlie, 23 May 2025).
• Stopping criteria: Set the evidence tolerance (e.g., $\mathrm{tol} \ll \Delta\log Z_\text{target}$). Terminate when the maximum possible remaining $L_\text{max,live} \cdot X_i$ is negligible (Maturana et al., 2017, Fowlie, 23 May 2025).
• Handling “unrepresentative priors”: Posterior repartitioning (PR) techniques adjust the prior-likelihood factorization to rescue efficiency without biasing final inference (Chen et al., 2018).

These diagnostics are algorithm-agnostic and essential for avoiding the common pathologies of overconfident or systematically biased evidence and posteriors.

6. Algorithmic Innovations and Modern Implementations

Recent research advances include:

• PolyStan: Stan interface to PolyChord, robust for multimodal, degenerate, and discrete-latent models; provides internal diagnostic and “black-box” inference for complex hierarchies (Fowlie, 23 May 2025).
• NAUTILUS: Combines deep-learning with INS, using neural regressors to tightly learn proposal densities within the live region, dramatically improving efficiency and scaling (Lange, 2023).
• i-nessai: Normalizing flow-based INS designed for high-dimensional, multimodal posteriors as in PTA data, achieving ESS per likelihood call gains by $\sim10^2-10^3$ compared to parallel-tempering MCMC (Villa et al., 3 Nov 2025).
• PolySwyft: Merges NS with neural ratio estimation for likelihood-free settings, trading rounds of NRE for simulator calls and using KL-divergence–based adaptive termination (Scheutwinkel et al., 9 Dec 2025).
• Dynamic nested sampling (dynesty): Nets substantial efficiency boosts via adaptive allocation of live points and algorithmic sub-batching; supports Hamiltonian and slice-based within-volume moves (Speagle, 2019).

A representative summary of methods and capabilities:

Implementation	Multi-modal	INS/Reuse	Likelihood-free	Neural Proposals	Field-theoretical $L(X)$	Diagnostic Tests
MultiNest	Yes	No	No	No	No	Basic
PolyChord/Stan	Yes	No	No	No	No	Extensive
dynesty	Yes	No	No	No	No	Extensive
NAUTILUS	Yes	Yes	No	Yes	No	Yes
i-nessai	Yes	Yes	No	Yes	No	Yes
PolySwyft	Yes	N/A	Yes	Yes (NRE)	No	Yes
Field-theory (IFT)	Yes	N/A	N/A	N/A	Yes	N/A

7. Empirical Performance and Application Domains

Bayesian nested sampling retrievals have achieved state-of-the-art performance in a wide range of scientific inference tasks:

• Astrophysics/exoplanets: Used for atmospheric retrieval under both physics-based and data-driven forward models (Martinez et al., 2022, Lange, 2023). NS is the gold standard for accurate uncertainty quantification and rigorous model selection, but can underestimate uncertainties under strong model misspecification; CNN surrogates have emerged as a complementary tool for speed.
• Cosmology: Sequential evidence calculation with field-based $L(X)$ post-processing has been shown to reduce error bars and improve support for hierarchical or chained inference (Alsing et al., 2021, Westerkamp et al., 2023).
• PTA/gravitational-wave timing: i-nessai yields orders-of-magnitude gains in ESS/$N_{\text{likelihood}}$ over PTMCMC for $D\gtrsim 50$ with strong multimodality, and preserves accuracy of both evidence and posterior (Villa et al., 3 Nov 2025).
• Simulation-based inference: PolySwyft offers KL-driven self-validation and typically achieves $\sim10\times$ faster reliable convergence on high-D multimodal posteriors relative to plain NS or truncated NRE (Scheutwinkel et al., 9 Dec 2025).
• Phylogenetics: Application of NS has hedged issues of complex combinatorial trees, provided marginal likelihoods for evolutionary model selection, and retained uncertainty bounds at practical computational cost (Maturana et al., 2017).

Benchmarks consistently show that advanced INS, neural-proposal, and phantom-point powered methods (e.g., NAUTILUS, i-nessai, PolySwyft) reduce likelihood evaluations by $5\times$–$100\times$ over traditional NS or MCMC, especially in high $D$. Field-theoretical post-processing yields factor $\sim5$–$10$ reductions in uncertainty in $Z$ for low $N_{\text{live}}$ (Westerkamp et al., 2024, Westerkamp et al., 2023).

---

In summary, Bayesian nested sampling retrievals underpin rigorous statistical inference workflows where both parameter estimation and model comparison are required, especially for highly complex or multimodal targets. The continued development of sampling, diagnostic, and postprocessing methodologies has expanded their applicability and efficiency across the physical sciences, enabling robust, reproducible, and interpretable inference in domains characterized by intractable likelihoods or high-dimensional parameter spaces.