---
title: Nested Sampling for Bayesian Inference
url: https://www.emergentmind.com/topics/nested-sampling
type: topic
---

# Nested Sampling for Bayesian Inference

Nested sampling is a Monte Carlo algorithm introduced by Skilling (2004) for the efficient computation of Bayesian evidence (marginal likelihoods) and posterior inference in high-dimensional, multimodal, or degenerate probability distributions. The method reformulates the integral for Bayesian evidence into a one-dimensional integral over the prior volume, facilitating rigorous model comparison while producing posterior samples as a byproduct. Its sampling strategy and error quantification, adaptability to various domain geometries, and robust diagnostic and parallelization procedures have made it a cornerstone methodology in astronomy, cosmology, statistical physics, and engineering applications.

## 1. Mathematical Foundations and Evidence Integral

In Bayesian inference, the evidence for model parameters $\theta \in \Omega$ is given by
\[
Z = \int_{\Omega} L(\theta)\,\pi(\theta)\,d\theta,
\]
where $L(\theta)$ is the likelihood and $\pi(\theta)$ is the prior density on $\Omega$. Nested sampling reformulates this high-dimensional integral as a one-dimensional integral over the “prior volume” $X$:
\[
X(\lambda) = \int_{L(\theta) > \lambda} \pi(\theta) d\theta,
\]
so that $X(\lambda)$ is a monotonically decreasing function as the threshold $\lambda$ increases. The evidence integral becomes
\[
Z = \int_0^1 L(X) dX,
\]
where $L(X)$ is the inverse function of $X(\lambda)$, mapping a prior mass $X$ to the corresponding likelihood level. This transformation is exact and allows the computation of $Z$ via a numerically stable quadrature over $X$ [2101.09675][2205.15570][2310.03040][1312.5638].

## 2. Core Algorithm and Implementation

Nested sampling maintains a set of $N$ “live” points $\{\theta_i\}$ sampled from the prior. At each iteration, it:

1. Identifies and removes the live point with the smallest likelihood, $L_i$.
2. Estimates the shrinkage of the prior volume via an order-statistics result: $t_i \sim \operatorname{Beta}(N,1)$, so $X_i = t_i X_{i-1}$, with $X_0 = 1$.
3. Accumulates the evidence contribution $Z \leftarrow Z + L_i (X_{i-1} - X_i)$.
4. Replaces the discarded point by sampling from the prior restricted to $L(\theta) > L_i$.
5. Repeats until the remaining possible evidence contribution is negligible.

Posterior samples and weighted estimates for any function $f(\theta)$ are produced from the sequence of dead points and their associated weights $w_i = X_{i-1} - X_i$ [2101.09675][2205.15570][1312.5638][1005.0157]. Deterministic approximations $X_i \approx \exp(-i/N)$ are commonly employed in practical settings.

## 3. Sampling Strategies: Constrained Priors and Algorithmic Variants

The critical computational step in nested sampling is drawing new live points from the constrained prior $\pi(\theta) \,\mathbb{I}[L(\theta) > L_i]$, where $\mathbb{I}$ is the indicator function. Principal strategies include:

- **Region Samplers** (e.g., MultiNest): Define simple geometric shapes (ellipsoids, clusters) containing the live points, sampling uniformly within or via rejection [1407.5459][2205.15570].
- **Markov Chain Methods** (e.g., PolyChord, slice sampling): Evolve the state via MCMC tailored to respect the hard likelihood constraint [2012.15286][2101.09675][2308.05816].
- **Hamiltonian/Geometric Methods**: Utilize reflected Hamiltonian trajectories (CHMC [1005.0157], Galilean MC [1312.5638]), yielding high-probability exploration within constrained contours.
- **Geometric Samplers**: For domains such as tori and spheres, employ wrapped proposals or projections to avoid boundary inefficiencies; e.g., geometric nested sampling (GNS) applies embedded move sets on non-Euclidean parameter spaces [2002.04123][1905.09110].

Advanced region samplers employ machine learning—normalizing flows or bijectors—to model arbitrary priors or to efficiently represent complex constraint boundaries [2102.12478][2205.02030].

### Key Algorithmic Variants

| Variant                       | Sampling Kernel             | Feature                                           |
|-------------------------------|-----------------------------|---------------------------------------------------|
| Ellipsoidal/Cluster Samplers  | Rejection in union regions  | Fast in low $d$, poor in high $d$ multimodal      |
| Slice sampling/MCMC           | Local moves (constraint)    | Polynomial scaling in $d$, robust to degeneracies |
| Hamiltonian MC                | Reflected HMC, Galilean     | Fast, handles thin curving regions                |
| Geometric NS                  | Domain-wrapped proposals    | Efficiency on circles, spheres, tori              |
| Snowballing NS                | Increasing $N$, fixed steps | Asymptotically unbiased, MCMC step stabilization  |
| Bijector-based NS             | Learned invertible transforms| Arbitrary prior/geometric support                |

## 4. Error Analysis, Uncertainty Quantification, and Diagnostics

Nested sampling provides principled error estimates for the computed evidence. The dominant source of Monte Carlo error arises from the stochastic nature of the volume shrinkage $t_i$. Leading-order uncertainty in the log-evidence is
\[
\sigma_{\ln Z} \approx \sqrt{H/N},
\]
where $H$ is the information gain (Kullback–Leibler divergence) from prior to posterior mass in $X$ space [2101.09675][2211.03258][1102.0996].

Skilling’s information-theoretic variance and Keeton’s moment-propagation variance agree to leading order, both reducing to $M \sigma_Z^2/Z^2 \simeq -1 + \langle -\ln X \rangle$ as long as $L(X)$ is locally smooth and $\Delta Z/Z \ll 1$ [2211.03258][1102.0996]. Statistical error from a single NS run (via volume-shrinkage resampling) closely matches the run-to-run variance over independent runs.

Diagnostic tests for correctness and implementation-specific bias are essential:

- **Insertion index uniformity**: Checks the uniformity in the rank at which new live points enter the likelihood-sorted list. Significant deviations (via KS tests) flag failures in constrained prior sampling or the presence of likelihood plateaus [2006.03371][2101.09675].
- **Shrinkage tests**: Compare actual volume shrinkages to the theoretical Beta distribution, identifying over- or under-compression, e.g., due to region sampling errors [1407.5459].
- **Thread-based and multi-run variance diagnostics**: Partitioning and comparing “live-point threads” across runs to gauge implementation variance and mode-finding robustness [1804.06406].

Visual tools such as $log\,X$-sample trace diagrams and bootstrap-based uncertainty plots add further diagnostic granularity [1804.06406][1905.04768].

## 5. Algorithmic Extensions: Geometries, Dynamic NS, Parallelization

Nested sampling is highly adaptable to different geometries:

- Wrapped proposal distributions for periodic domains (circles/tori) and projection moves for spheres prevent boundary-induced sampling bias and increase acceptance rates in non-trivial geometries [2002.04123][1905.09110].
- Learned bijective transforms allow arbitrary priors and enable nested sampling in complex constraint spaces without analytic prior transforms [2102.12478].

**Dynamic Nested Sampling** (dynesty, dyPolyChord) dynamically varies the number of live points or allocates sampling density to regions with the highest evidence or posterior mass, optimizing resource allocation during the run [2101.09675][2205.02030].

**Parallelization** can be exploited by batch removal/insertion of live points or by farmed constrained-prior samplers, with careful attention to shrinkage-statistic variance [1603.02516][2101.09675].

**Snowballing Nested Sampling** progressively increases the number of live points, stabilizing MCMC-proposal parameters and converging asymptotically to unbiased evidence and posterior estimates without requiring ever-longer MCMC chains at fixed $N$ [2308.05816].

## 6. Applications, Performance, and Limitations

Nested sampling is applied across a range of scientific domains:

- **Bayesian model selection**: Calculation of Bayes factors for cosmology, particle physics, and model choice scenarios [2205.15570][2310.03040].
- **Multimodal and degenerate inference**: Simultaneous localization and mapping (SLAM) factor graphs, high-dimensional Gaussian mixtures, object detection, particle event-generation phase space [2109.10871][1312.5638][2205.02030].
- **Statistical physics**: Computation of partition functions, free energies, and thermodynamic observables in the Potts and Ising models [1603.02516].
- **Engineering and rare event estimation**: Quantification of extremely small probabilities in reliability engineering or finance [2310.03040].

Algorithmic performance is competitive or superior to thermodynamic integration and traditional MCMC/posterior samplers, especially in high dimensionality or severe multi-modality. For evidence estimation, NS achieves uncertainties scaling as $\sim 1/\sqrt{N}$, provided all posterior modes are populated by live points.

**Limitations:** Failure to uniformly sample from the constrained prior (e.g., due to inadequate region construction or insufficient MCMC decorrelation) induces evidence bias or underexplored modes [1407.5459][2006.03371]. Plateaus and discontinuities in the likelihood lead to non-uniformity in the transformed $X$-space, necessitating special handling for both evidence and uncertainty estimation [2310.03040][2101.09675]. In very high-dimensional problems, region samplers become exponentially inefficient; step-based or flow-based adaptations are required [2205.15570][2012.15286]. Memory and computational requirements scale with the product of the number of live points and parameter dimension.

## 7. Software Ecosystem, Best Practices, and Current Trends

Well-maintained, high-performance open-source NS implementations include MultiNest, PolyChord, dynesty, UltraNest, cpnest, JAXNS, and ecosystem tools for diagnostics (nestcheck), visualization (anesthetic), and postprocessing [2012.15286][1905.04768][1804.06406].

**Best Practices:**

- Select the number of live points commensurate with posterior complexity and evidence tolerance.
- Employ diagnostic checks (insertion rank, shrinkage, multi-run variance) routinely [2006.03371][1804.06406][2101.09675].
- Use appropriate sampling kernels for the problem’s geometry and dimensionality.
- Publish run metadata (priors, code, evidence, diagnostics) and, where possible, the sequence of discarded (“dead”) points and weights for transparency and replication [2205.15570].
- Address plateaus and rare events with algorithmic extensions as required [2310.03040].

**Trends** in the field include the adoption of flow-based proposals, dynamic allocation of sampling effort, ensemble and parallelized approaches, and robust error estimation for both evidence and posterior moments [2101.09675][2102.12478][2308.05816]. The approach continues to adapt to new challenges in high-dimensional, multimodal, non-Euclidean, or non-standard integration domains.

---

**References**  
- [1312.5638], [1005.0157], [2101.09675], [2205.15570], [2012.15286], [2308.05816], [2002.04123], [1905.09110], [1804.06406], [1905.04768], [2102.12478], [2211.03258], [1102.0996], [1603.02516], [1407.5459], [2006.03371], [2109.10871], [2205.02030], [2310.03040].

Source: https://www.emergentmind.com/topics/nested-sampling