Dynamic Nested Sampling for Bayesian Inference

Updated 15 November 2025

Dynamic Nested Sampling is an adaptive Bayesian technique that varies the number of live points to optimally balance evidence and posterior estimation.
It employs importance functions and iterative batch strategies, such as snowballing, to focus computational effort on high-impact regions in the parameter space.
This approach enhances statistical efficiency and reduces likelihood evaluations, often achieving significant speedups and improved diagnostic stability.

Dynamic Nested Sampling (DNS) is a class of algorithms that generalize the classic nested sampling approach for Bayesian evidence computation and posterior inference by adaptively varying the number of live points (“particles”) during sampling. DNS optimally allocates computational resources to regions of the parameter space where they most reduce uncertainty in the evidence or posterior estimates. Unlike classic (static) nested sampling, which uses a fixed number of live points throughout a run, DNS dynamically increases or redistributes live points during the calculation, leading to improved statistical efficiency and, in some formulations such as snowballing nested sampling, more stable diagnostics and simplified calibration.

1. Principles of Nested Sampling and the Motivation for Dynamic Variants

Standard nested sampling, as introduced by Skilling (2004), is designed to estimate the Bayesian evidence

$Z = \int_{\Theta} L(\theta)\,\pi(\theta)\,\mathrm{d}\theta$

where $\pi(\theta)$ is the prior and $L(\theta)$ the likelihood. The algorithm maintains $K$ live points sampled under the likelihood constraint $L > L_{\min}$ , and at each iteration removes the lowest-likelihood point and replaces it with a new sample satisfying the constraint. Evidence and posterior samples are then constructed from the sequence of “dead” points, with weights determined by prior volume shrinkage, $X_i = \exp(-i/K)$ .

In static NS, the number of live points $K$ is fixed throughout the run. However, posterior mass $\propto L(X) X$ concentrates sharply in log-prior volume, causing static NS to distribute dead points inefficiently—over-sampling regions that are largely irrelevant for either evidence or posterior estimation. Increasing $K$ improves evidence accuracy but at the cost of exponentially more likelihood evaluations before reaching posterior-dominated regions. Conversely, small $K$ accelerates reaching the posterior peak but sacrifices evidence accuracy, creating an unavoidable trade-off. DNS was introduced to resolve this inefficiency by reactively or proactively altering $K$ during the run (Higson et al., 2017, Buchner, 2021, Speagle, 2019, Buchner, 2023).

2. Dynamic Allocation Strategies

DNS operates by allocating live points where they are most informative for the statistical quantity of interest, which may be Bayesian evidence, posterior moments, or a user-defined composite objective. This is typically realized as follows:

Importance Functions: Live point allocation is guided by an importance or utility function $I(X)$ , such as

$I(X) = f I_{\rm post}(X) + (1-f) I_{\rm ev}(X)$

with $I_{\rm post}(X) = p(\theta)\propto L(X)X/Z$ for posterior accuracy, and $I_{\rm ev}(X)$ for evidence. Parameter $f\in[0,1]$ trades off between posterior and evidence accuracy (Speagle, 2019, Higson et al., 2017, Buchner, 2021). Dead points (removed samples) are ranked by their importance, and intervals in $X$ or likelihood space with high $I$ are targeted for increased live point density.

Iterative Batch Growth: After an initial static NS run with a minimal $K_{\rm base}$ , the importance profile is estimated; new static NS “batches” with higher $K_{\rm batch}$ are launched targeting specific likelihood intervals, and their dead points merged with the global run, updating the volume and weight estimates. This process is repeated until stopping criteria are met (Speagle, 2019, Higson et al., 2017).
Incremental and Uniform Growth (Snowballing): Snowballing nested sampling (SNS) (Buchner, 2023) eschews local reallocation and instead uniformly and incrementally increases $K$ over all levels (“snowballing”). The method fixes the MCMC chain length $M$ and adds batches of $\Delta K$ new live points at each extension, resuming the NS run while memoizing any repeated MCMC likelihood-restricted prior sampling (LRPS) to maximize efficiency.

The table below contrasts core strategies in DNS implementations:

Variant	Live Point Allocation	Targeted Region
DNS (Higson et al.)	$I(X)$ adaptive	Posterior/evidence
SNS	Uniform $\Delta K$ increments	All levels (global)

3. Algorithmic Formulations and Pseudocode

DNS frameworks can be formalized as agents operating on the breadth-first search (BFS) formulation of nested sampling, in which each node represents a sample with likelihood $L$ , and branches correspond to replacement operations (Buchner, 2021). The difference between static and dynamic versions is the live point spawning rule—static agents always replace each dead node with one child, while dynamic agents add children adaptively.

Canonical iterative batch-based DNS pseudocode (in the style of (Speagle, 2019, Higson et al., 2017)):

run static NS (K_base); record {θ_i, L_i, X_i, w_i}
while not converged:
    Compute importance I_i for all dead points
    For indices with I_i > threshold:
        Run static NS batch with K_batch in corresponding L-interval
        Merge batch dead points into global {θ_i, L_i, X_i, w_i}
    Update convergence/termination criteria

The SNS pseudocode (Buchner, 2023) is conceptually even simpler:

K = K0; run NS(K, M) # initial run with K0 live points, M MCMC steps
while not converged:
    K = K + ΔK
    resume NS(K, M), memoizing LRPS(L_min) calls
    output updated Z, posterior samples

Theoretical properties ensure that, under exact LRPS sampling, both adaptive and uniform- $K$ growth asymptotically recover unbiased evidence and posterior estimates as $K\to\infty$ or $M\to\infty$ (Buchner, 2023).

4. Implementation, Diagnostics, and Error Estimation

Efficient DNS implementations address two key challenges: (1) accurate and efficient LRPS and (2) robust convergence monitoring. Popular approaches for LRPS include multi-ellipsoid decomposition (Speagle, 2019), overlapping balls (Speagle, 2019), multivariate slice sampling (Buchner, 2023, Speagle, 2019), and Hamiltonian slice (Speagle, 2019). For high-dimensional or degenerate targets, Hamiltonian slice or adaptation-free multivariate slice sampling is preferred due to robustness and scaling properties.

Monitoring and error estimation utilize standard NS variance theory, augmented with the following:

Standard Error Estimates: Posterior and evidence error estimates are derived from the variance of log-volumes and resampling replicates, as in Skilling (2006) and its generalizations (Speagle, 2019, Higson et al., 2017).
Bootstrapped Volumes: Randomized shrinkage, e.g., $X_i \sim \text{Beta}(1, K)$ , allows bootstrap estimation of uncertainties (Buchner, 2021).
K-fold Subsampling: Removing root branches and re-estimating $Z$ for ensemble error bars (Buchner, 2021).
MCMC Diagnostics: Acceptance fraction and effective sample size (ESS) per LRPS chain (Buchner, 2023). If diagnostics indicate poor mixing, practitioners may pause snowballing and increase $M$ .
Insertion Rank Test: Statistical test (Wilcoxon–Mann–Whitney U or KS-test) on insertion ranks detects non-uniformity in LRPS, providing real-time guarantee of sample independence (Buchner, 2021).

A summary of recommended, data-based parameter choices is given below:

Parameter	Guideline
$K_0$ (init. live points)	$\max(10, d)$ ; check mode discovery
$M$ (MCMC steps)	$10\times d$
$\Delta K$ (SNS)	$\sim K_0$ per increment
$f$ (DNS, post/evidence)	$0.8$ for balanced runs

5. Theoretical and Practical Performance

DNS achieves superior efficiency by concentrating sampling effort in posterior-dominated regions or intervals where evidence error is largest. Theoretical error analysis shows that with adaptive live point allocation, the variance in log-evidence scales as

$\mathrm{Err}[\ln Z] \approx \sqrt{\sum_i \frac{\Delta H_i}{K_i}}$

where $K_i$ is the live point number in bin $i$ and $\Delta H_i$ the partial relative Shannon information (Speagle, 2019, Higson et al., 2017). When $K_i$ is increased in regions of large contribution to $\mathrm{Var}(Z)$ or posterior moments, the overall variance is minimized for a given computational budget (Higson et al., 2017).

Empirical results include:

Speedup factors up to $\sim 72$ for parameter estimation and $\sim 7$ for evidence in high-dimensional Gaussian-like models, compared to static NS (Higson et al., 2017).
In practical astronomy problems, order-of-magnitude improvements in effective sample size (ESS) per likelihood call over vanilla MCMC and static NS (Speagle, 2019).
SNS provides monotonic improvement in evidence and posterior accuracy as $K$ grows, enabling continuous refinement without the need for $K$ or $M$ pre-tuning (Buchner, 2023).

DNS achieves significant reductions in the total number of likelihood evaluations required for a target precision, typically halving or thirding the evaluation count for moderate dimensions and high-information problems (Buchner, 2021).

6. Comparison of Dynamic Nested Sampling Variants

DNS encompasses several algorithmic formulations. Notable examples include:

Dynamic Nested Sampling (Higson et al.): Adapts $K_i$ based on local importance, targeting evidence or posterior. Extensible to user-weighted objectives. Implemented in dynesty, dyPolyChord, and others (Higson et al., 2017, Speagle, 2019).
Snowballing Nested Sampling (SNS): Adds live points uniformly in batches, preserving standard NS diagnostics and error bars. No need for pre-tuned $K$ or $M$ . Easily overlays existing NS code. MCMC diagnostics guide adaptation of $M$ as needed (Buchner, 2023).
Diffusive Nested Sampling: Moves chains up and down in likelihood, accumulating samples across levels. Lacks built-in evidence error estimates (Buchner, 2023).

The main distinctions are summarized below:

Method	Allocation	Diagnostics	Error Bars	Suitability
DNS	Local, importance-based	Generalized (ESS, etc)	Variance, bootstrap	Posterior/evidence targeting
SNS	Uniform increments	Standard NS + MCMC	Classic NS	General, no pre-tuning, robust updates
Diffusive NS	Level-spanning chains	No $Z$ error bars	N/A	Model exploration, not $Z$ estimation

7. Implementation Guidance and Practical Considerations

DNS and SNS can be straightforwardly wrapped around or incorporated into existing NS codes with minimal overhead. Key points for successful application are:

Start with a small $K_0$ and moderate $M$ for rapid initial feedback; adapt as necessary based on convergence diagnostics.
For batch-incremental approaches, memoization of repeated LRPS calls efficiently reuses prior calculations (Buchner, 2023).
Always monitor chain mixing and ESS, especially as $K$ or $M$ are changed, to avoid bias in LRPS-derived samples.
SNS’s convergence can be monitored by stability of evidence estimates within Monte Carlo error bars; DNS by convergence of posterior/evidence stopping metrics.
Error estimation should involve both internal (bootstrap, volume resampling) and external (K-fold root branch removal) checks to ensure robust uncertainty quantification (Buchner, 2021).

While DNS approaches introduce more hyperparameters compared to static NS (e.g., batch size, importance weights, allocation strategy), their datadriven adjustability and extensibility enable practitioners to optimize performance for a wide array of Bayesian model inference tasks. DNS advances the field by allowing computational effort to be focused precisely where statistical uncertainty is greatest, rather than wasted in regions of negligible posterior or evidence contribution.