Statistical Uncertainty in Nested Sampling

• The topic is defined as the inherent stochastic error from finite live points in nested sampling, affecting the accuracy of Bayesian evidence approximations.
• Leading analytical expressions, such as Skilling’s formula, highlight how error scales with the number of live points and the complexity of the likelihood function.
• Diagnostic strategies and rectification techniques—including dynamic nested sampling and post-hoc field-based methods—help mitigate systematic biases and ensure robust uncertainty quantification.

Statistical uncertainty in nested sampling (NS) refers to the inherent stochastic error arising from the algorithm’s use of a finite number of live points to approximate likelihood-weighted integrals. NS is widely employed for Bayesian evidence estimation and posterior inference, but its accuracy is inextricably tied to the interplay of stochastic volume contractions, systematic effects from algorithm settings, and the geometry and properties of the target problem.

1. Fundamental Framework of Statistical Uncertainty in Nested Sampling

Nested sampling estimates the Bayesian evidence,

$Z = \int L(\theta)\,\pi(\theta)\,d\theta$

by transforming a high-dimensional integral into a one-dimensional integral over prior mass $X$, with $X(L^*) = \int_{L(\theta)>L^*}\pi(\theta)\,d\theta$. The likelihood as a function of $X$ yields

$Z = \int_0^1 L(X)\,dX.$

At iteration $i$, the remaining prior mass is $X_i \approx \exp(-i/N_{\rm live})$, where $N_{\rm live}$ is the number of live points. The integral is approximated stochastically using weights $\Delta X_i = X_{i-1} - X_i$, yielding the evidence estimator:

$Z \approx \sum_{i=1}^M L_i\,\Delta X_i$

where $L_i$ is the likelihood of the $i$th dead point. The shrinkage factors $t_i = X_i/X_{i-1}$ are i.i.d. draws from the $\operatorname{Beta}(N_{\rm live},1)$ distribution, and their stochasticity is the primary intrinsic source of statistical uncertainty (Dittmann, 2024, Fowlie et al., 2022, Keeton, 2011, Latz et al., 2023, Westerkamp et al., 2024).

2. Leading-Order Analytic Expressions for Uncertainty

The dominant source of statistical error in $\ln Z$ arises from randomness in the shrinkages $t_i$, which control how quickly the sequence of $X_i$ approaches the posterior bulk. The canonical formula for the standard error in $\ln Z$ is (Skilling 2004):

$$\sigma[\ln Z] \approx \sqrt{\frac{H}{N_{\rm live}}}$$

with $H$ the information gain (Kullback–Leibler divergence) between the posterior and prior:

$$H = \int p(\theta) \log\left[\frac{p(\theta)}{\pi(\theta)}\right] d\theta,$$

where $p(\theta) = L(\theta)\,\pi(\theta)/Z$. In a run, $H$ is estimated by

$$\hat{H} \approx \sum_{i} \frac{w_i L_i}{Z} \log\left(\frac{L_i}{Z}\right),\quad w_i=\Delta X_i.$$

Both Skilling’s entropic approach and Keeton’s moment-based expansion yield, in the large-$N_{\rm live}$ regime, the same leading term in $N_{\rm live}^{-1}$ for $\operatorname{Var}[\ln Z]$ (Fowlie et al., 2022, Keeton, 2011).

Empirical and analytic results show that as $N_{\rm live}$ increases, the statistical uncertainty in $\ln Z$ decreases as $N_{\rm live}^{-1/2}$, and halving the error requires quadrupling the number of live points (Dittmann, 2024, Buchner, 2021, Pfeifenberger et al., 2016).

3. Empirical Characterization and Hyperparameter Dependence

Systematic investigation using analytically tractable distributions (e.g., multivariate Gaussian, Rosenbrock, log-Gamma) reveals pronounced dependence of statistical errors and bias on both $N_{\rm live}$ and the “sampling efficiency” hyperparameter $\epsilon$ in MultiNest-style algorithms:

• Evidence bias increases with $\epsilon$ and dimension $D$: For $D = 10,\,N_{\rm live}=300,\,\epsilon=0.3$, the mean evidence bias is $\Delta\ln Z \approx 1.35\pm0.26$; for $D=20$, $\Delta\ln Z \gtrsim 5$ (Dittmann, 2024).
• Posterior errors in $L_1$ distance scale as $N_{\rm live}^{-1/2}$ and are largely insensitive to $\epsilon$ for $\epsilon \le 1$ when modes are found.
• Non-ellipsoidal (Rosenbrock) and heavy-tailed (log-Gamma) targets require much lower $\epsilon$ to avoid bias.
• Noisy likelihoods induce systematic underestimation of posterior credible intervals independent of $N_{\rm live}$ unless $\epsilon$ is reduced well below default values.
• For highly ill-conditioned or jagged likelihoods, MultiNest’s systematic bias can dominate over stochastic uncertainty by orders of magnitude, especially at large $D$ (Dittmann, 2024).

4. Parameter Estimation: Distinct Sources of Uncertainty

For parameter estimation, two distinct contributions to statistical error exist (Higson et al., 2017):

1. Uncertainty in weight assignment ($\Delta X_i$ and posterior weights $p_i$) due to random shrinkages.
2. Within-shell error: Using a single sample to represent the full contour-average for each iso-likelihood shell induces an additional variance unique to parameter estimation.

This dual-origin variance means that the “simulated-weights” approach (using only random $t_i$) underestimates the true error, whereas bootstrap methods over thread-split runs capture both terms. In practice, thread-splitting and bootstrap over single runs provide accurate uncertainty quantification for posterior expectations (Higson et al., 2017).

5. Diagnostic and Practical Strategies for Quantifying and Controlling Uncertainty

Rigorous diagnostics are essential:

• Vary $N_{\rm live}$ and $\epsilon$ over factors $\gtrsim 3$ and demand stability of $\Delta\ln Z$ and posterior summaries.
• Compare empirical scatter in $\ln Z$ from independent runs to the analytic $\sigma[\ln Z]$; excess scatter signals non-convergence.
• Check run-to-run $L_1$ distances and credible interval widths: order-unity variation indicates breakdown of convergence assumptions (Dittmann, 2024).

Empirically validated approaches for error estimation:

• Moment-based estimators, Skilling’s $\sqrt{H/N_{\rm live}}$ formula, bootstrap over shrinkage sequences, and multiple-run ensemble variance all yield comparable results in well-behaved cases (Keeton, 2011, Latz et al., 2023, Fowlie et al., 2022).
• Dynamic nested sampling can allocate $N_{\rm live}$ adaptively to high-importance regions, reducing total uncertainty efficiently (Buchner, 2021, Latz et al., 2023).
• Field-based Bayesian inference (Information Field Theory) for volume estimation as a post-processing step can reduce error in $\ln Z$ by $30$–$50\%$ for small $N_{\rm live}$, provided the likelihood–prior–volume curve is smooth and monotonic, but can introduce bias or instability if monotonicity is violated or $N_{\rm live}\gg 10^2$ (Westerkamp et al., 2024, Westerkamp et al., 2023).

6. Systematics, Pathological Cases, and Limits of Applicability

Heavy tails, multimodal distributions, and noise can invalidate classical error formulas:

• For targets with heavy-tailed $p(y=-\ln X)$, both Skilling and Keeton uncertainties can diverge, and empirical variance in $\ln Z$ continues to grow with the number of dead points unless manual truncation is imposed (Fowlie et al., 2022).
• The moment-propagation approach remains robust in moderately pathological cases where Skilling’s information diverges.
• Strongly jagged or multimodal $L(X)$, or high-frequency noise, can lead to credible intervals or evidences that are systematically too narrow, even when analytic statistical uncertainty appears small (Dittmann, 2024).
• Diagnostic checks—effective sample size, posterior bulk moments, and stability over run repetitions—are critical for ensuring the validity of reported uncertainty.

7. Rectification and Error Reduction Techniques

When NS runs produce under-resolved or biased results, several rectification strategies are effective:

• MCMC warm start: Initialize parallel-walker MCMC chains (e.g., emcee) from a KDE fit to MultiNest/posterior samples; posterior bias can “burn out” after moderate MCMC burn-in (Dittmann, 2024).
• Warm-starting more robust nested samplers: Injecting MultiNest live points into codes using slice sampling (UltraNest, dynesty, PolyChord) allows recovery of unbiased evidences and credible regions, with empirical tests showing accurate rectification even in challenging cases (Dittmann, 2024).
• Post-hoc field-based volume inference: Applying GP/IFT-based reconstruction of $L(X)$ as a function allows improved posterior credible intervals and reduced error in $\ln Z$ at fixed $N_{\rm live}$ (Westerkamp et al., 2023, Westerkamp et al., 2024). Performance deteriorates for large $N_{\rm live}$ or in non-smooth settings.

Summary Table: Core Results on NS Statistical Uncertainty

Key Concept	Formula / Criterion	Reference
Leading-order evidence error	$\sigma[\ln Z]\approx\sqrt{\frac{H}{N_{\rm live}}}$	(Dittmann, 2024, Keeton, 2011)
Empirical $L_1$ posterior error	$L_1 \sim N_{\rm live}^{-1/2}$	(Dittmann, 2024)
Failure in high $D$, large $\epsilon$	Evidence bias $>\!\!1$ in $\ln Z$	(Dittmann, 2024)
Noise-induced posterior width bias	Up to $8\%$ underestimation at $\epsilon=0.3$	(Dittmann, 2024)
Field-theoretic post-processing uncertainty	Empirically $30$–$50\%$ lower $\sigma[\ln Z]$	(Westerkamp et al., 2023, Westerkamp et al., 2024)
Pathology (divergent $\operatorname{Var}[\ln Z]$)	Occurs for heavy-tailed $p(y)$ or infinite $KL$	(Fowlie et al., 2022)

Stringent practice demands varying $N_{\mathrm{live}}$ and algorithmic hyperparameters, validating analytic/empirical error estimates versus run-to-run scatter, and deploying diagnostics for posterior stability. When convergence is in doubt or systematic errors dominate, rapid but potentially biased NS outputs should be used mainly to initialize more robust inference frameworks or to supply field-theoretically regularized volume curves, leveraging the speed of NS while managing its limitations effectively (Dittmann, 2024, Westerkamp et al., 2023, Westerkamp et al., 2024).