Bootstrap Confidence Intervals

• Bootstrap confidence intervals are nonparametric resampling techniques that estimate uncertainty in parameters by repeatedly sampling from the empirical distribution.
• They encompass methods like Normal, Studentized, Percentile, BCa, and Bayesian, each with distinct tradeoffs in coverage, interval width, and computational demands.
• Empirical studies show that method choice depends on sample size and data characteristics, with studentized intervals often providing superior coverage for challenging scenarios.

A bootstrap confidence interval (CI) is a nonparametric inferential technique that estimates uncertainty about a parameter—such as a mean or quantile—using repeated re-sampling of data and the empirical distribution of an estimator. The bootstrap formalism encompasses a diverse suite of procedures, each with distinct theoretical properties and practical tradeoffs regarding coverage probability, interval width, and computational complexity. The methodology is applicable both in classical i.i.d. settings and a variety of dependent-data regimes, and extends from estimating simple statistics to multivariate parameters in structured models. Recent simulation studies have systematically compared major bootstrap CI paradigms, elucidated boundary effects, and provided comprehensive practical guidance for applied research (Justus et al., 2024).

1. Major Classes and Mathematical Formulation

The principal bootstrap CI methods differ in how they quantify estimator variability and adjust for bias:

• Normal (Basic z) Bootstrap CI: This relies on the empirical bootstrap estimate of standard error. For estimator $\hat\theta$ and $B$ bootstrap replicates $\hat\theta^{(b)}$, the standard error is

$$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$

and the CI is

$$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$

• Studentized Bootstrap CI (Bootstrap-t): For each bootstrap sample, one computes a studentized pivot,

$$t^{(b)} = \frac{ \hat{\theta}^{(b)} - \hat{\theta} }{ \hat{SE}(\hat{\theta}^{(b)}) },$$

where the denominator is estimated by a secondary (nested) bootstrap. The CI comprises

$$\bigl( \hat\theta - t_{1-\alpha/2}^* \hat{SE}(\hat\theta),\; \hat\theta - t_{\alpha/2}^* \hat{SE}(\hat\theta) \bigr),$$

using bootstrap quantiles $t_{p}^*$ (Justus et al., 2024).

• Quantile (Percentile) Bootstrap CI: This method directly inverts the empirical distribution of $\hat{\theta}^{(b)}$:

$$(\hat{\theta}_{\alpha/2},\, \hat{\theta}_{1-\alpha/2}),$$

where $B$0 denotes the $B$1-th empirical quantile among the bootstrap replicates.

• Bias-Corrected and Accelerated (BCa) CI: Accounts for both bias and skewness via a bias-correction term $B$2 and an acceleration parameter $B$3, estimated via jackknife replicates, shifting the percentile cutoffs to

$B$4

The CI endpoints are given by the corresponding quantiles of the bootstrap distribution (Justus et al., 2024).

• Bayesian Bootstrap CI: Constructs “credible intervals” using random Dirichlet weights on observed data, yielding quantile-based intervals for estimators constructed as weighted functionals (Justus et al., 2024).

Each approach applies to both scalar and vector-valued estimands, with extensions available for constrained, high-dimensional, and dependent data settings (Justus et al., 2024, 1706.02150, Cho et al., 2021).

2. Simulation-Based Comparisons and Empirical Findings

In extensive Monte Carlo designs—spanning diverse underlying distributions (normal, exponential, Laplace, uniform, student-t), sample sizes ($B$5), and weak to moderate autocorrelation (induced via Farlie-Gumbel-Morgenstern copulas, Spearman $B$6 in $B$7)—the performance of the five principal bootstrap CI variants was systematically benchmarked (Justus et al., 2024). Key findings include:

• Coverage: Studentized bootstrap CIs consistently yield coverage rates closest to the nominal level (typically $B$8), particularly for small $B$9 and heavy-tailed data. Other procedures (normal, quantile, BCa, Bayesian) perform comparably, with Bayesian intervals exhibiting mild undercoverage in many settings.
• Interval Length: Studentized intervals tend to be the longest—sometimes by $\hat\theta^{(b)}$0–$\hat\theta^{(b)}$1—and can have heavy right tails (especially when estimating $\hat\theta^{(b)}$2 or under heavy tails). Bayesian and BCa intervals are shortest, with quantile and normal in between.
• Combined Performance Indicator $\hat\theta^{(b)}$3: To evaluate methods on a tradeoff frontier, $\hat\theta^{(b)}$4 was employed (with $\hat\theta^{(b)}$5 the empirical coverage rate, $\hat\theta^{(b)}$6 the median or mean interval length). All methods score similarly in the aggregate for location estimation; for scale parameters, studentized intervals lead. Using mean length, BCa and Bayesian CIs dominate, reflecting the effect of long outlier intervals for studentized methods (Justus et al., 2024).
• Dependence: All methods are robust to mild autocorrelation ($\hat\theta^{(b)}$7), though positive dependence reduces coverage uniformly across approaches.

In summary, while studentized intervals usually maximize coverage, BCa and Bayesian methods offer more concise inference at a small cost to coverage, and ordinary percentile or normal CIs become increasingly adequate as $\hat\theta^{(b)}$8 grows (Justus et al., 2024). The overall balance depends on whether accurate coverage, minimal length, or computational efficiency is prioritized by the investigator.

3. Method Selection, Limitations, and Best Practices

The practical implications and caveats of method choice are critically informed by simulation and theoretical evidence (Justus et al., 2024):

• Studentized CI: Recommended when priority is placed on coverage, especially with small samples or heavy-tailed data, at the expense of longer and sometimes highly variable intervals, plus intensive computation due to nested resampling.
• Bayesian and BCa CI: Deliver the shortest intervals—with BCa typically producing a more desirable balance between interval width and coverage. Downside: small but systematic undercoverage is observed, especially in the Bayesian method due to its restriction to observed support in continuous models or the instability of BCa acceleration in small or discrete samples.
• Normal and Percentile CI: For location estimation in moderate to large samples ($\hat\theta^{(b)}$9), these methods are computationally frugal and perform adequately in both coverage and length.
• Autocorrelation and Dependence: None of the core methods are robust to strong dependence; for substantial autocorrelation or time series, block or stationary bootstraps are required.
• Failure Modes: Studentized CIs can break when estimated standard errors from the inner bootstrap are near zero, leading to unbounded $$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$0 statistics and unusably wide intervals.
• No Uniformly Best Method: There is no universally superior bootstrap CI; the optimal choice depends on tolerance for undercoverage, interpretability of interval width, computational burden, and the empirical context (Justus et al., 2024).

4. Finite-Sample Effects and Coverage Stability

Recent exact finite-sample analyses have demonstrated that naive nominal-level percentile or basic bootstrap intervals often fail to attain their stated coverage, sometimes dramatically so. In particular:

• For estimators such as the sample mean from normal data, neither the parametric nor nonparametric bootstrap at finite $$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$1 achieves nominal coverage $$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$2; with $$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$3 and number of bootstrap replicates $$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$4 too small, coverage can be strictly sub-nominal or, in some cases, degenerate (Wang et al., 2024).
• In some scenarios, the expected length of bootstrap intervals can be anomalously small—narrower than the theoretically optimal $$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$5-interval—giving a false impression of efficiency.

Recommendations emphasize recalibrating the bootstrap quantiles by simulation or exact formula, or preferring conservative exact procedures (Clopper-Pearson, BCa, studentized bootstrap-$$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$6) in small or moderate samples. When actual (finite-$$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$7) coverage is critical to decision-making, exact or high-precision simulation becomes essential (Wang et al., 2024).

5. Extensions: Advanced and Structured Bootstrap CIs

Bootstrap CIs have been extensively generalized:

• Smoothed and Isotonic Settings: For models lacking standard asymptotic pivots (e.g., isotonic regression, current status models), smoothed or boundary-corrected bootstrap approaches yield accurate CIs with nonclassical convergence rates and improved behavior at boundaries (Groeneboom et al., 2016, Sen et al., 2012, Groeneboom et al., 2023).
• High-Dimensional and Penalized Estimation: Bootstrap methods tailored for Lasso and sparse estimation (e.g., Bootstrap Lasso+Partial Ridge) provide valid inference under relaxed sparsity regimes (“cliff-weak” sparsity), outperforming both naive bootstrap lasso (which requires a strong beta-min assumption) and de-sparsified methods (which yield unnecessarily long intervals) (1706.02150).
• Change-Point and Dependent Data: Resampling block or segment-based bootstraps enable valid uncertainty quantification in multiple change-point detection, even under nonstandard limiting distributions for the estimator (Cho et al., 2021).
• Weighted/Dirichlet/Bayesian Bootstraps: Fractional-random-weight and Bayesian bootstraps are essential when resampling may exclude key instances (e.g., heavy censoring, rare event logistic regression), preserving estimator existence and inferential validity (Gotwalt et al., 2018).
• Coverage in Constrained Space: Bootstrap percentile intervals can over- or under-cover in the presence of parameter-space boundaries, notably exceeding nominal coverage at boundaries and potentially displaying undercoverage in certain two-sample ordered parameter scenarios (Wang et al., 2017).
• Small-Sample Weighted Bootstrap-t: Choosing continuous positive weights (e.g., Beta$$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$8) for the bootstrap-t statistic ensures second-order accuracy and finiteness, delivering stable inference when traditional bootstrap-t or BCa intervals may be infinite or under-cover, especially on discrete or highly tied data (Owen, 13 Aug 2025).

6. Implementation and Metrics for Evaluation

Empirical comparisons of bootstrap CIs are generally assessed by the following metrics:

Metric	Definition	Role in Evaluation
Coverage Rate ($$\hat{SE}(\hat\theta) = \sqrt{ \frac{1}{B-1} \sum_{b=1}^B (\hat{\theta}^{(b)} - \bar{\hat{\theta}} )^2}$$9)	Proportion of CIs containing the true parameter	Measures validity
Median Length ($$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$0)	Median CI length across replicates	Measures informativeness
Combined Indicator ($$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$1)	$$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$2 using $$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$3 as median or mean	Captures tradeoff; scalarized

The combined indicator $$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$4 is designed to capture the Pareto tradeoff between high coverage and minimal length, but is not a universal utility and has no canonical setting, echoing the absence of a generally accepted loss function for confidence intervals (Justus et al., 2024, Owen, 13 Aug 2025).

7. Summary Table of Bootstrap CI Methods (Justus et al., 2024)

Method	Coverage	Median Length	Computation	Noteworthy Properties
Studentized	Highest	Longest	High (nested)	Robust to small $$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$5, heavy tails
BCa	Moderate	Short	Moderate	Bias & skew correction
Bayesian	Slightly low	Shortest	Moderate (weights)	Undercoverage in continuous data
Normal ($$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$6)	Mod-high	Moderate	Low	Simple, suffices for large $$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$7
Percentile	Mod-high	Moderate	Low	Simple, widely used

In large samples, differences among methods vanish and computationally simple normal or percentile CIs are typically adequate. For moderate or small $$\bigl( \hat\theta - z_{1-\alpha/2}\hat{SE}(\hat\theta),\; \hat\theta + z_{1-\alpha/2}\hat{SE}(\hat\theta) \bigr).$$8, or heavy-tailed data, studentized and BCa/Bayesian alternatives warrant consideration adjusted to the inferential goal.

---

For authoritative methodological details, simulation results, caveats, and practical recommendations, see Justus, Rodrigues, and Sousa, "Bootstrap confidence intervals: A comparative simulation study" (Justus et al., 2024).