Nonparametric Bootstrap Methods

• Nonparametric bootstrap is a data-driven resampling technique that uses the empirical distribution to approximate the sampling distribution of a statistic.
• It provides estimators for standard errors, bias, and confidence intervals through methods like percentile, BCa, and double bootstrap to improve inferential accuracy.
• Extensions include adaptations for dependent data, nonstandard M-estimators, high-dimensional settings, and Bayesian frameworks, broadening its practical applications.

The nonparametric bootstrap is a ubiquitous, data-driven resampling technique for approximating the sampling distribution of estimators and test statistics without reliance on strong parametric modeling assumptions. Originating in Efron's seminal work, it has since found foundational and algorithmic extensions spanning inference for both IID and dependent data, smoothing-based function estimation, high-dimensional and functional settings, bias correction, and Bayesian nonparametrics.

1. Core Principles and Foundational Algorithm

The nonparametric bootstrap constructs a plug-in estimator for the law of a statistic $T_n = t(X_1, ..., X_n)$, where $(X_1,\ldots,X_n)$ is an IID sample from unknown distribution $F$. The empirical distribution $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$ replaces $F$, and bootstrap samples are generated by resampling $(X_1^*, ..., X_n^*)$ IID from $F_n$. For each replicate, the statistic $T_n^{*(b)} = t(X_1^{*(b)},...,X_n^{*(b)})$ is computed. The ensemble $\{T_n^{*(1)}, ..., T_n^{*(B)}\}$ forms the empirical bootstrap distribution (Ghysels et al., 12 Nov 2025, Zrimšek et al., 2024).

This general framework provides estimators for:

• Standard errors: $$\widehat{\mathrm{SE}}(T_n) = \sqrt{\frac{1}{B-1}\sum_{b=1}^B (T_n^{*(b)}-\bar{T}^*)^2}$$.
• Bias: $(X_1,\ldots,X_n)$0, with $(X_1,\ldots,X_n)$1.
• Confidence intervals via quantile (percentile), bias-corrected, or accelerated methods (see below).

Theoretical guarantees include first-order consistency for smooth functionals: under mild regularity, the conditional distribution of $(X_1,\ldots,X_n)$2 given data converges to that of $(X_1,\ldots,X_n)$3 (Ghysels et al., 12 Nov 2025, Zrimšek et al., 2024).

2. Construction of Bootstrap Confidence Intervals

Several variants are in common use (Zrimšek et al., 2024):

• Percentile intervals: $(X_1,\ldots,X_n)$4 empirical quantiles of the bootstrap $(X_1,\ldots,X_n)$5.
• Basic/reverse percentile: $(X_1,\ldots,X_n)$6.
• Bias-Corrected (BC): quantiles are adjusted for bias via the empirical fraction of $(X_1,\ldots,X_n)$7.
• Bias-Corrected and Accelerated (BCa): further corrects for skewness by a jackknife acceleration term.

The double bootstrap, involving an inner resampling loop within each outer bootstrap sample, yields nearly uniform nominal coverage for diverse functionals ($(X_1,\ldots,X_n)$8), outperforming single-stage intervals in simulations. It achieves this by re-calibrating percentile levels to account for persistent coverage deficiencies, especially in moderate sample sizes or for extreme quantiles (Zrimšek et al., 2024).

For central quantiles, the percentile and BCa methods have comparably good performance for smooth statistics, though coverage deteriorates in the tails for small $(X_1,\ldots,X_n)$9 and highly irregular functionals (Ghysels et al., 12 Nov 2025).

3. Advanced Extensions for Dependent and Structured Data

For time series and dependent data instances, classical bootstrap resampling fails to respect serial correlation, leading to invalid inference (Chandy et al., 7 Nov 2025, Chwialkowski et al., 2014). To address this:

• Block Bootstraps: Observed data are partitioned into overlapping or nonoverlapping blocks of length $F$0; samples are constructed by concatenating randomly drawn blocks (possibly wrapped around the series). For Kolmogorov–Smirnov test statistics with unknown marginal parameters, the block bootstrap must incorporate:
  • Parameter re-estimation within each replicate.
  • A bias-correction centering step: the raw empirical process is recentered by subtracting the bootstrap mean and correcting for the difference in parameter estimates (Chandy et al., 7 Nov 2025).
  • Canonical block length $F$1 suffices for moderate dependence; cross-validation or plug-in rules may degrade under strong dependence.
• Wild Bootstrap for $F$2-Statistics: For kernel-based independence/MMD tests on temporally dependent data, auxiliary random multipliers $F$3 are used, carefully constructed to mimic the limiting law under weak dependence. This construction avoids the severe size distortion seen under permutation or IID bootstrapping and achieves correct Type I error for degenerate kernel statistics (Chwialkowski et al., 2014).

4. Smoothing, Bandwidth Selection, and Function Estimation

For nonparametric function estimation (e.g., regression, density, Sobol index estimation), bias of the estimator is typically the limiting factor for valid inference. The naive bootstrap reproduces this bias inconsistently unless care is taken (Cheng et al., 2017, Groeneboom et al., 2023, Hall et al., 2013, Solís, 2018):

• Debiased Bootstrap Bands: Confidence bands for a function $F$4 are constructed by bootstrapping a bias-corrected estimator (e.g., debiased kernel density or local-polynomial regression), with only standard bandwidth selection needed. This approach gives asymptotically honest simultaneous bands without undersmoothing or special pilot corrections (Cheng et al., 2017).
• Smoothed Bootstrap for Monotone Regression: For isotonic regression, the ordinary bootstrap is inconsistent. The smoothed least squares estimator (SLSE) involves smoothing the step-wise LSE by convolution with a kernel and constructing bootstrap samples using oversmoothing; percentile or Studentized intervals are built from repeated bootstrap SLSEs (Groeneboom et al., 2023).
• Bootstrap Bandwidth Selection: For kernel-based estimation of Sobol indices, cross-validated bandwidths tend to have structural bias in finite samples. Bootstrap-residual approaches reconstruct the residual distribution, simulate new data, and select the bandwidth via a re-minimized Bootstrap Least Squares criterion; this corrects finite-sample bias and produces more accurate sensitivity estimates (Solís, 2018).
• Quantile Calibration for Pointwise Confidence Bands: The Hall–Horowitz method constructs bands by using quantiles of the coverage distribution across $F$5 to automatically adjust for inconsistent bootstrap bias estimation. This yields bands of correct coverage over the majority of the domain, circumventing the traditional need for undersmoothing (Hall et al., 2013).

5. Limiting Validity, Nonstandard M-Estimators, and High-Dimensional Variants

Nonstandard (Cube-Root) Rates and Shape Correction

For M-estimators with cube-root asymptotics (e.g., isotonic MLE, maximum-score, current status), the naive bootstrap is inconsistent: it fails to reproduce the non-Gaussian (Chernoff-type) limiting distribution due to a misalignment of the local quadratic drift (Cattaneo et al., 2017, Groeneboom et al., 2017). This has been established for current status MLEs and related problems. Nonetheless:

• $F$6 convergence rates for the bootstrap remain correct.
• For smooth functionals of the MLE (Hadamard differentiable), bootstrap distribution is asymptotically valid.
• Shape-corrected bootstraps: modifying the criterion function in each replicate to match the population quadratic drift restores consistency for the limiting law (Cattaneo et al., 2017).

High-Dimensional Statistics

In settings with high-dimensional sample covariance matrices ($F$7), the ordinary bootstrap does not preserve spectral properties. The proposed “$F$8 out of $F$9” bootstrap achieves spectral distributional consistency as $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$0, and correctly approximates the distribution of linear spectral statistics under suitable conditions. Representative subpopulation conditions on the data and specific resampling/projection schemes are crucial for validity (Dette et al., 2024).

6. Bayesian and Empirical-Bayesian Bootstrap

The Bayesian bootstrap arises as the noninformative Dirichlet process posterior on the empirical support, where the weights on the observed data are Dirichlet$$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$1, generated via normalized exponentials. For smooth functionals, five first-order approaches (Bayesian, Bayesian bootstrap, Rubin's degenerate-prior, Efron's classical bootstrap, and delta-method) are asymptotically equivalent; all yield the same Gaussian limiting law for standard estimators (Hjort, 12 May 2026).

The empirical-Bayes bootstrap estimates the concentration parameter in the Dirichlet prior using data and generates non-uniform weights. This augments the procedure for small $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$2, heavy tails, or when prior structure is known. Bayesian and empirical-Bayesian bootstrap methods generalize to semiparametric regression, censored data, and hazard models (Hjort, 12 May 2026).

7. Specializations, Limitations, and Empirical Applications

• Multivariate Extreme Value Inference: The spectral bootstrap for multivariate generalized Pareto variables preserves joint tail dependence by resampling empirical spectral angles and generating radii from the exponential marginal law. Unlike naive resampling, this facilitates robust estimation of tail risk metrics (e.g., multivariate expected shortfall) in high dimensions (Madhar et al., 30 Sep 2025).
• Functionals and Extreme Tails: Bootstrap percentile intervals are first-order valid in central regions and moderate deviations, but may misrepresent variability in extreme tails. Studentization, Edgeworth correction, and $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$3-out-of-$$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$4 methods can improve performance in such regimes (Ghysels et al., 12 Nov 2025).
• Quantum Inference: Nonparametric bootstrap distributions for classical shadows exhibit heavy tails and skewness absent from Gaussian/information-theoretic bounds. Resampling provides empirically sharper quantifications of risk (Expectation-Value-at-Risk, Expected Shortfall) (Ghysels et al., 12 Nov 2025).

Typical practical guidelines derived from simulation studies and theoretical considerations (Zrimšek et al., 2024):

• For independent data and standard functionals, double bootstrap or BCa intervals yield uniformly accurate CIs with $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$5.
• For dependent or time series data, use block bootstraps with $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$6; always center and bias-correct the empirical process.
• In nonstandard, non-smooth, or shape-constrained models, supplement bootstrapping with smoothing or shaped objective correction.

Table: Bootstrap Strategies for Inferential Settings

Setting	Standard Bootstrap	Issue or Adaptation
IID/smooth statistics	Percentile, BCa, Double Bootstrap	Uniformly valid; double bootstrap preferred (Zrimšek et al., 2024)
Dependent/serial data	Ordinary resampling	Invalid; use block bootstrap, bias correction (Chandy et al., 7 Nov 2025, Chwialkowski et al., 2014)
Shape constraints (isotonic, MLEs)	Nonparametric LSE, MLE	Inconsistent law; use smoothed estimator, shape-corrected bootstrap (Cattaneo et al., 2017, Groeneboom et al., 2023)
High-dimensional covariance	Ordinary resampling	Fails; use $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$7 out of $$F_n(x) = n^{-1} \sum_{i=1}^n \mathbf{1}\{X_i \le x\}$$8, Representative Subpopulation Condition (Dette et al., 2024)
Bayesian nonparametric	Bayesian/Empirical Bayes BB	Dirichlet-weighted empirical support (Hjort, 12 May 2026)

• Nonparametric block bootstrap for serial dependence: (Chandy et al., 7 Nov 2025)
• Bootstrap for classical/shadow quantum inference: (Ghysels et al., 12 Nov 2025)
• Bootstrap confidence intervals: (Zrimšek et al., 2024)
• Debiased estimator bootstrap/inference bands: (Cheng et al., 2017)
• Bootstrap bandwidth selection for Sobol indices: (Solís, 2018)
• Wild bootstrap for degenerate kernel tests: (Chwialkowski et al., 2014)
• Shape-corrected bootstrap for cube-root problems: (Cattaneo et al., 2017)
• High-dimensional covariance bootstrap: (Dette et al., 2024)
• Monotone regression—SMLE/smoothed boot: (Groeneboom et al., 2023)
• Bias/coverage-quantile bootstrap for bands: (Hall et al., 2013)
• Bootstrap for current status model: (Groeneboom et al., 2017)
• Bayesian/empirical Bayesian bootstrap: (Hjort, 12 May 2026)
• Spectral bootstrap for multivariate extremes: (Madhar et al., 30 Sep 2025)

These works codify the nonparametric bootstrap as a pivotal inferential tool, motivating continuous refinement of resampling logic, smoothing, and centering to adapt to complex data-generating regimes and estimator properties.