Boot-Both Resampling Methods

• Boot-Both Resampling is a set of techniques that merge residual, weighted, and subsample bootstraps to improve model robustness and reliability.
• It simultaneously perturbs data axes to capture multiple sources of variability, leading to enhanced convergence and uncertainty quantification.
• Empirical studies demonstrate that Boot-Both methods outperform conventional bootstraps in handling heteroscedastic, high-dimensional, and dependent data.

“Boot-Both Resampling” encompasses a family of resampling methodologies that combine multiple bootstrap-related techniques, such as weighted/residual, parametric/nonparametric, or system/input dual resamplings, to improve statistical inference, stability, and uncertainty quantification across diverse applications. These methods have become central in modern statistics and machine learning, addressing both computational challenges and methodological limitations arising in high-dimensional, dependent, noisy, or structurally complex data scenarios. The following sections detail core technical developments, representative algorithms, and empirical performance of “Boot-Both Resampling” as established in recent research.

1. Formal Definitions and Resampling Paradigms

The “Boot-Both” paradigm refers to resampling schemes that simultaneously perturb multiple axes or dimensions in the data or methodology. This includes:

• Residual and Weighted Bootstrap: Applied in time series models, where residual bootstrapping resamples estimated innovations, while weighted bootstrap assigns random multipliers to observations (often with moment and exchangeability constraints) to better approximate heteroscedasticity and heavy-tailed phenomena (Bhattacharya et al., 2012).
• Subsampled Double Bootstrap (SDB): For massive data, SDB replaces classical bootstrap and Bag-of-Little-Bootstraps (BLB) by drawing many random subsets, resampling each only once, and aggregating roots (e.g., estimator differences), efficiently approximating the true sampling distribution both for i.i.d. and dependent (time series) settings (Sengupta et al., 2015).
• Boot-Both in Evaluation Metrics: In metric reliability studies, “Boot-Both” signifies simultaneous bootstrapping of both systems and input data, forming a joint empirical distribution of correlations or statistics (see pseudocode in (Deutsch et al., 2021)).

These methods generalize the classic bootstrap by intentionally incorporating multiple sources of variability and provide more robust inferential guarantees under model misspecification, dependency, and computational constraints.

2. Theoretical Guarantees and Asymptotic Properties

Boot-Both strategies are theoretically motivated to address cases where conventional resampling fails.

• Consistency and Convergence: Weighted bootstrap schemes for AR and ARCH models are shown to produce estimators whose resampled distribution converges to the same asymptotic law as the original estimator, even under strong heteroscedasticity or heavy-tailed errors. The core result is

$$P_B\left\{\sqrt{n}\sigma_n^{-1}\left(\hat{\theta}^* - \hat{\theta}\right) \leq x \mid \text{data}\right\} - P\left\{\sqrt{n}\left(\hat{\theta} - \theta\right) \leq x\right\} \rightarrow_p 0,$$

with technical conditions on the multiplier sequence (e.g., mean, variance, exchangeability) (Bhattacharya et al., 2012).

• Gaussian Process Limits: In SDB, the limiting empirical process given by

$$\hat{G}_{n,(b)}^B(f) = \frac{1}{\sqrt{n}} \sum_{i=1}^b (W_{i,n} - n/b) f(X_{R^{-1}(i)})$$

is proven to converge to the same centered Gaussian process as the classical bootstrap, under Donsker and mixing conditions (Sengupta et al., 2015).

• Finite Sample Stability: Bagging techniques, viewed as Boot-Both in the context of multiple resampled bags, yield explicit finite-sample stability guarantees bounded by

$$\beta^2 := \frac{rad_\mathcal{H}^2(\mathcal{W})}{n-1}\cdot p_{\mathcal{Q}_n}(1-p_{\mathcal{Q}_n}),$$

for a Hilbert-space output (Soloff et al., 15 May 2024).

A plausible implication is that Boot-Both resampling mitigates the degeneracies caused by dependency, small-sample bias, or high-dimensional interpolation phenomena.

3. Algorithmic Implementations and Modular Schemes

The implementation of Boot-Both resampling schemes varies according to the model and data structure:

• Weighted Bootstrap in Time Series: Assign i.i.d. weights $w_{nt}$ (mean 1, $Var(w_{nt}) = o(n)$), forming weighted sum statistics or objective functions $Q_{nB}(\theta)$, then solve for the estimator as a function of weighted data (Bhattacharya et al., 2012).
• SDB for Massive Data: Randomly draw subsets $\mathcal{I}_s$ of size $b \ll n$, compute $\hat{\theta}_s$, resample (weighted multinomial or blockwise for time series) to full size $n$, obtain $\hat{\theta}_s^*$, and repeat for many $s$ to construct empirical quantile or confidence region estimates (Sengupta et al., 2015).
• Dual-Axis Bootstrapping for Metrics: For a matrix $M$ of scores, sample rows (systems) and columns (inputs) with replacement, compute the statistic (e.g., correlation), and repeat to estimate the distribution and intervals (Algorithm 1 in (Deutsch et al., 2021)).
• Cluster and Block Bootstrap in Mixed Effects Models: Resample at cluster or case level, optionally with parametric or residual block adjustments, reflate BLUPs and residuals for unbiased estimation, parallelized via helper utilities (Loy et al., 2021).

A summary of implemented schemes:

Context	Resampling Axis 1	Resampling Axis 2
Time series models	Residuals	Weights/multipliers
Metric evaluation	Inputs/documents	Systems/models
Clustered data	Clusters	Cases (within cluster)
Massive data (SDB)	Subsets	Full sample upsampling

4. Empirical and Simulation Results

Empirical evaluation demonstrates that Boot-Both techniques systematically outperform conventional bootstrapping or single-axis resampling under key conditions:

• In AR models with sharp heteroscedastic effects, weighted bootstrap distributions closely track the true estimator distribution, while standard residual bootstrap shows severe tail discrepancies (Bhattacharya et al., 2012).
• In massive data scenarios, SDB achieves lower joint error rates and faster convergence to confidence interval quantiles than BLB and classical bootstrap, due to higher data coverage and computational efficiency (Sengupta et al., 2015).
• Boot-Both resampling for summarization metrics produces confidence intervals with coverage much closer to the nominal rate compared to single-axis methods, and Perm-Both permutation procedures achieve higher test power (Deutsch et al., 2021).
• Bagging (averaged resampling) yields several orders of magnitude reduction in mean-squared perturbation for regression, synthetic control, and function estimation tasks, stabilizing outputs in both Hilbert and Banach spaces (Soloff et al., 15 May 2024).
• In optimization under noise, adaptive Boot-Both resampling based on bootstrap probability of dominance dynamically allocates evaluation resources and outperforms static/dynamic baselines across a range of noise levels (Budszuhn et al., 27 Mar 2025).

5. Limitations and Contextual Dependencies

Although Boot-Both methods offer manifold improvements, several limitations and caveats have been observed:

• High-dimensional regimes: As $n$ and $d$ grow with fixed $\alpha = n/d$, Boot-Both resampling schemes (bootstrap, subsampling, jackknife) exhibit “double-descent”-like behavior. Reliable uncertainty estimates are only attainable for large $\alpha$ (sample-rich scenarios); in the over-parameterized regime ($\alpha < 1$, common in deep learning), resampling-based predictions are inconsistent—even with optimal regularization (Clarté et al., 21 Feb 2024).
• Online Streaming Data: Fast autoregressive multiplier bootstraps achieve correctness and constant update cost but can have slower convergence rates and may generate negative weights, potentially problematic for strictly positive data or statistics (Palm et al., 2023).
• Sparse Evaluations in Optimization: Direct estimation of bootstrap dominance probabilities can be unreliable for decision points with few observations; transferring dispersion data from global error sets assumes homoscedasticity—a plausible implication is that further refinements would be needed for adaptive resampling under unknown noise distributions (Budszuhn et al., 27 Mar 2025).

6. Practical Impact and Future Directions

The proliferation of Boot-Both methodologies across statistics, ML, and optimization marks a shift toward resampling tools that are adaptive, robust, and scalable.

• Massive-data SDB, weighted bootstraps for dependence/heteroscedasticity, and dual-axis CI estimation have become central to big-data analytics, online learning, and robust model evaluation.
• Recent theoretical advances guarantee non-asymptotic stability even for outputs in general metric spaces (functions, distributions, control weights), supporting reproducibility and interpretability in causal inference and Bayesian analysis (Soloff et al., 15 May 2024).
• There is ongoing research into online variants for streaming data with dependency, enhanced correction factors for high-dimensional bias/variance estimation, and further refinement of adaptive decision procedures under stochastic noise in multi-objective optimization.

This topic continues to evolve, with current challenges including developing strictly positive weighting schemes for AR-bootstraps, extending consistency results to nonstationary or functional data, and creating theoretically principled corrections for double-descent phenomena in deep or overparameterized models. The broad applicability and modularity of Boot-Both resampling remain central for robust statistical inference in modern data-rich environments.