---
title: Multiplier Bootstrap for Empirical Processes
url: https://www.emergentmind.com/topics/multiplier-bootstrap-for-empirical-processes
type: topic
---

# Multiplier Bootstrap for Empirical Processes

A multiplier bootstrap for empirical processes is a core nonparametric and high-dimensional inference tool, yielding nonasymptotic, theoretically justified finite-sample approximations to suprema of empirical processes, particularly for classes of functions with growing complexity and in settings where classical weak convergence is unavailable or intractable. This approach is central for constructing confidence bands, calibrating multiple testing procedures, and controlling the size and power of nonparametric tests. The method operates by resampling the empirical process via independent (often Gaussian or sub-Gaussian) random multipliers, and has been rigorously justified through strong coupling and Gaussian comparison inequalities, with explicit error bounds under minimal VC-type complexity and envelope conditions [1502.00352].

## 1. Definition and Formulation

Given i.i.d. data $(X_i)_{i=1}^n$ taking values in a measurable space $(S,\mathcal S)$, and a class of real-valued functions $\mathcal F$ on $S$ with envelope $F$, the centered empirical process is
\[
G_n(f) = \frac{1}{\sqrt n} \sum_{i=1}^n \bigl( f(X_i) - P f \bigr),\quad f \in \mathcal F,\qquad P f = \mathbb{E}[f(X_1)].
\]
The multiplier bootstrap process is defined as
\[
G_n^*(f) = \frac{1}{\sqrt n} \sum_{i=1}^n e_i \bigl( f(X_i) - P_n f \bigr),\quad f\in\mathcal F,
\]
where $e_i$ are i.i.d. random multipliers, independent of the data, with $\mathbb E e_i = 0$, $\mathbb E e_i^2 = 1$, and typically $\mathbb E[|e_i|^q] < \infty$ for some $q \ge 4$ [1502.00352].

The main object of interest is the supremum over $\mathcal F$ of a possibly biased empirical process,
\[
Z = \sup_{f \in \mathcal F}\{B(f) + G_n(f)\},
\]
where $B:\mathcal F \to \mathbb R$ is a bias functional, and its bootstrap analogue
\[
Z^* = \sup_{f \in \mathcal F}\{B(f) + G_n^*(f)\}.
\]

## 2. Key Theoretical Results and Error Bounds

Let $\mathcal F$ be a VC-type class with envelope $F$ and uniform entropy numbers: for all finitely-supported measures $Q$,
\[
N\Big(\mathcal F, \|\cdot\|_{Q,2}, \epsilon\|F\|_{Q,2}\Big) \le (A/\epsilon)^v,\quad 0<\epsilon<1.
\]
Assume moment/tail conditions: for some $b \ge \sigma > 0$, $q \ge 4$,
\[
\sup_{f \in\mathcal F} P|f|^k \le \sigma^2 b^{k-2},\; k=2,3,4, \qquad \|F\|_{P,q} \le b.
\]
Let $K_n = \log \mathcal{N}_B(n) + v(\log n \vee \log(Ab/\sigma))$ (the effective function class complexity).

Under $K_n^3 \ll n$ (complexity grows sub-polynomially with $n$), Chernozhukov–Chetverikov–Kato [1502.00352] show that there exist $C_1, C_2 > 0$ (depending only on $q$) and an event of probability $1 - O(n^{-1})$ under which
\[
P(|Z - Z^*| > C_1\delta_n \mid X_{1:n}) \le C_2(\gamma + n^{-1}),
\]
where
\[
\delta_n = bK_n n^{-1/2+1/q} + (b\sigma^2 K_n^2)^{1/2} n^{-1/4+1/(2q)}.
\]

Moreover, for the Kolmogorov distance,
\[
\sup_{t \in \mathbb R} \left|P(Z^* \le t \mid X_{1:n}) - P(\widetilde Z \le t)\right| \le C_1 \delta_n + C_2 (\gamma + n^{-1}) + {\rm anticonc},
\]
where "anticoncentration" is of the order $\sigma^{-1}(\delta_n+\cdots) \sqrt{\log N(\mathcal F, e_P, \delta)}$ and $\widetilde Z$ is a copy of the Gaussian limit $\sup_{f\in \mathcal F}\{B(f) + G_P(f)\}$ [1502.00352].

## 3. Coupling Construction and Proof Techniques

The coupling is constructed not via Hungarian embedding, but using Slepian–Stein interpolation and Gaussian comparison inequalities. The argument proceeds by discretizing $\mathcal F$ at scale $\epsilon$ so that $\mathcal F$ is covered by $N\leq e^{C K_n}$ points, then applying a two-sample Slepian–Stein interpolation to compare
\[
\max_j \{B(f_j) + G_n(f_j)\} \quad \text{and} \quad \max_j \{B(f_j) + G_P(f_j)\},
\]
with empirical covariance deviations controlled via Talagrand/Bousquet inequalities, and exploiting Gaussian anticoncentration (Nazarov's inequality). The discretization error is of order $O(\epsilon)$ [1502.00352].

## 4. Statistical Applications

- **Uniform Confidence Bands:** For inference on $\{P f\}_{f\in\mathcal F}$, define bootstrap quantiles $Z^*_\alpha = \inf\{t : P(Z^* \le t \mid X_{1:n}) \ge 1-\alpha\}$. The band
  \[
  \left\{f: \sqrt{n}|P_n f - P f| \le Z^*_\alpha\right\}
  \]
  has coverage approximately $1-\alpha$, with error up to $\delta_n$ [1502.00352].

- **Nonparametric Test Power:** Under alternatives with $B(f) \neq 0$ for some $f$, the distribution of $\sup_{f\in \mathcal F}\{B(f) + G_n(f)\}$ is well approximated by its bootstrap counterpart, permitting uniform control of power and size over growing classes of alternatives and derivation of local separation rates [1502.00352].

## 5. Adaptation to Growing Complexity Classes

The analysis permits all constants $(A, v, b, \sigma, q)$ to depend on $n$, provided $K_n^3 \ll n$. Thus, the supremum of the empirical process indexed by classes $\mathcal F_n$ with rapidly growing VC-dimension may be bootstrapped nonasymptotically with explicit rate control:
\[
\delta_n = O\left(bK_n n^{-1/2+1/q} + (b\sigma^2 K_n^2)^{1/2} n^{-1/4+1/(2q)}\right).
\]
Consequently, even very high-dimensional, non-Donsker, or highly composite classes can be accommodated, provided sub-polynomial growth of class complexity relative to the sample size [1502.00352].

## 6. Generalizations and Related Methodologies

The general theory has been extended in several directions:
- **High-Dimensional Maxima and General $\kappa$-th Order Statistics:** Bootstrap error bounds for the $\kappa$-th largest coordinate under sub-exponential tails and non-i.i.d. settings, allowing $p \gg n$ [2508.14400].
- **Empirical and U-Process Extensions:** The JMB and dependent multiplier schemes facilitate strong approximations for suprema of U-processes under non-i.i.d. and mixing conditions, crucial for change-point analysis and confidence region construction in time series [1708.02705, 1412.5875].
- **Dependent and Block Bootstrapping:** Multiplier bootstraps have been developed for weakly dependent (mixing) and Markovian data via block or regenerative schemes, preserving strong approximation rates under dependence [1511.00420, 2510.17648].
- **Copula and Tail Inference:** For the estimation and inference of tail copula functionals, direct multiplier schemes and checkerboard smoothing enable valid confidence bands and hypothesis testing without explicit limiting covariance estimation [1102.0110, 2601.10252].

## 7. Implementation and Practical Considerations

- The method is fully algorithmic, requiring only simulation of i.i.d. multiplier sequences, empirical centering, and computation of process suprema.
- The approach is robust to multiplier choice (Gaussian, Rademacher, etc.), provided the moment/tail conditions are met.
- The coupling error and bootstrap coverage error are nonasymptotically controlled with explicit rates, supporting reliable inference even for moderate sample sizes and high dimensionality.
- The Gaussian comparison framework extends naturally to studentized/variance-stabilized statistics, local U-processes, projection kernels, and other empirical process-based statistics, facilitating calibration for modern, complex inference tasks [1502.00352].

---

**References:**  
Chernozhukov, V., Chetverikov, D., Kato, K. "Empirical and multiplier bootstraps for suprema of empirical processes of increasing complexity, and related Gaussian couplings" [1502.00352].

Source: https://www.emergentmind.com/topics/multiplier-bootstrap-for-empirical-processes