Jackknife Variance Estimator

• Jackknife variance estimator is a resampling method that computes leave-one-out pseudo-values to assess the variability of complex estimators.
• It achieves ratio-consistency when the estimator's variance is dominated by its first-order Hájek projection, covering generalized U-statistics.
• The method unifies classical and infinitesimal jackknife techniques, enabling robust variance estimation in diverse settings like TDNN regression.

The jackknife variance estimator is a resampling-based method for quantifying the variability of complex estimators, especially those expressible as U-statistics or their generalizations. In the classical case, the estimator's variance is often dominated by its first-order (Hájek) projection, a condition under which the jackknife can provide a consistent estimate. Recent developments have unified and generalized these results, demonstrating that the jackknife—long treated as a simple, nonparametric method—achieves ratio-consistency for a very broad class of generalized U-statistics whenever Hájek projection dominance holds. This advances the theoretical footing of the jackknife, placing it on equal ground with infinitesimal jackknife techniques for such estimators and extending variance validity to situations well beyond the fixed-order case.

1. Generalized U-Statistics and Variance Estimation

Generalized U-statistics encompass a broad range of functionals defined as averages over data subsets, potentially with data-dependent kernels and dynamically growing orders (kernel sizes). For such estimators $\widehat{\theta}_n$ defined via

%%%%1%%%%

the jackknife constructs leave-one-out pseudo-values by recomputing $\widehat{\theta}_n^{(-i)}$ for each $i$, and estimates the variance as

$$\widehat{V}_J = \frac{n-1}{n} \sum_{i=1}^n \bigl(\widehat{\theta}_n^{(-i)} - \bar{\theta}_n\bigr)^2,$$

where $\bar{\theta}_n$ is the average of the leave-one-out estimates. The core theoretical question is under what conditions $\widehat{V}_J$ consistently estimates $\operatorname{Var}(\widehat{\theta}_n)$, particularly in regimes where both kernel order and $n$ may diverge.

2. Hájek Projection Dominance: Unification and Generalization

The new unified condition is the asymptotic dominance of the variance by the Hájek projection (the sum of first-order influences), sometimes referred to as "Hájek dominance." Explicitly, for a generalized U-statistic $U_n$ with variance decomposed as

$\operatorname{Var}(U_n) = V_1 + V_2 + \ldots + V_k$

(where $V_1$ is the variance of the linear projection, $V_2, \ldots, V_k$ are higher-order components), the condition for ratio consistency of the jackknife is

$$\frac{V_1}{\operatorname{Var}(U_n)} \to 1 \text{ as } n \to \infty,$$

or in concrete terms for many distributional estimators,

$$\frac{\operatorname{Var}\bigl(\text{Linear Hájek projection}\bigr)}{\operatorname{Var}(U_n)} \to 1.$$

This encompasses both classical fixed-order U-statistics (where $k$ is fixed) and more complex estimators with $k = k(n)$ diverging.

Under this dominance, the jackknife variance estimator is "ratio-consistent," i.e.

$$\frac{\widehat{V}_J}{\operatorname{Var}(\widehat{\theta}_n)} \overset{P}{\to} 1.$$

This result places the nonparametric jackknife on the same asymptotic footing as the infinitesimal jackknife for generalized U-statistics (Juergens, 15 Sep 2025).

3. Connections to Existing Literature and Unified Criteria

Earlier literature on jackknife and infinitesimal jackknife variance estimation focused on specific cases—classical (fixed kernel order) U- and V-statistics, certain plug-in estimators, or cases with known degeneracy structures. The Hájek projection dominance condition subsumes and generalizes these, providing a single criterion under which resampling-based variance estimation (jackknife and infinitesimal jackknife) is valid.

This unification clarifies that, for a broad class of statistics—including those with data-dependent, high-order, or even locally adaptive kernels—the validity of the jackknife depends only on the first-order term's asymptotic control, not on fixed-order assumptions or specific kernel regularity.

4. Illustration: Two-Scale Distributional Nearest-Neighbor Estimator

The two-scale distributional nearest-neighbor (TDNN) regression estimator provides a concrete illustration. For TDNN with subsample sizes $s_1$ and $s_2$ satisfying $0 < c \leq s_1/s_2 \leq 1-c < 1$ and $s_2 = o(n)$, and under a nonparametric regression model with regularity conditions,

• the variance of the estimator is asymptotically dominated by its first-order (Hájek) projection,
• the full kernel variance $\zeta_{s_2}^{(s_2)}(x)$ is bounded,
• and the first-order projection variance $\zeta_{s_2}^{(1)}(x)$ decays as $s_2^{-1}$.

The crucial Hájek dominance requirement is shown to hold:

$$\frac{s_2}{n} \left[ \frac{\zeta_{s_2}^{(s_2)}}{s_2 \zeta_{s_2}^{(1)}} - 1 \right] \to 0.$$

Thus, the jackknife variance estimator is consistent for the variance of the TDNN estimator even under weaker and more flexible conditions than previously required—no need for full fixed-order or strong degeneracy control. This considerably broadens the scenarios in which jackknife-based inference is justified for distributional regression problems (Juergens, 15 Sep 2025).

5. Theoretical and Practical Implications

The establishment of ratio-consistency of the jackknife under minimal and interpretable dominance conditions has several practical and theoretical implications:

• For any estimator where variance is asymptotically governed by the linear (Hájek) projection, the jackknife variance estimate is justified regardless of kernel complexity, growth of $k$, or specific degeneracy structure.
• The result ties together nonparametric jackknife and infinitesimal jackknife approaches within a single framework; both yield consistent variance estimates wherever Hájek dominance is established.
• In practice, this allows researchers to use simple leave-one-out jackknife variance estimation for wide classes of plug-in, smoothing, or nearest-neighbor based estimators, including those for which analytic variance expressions are intractable.

6. Future Directions and Open Questions

While Hájek dominance is a natural and often easily checked condition in many modern statistical settings, there remain interesting cases where higher-order projection terms may not be negligible—a subtle problem in high-dimensional regimes or certain degenerate applications. Future work may focus on extending ratio-consistency to settings with partial or local degeneracy, develop diagnostics for Hájek dominance in new classes of statistics, or refine jackknife corrections to adaptively account for detected higher-order contributions.

Additionally, the computational efficiency and stability of jackknife variance estimation—as compared to analytic or infinitesimal jackknife approaches—make it a compelling default for a range of machine learning and statistical applications involving resampling or complex sampling structures.

7. Summary Table: Consistency of Jackknife Variance Estimator under Hájek Dominance

Scenario	Hájek Dominance Condition	Jackknife Consistency?
Fixed-order U-statistic	Yes (classical)	Yes
Generalized U-statistic, $k = k(n)$	$V_1 / \operatorname{Var}(U_n) \to 1$	Yes
TDNN estimator ($s_2 = o(n)$ etc.)	Verified in (Juergens, 15 Sep 2025)	Yes
Degenerate or higher-order dominance	No	No

In summary, the jackknife variance estimator is now theoretically justified for a broad class of generalized U-statistics and related estimators under the singular, interpretable condition of asymptotic Hájek projection dominance, offering both a sharp theoretical tool and a practical method for consistent variance estimation (Juergens, 15 Sep 2025).