Incomplete U-Statistics: Theory & Applications

• Incomplete U-statistics are estimators that average a symmetric kernel function over a subset of data combinations, drastically reducing computational cost while preserving unbiasedness.
• They maintain key properties such as controlled variance and asymptotic normality, effectively bridging the gap between complete U-statistics and i.i.d. sums.
• Their flexible design—including random, Bernoulli, and deterministic sampling—supports scalable applications in machine learning, hypothesis testing, and network analysis.

An incomplete U-statistic is an estimator formed by averaging a symmetric kernel function over a stochastic or deterministic subset of all possible subsets of data points of a certain order. This approach drastically reduces computational cost compared to the classical complete U-statistic, which averages over all possible combinations. Incomplete U-statistics have emerged as a central tool in large-scale statistical inference, machine learning, hypothesis testing under constraints, kernel methods, and network analysis, as they provide unbiased or near-unbiased estimators while enabling scalability and efficient uncertainty quantification. Their theoretical properties—including variance, normal approximation rates, and concentration inequalities—interpolate between the regimes of full U-statistics and i.i.d. sums, depending on the sampling design, kernel degeneracy, and computational budget.

1. Formal Definitions and Construction

Given i.i.d. data $X_1,\dots,X_n$ and a symmetric kernel $h:M^m\to\mathbb{R}$ of order $m$, the complete U-statistic is

$$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$

An incomplete U-statistic is formed by averaging $h$ over a subset $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$: $$U'_{n,N} = \frac{1}{|S|} \sum_{(i_1,\dots,i_m)\in S} h(X_{i_1},\dots,X_{i_m}).$$ Sampling schemes for $S$ include:

• Sampling $N$ subsets uniformly at random (with or without replacement).
• Bernoulli sampling: each $m$-tuple is selected independently with probability $h:M^m\to\mathbb{R}$0, yielding a random number $h:M^m\to\mathbb{R}$1 of terms and

$h:M^m\to\mathbb{R}$2

where $h:M^m\to\mathbb{R}$3 (Leung, 2024, Leung et al., 2024, Chen et al., 2017).

• Deterministic designs: select $h:M^m\to\mathbb{R}$4 of special combinatorial structure—e.g., equireplicate or balanced incomplete designs (Miglioli et al., 23 Oct 2025, Duembgen et al., 2022, Kong et al., 2020).

Incomplete U-statistics generalize naturally to multi-dimensional, Banach-valued, or Hilbert-valued kernels and to the setting of random or diverging kernel order $h:M^m\to\mathbb{R}$5 (Giraudo, 2024, Song et al., 2019).

2. Statistical Properties: Unbiasedness, Variance, and Limiting Distributions

Unbiasedness

The incomplete U-statistic retains unbiasedness for the target functional: for any of the classic sampling schemes,

$h:M^m\to\mathbb{R}$6

(Leung et al., 2024, Maurer, 2022, Chen et al., 2017, Cabrera et al., 2024, Miglioli et al., 23 Oct 2025).

Variance Decomposition

Let $h:M^m\to\mathbb{R}$7, $h:M^m\to\mathbb{R}$8, and $h:M^m\to\mathbb{R}$9. In the Bernoulli sampling regime (Leung et al., 2024, Leung, 2024),

$m$0

The complete U-statistic variance is $m$1. The extra $m$2 term in the incomplete case provides non-vanishing variance even in degenerate situations ($m$3), a key property for robust inference under singular constraints (Leung et al., 2024, Chen et al., 2017, Miglioli et al., 23 Oct 2025).

In more general designs, the variance is controlled by the combinatorial geometry of $m$4, e.g., overlap counts and replication numbers in deterministic schemes (Miglioli et al., 23 Oct 2025, Maurer, 2022, Kong et al., 2020).

Central Limit Theorems and Normal Approximation

With appropriate moment assumptions, incomplete U-statistics satisfy CLTs whose asymptotic variance interpolates between complete U-statistics and i.i.d. averages:

• If $m$5, incomplete and complete U-statistics share variance and convergence rates.
• If $m$6, sampling noise dominates and normalization by $m$7 gives a CLT with variance $m$8 (Leung, 2024, Chen et al., 2017, Miglioli et al., 23 Oct 2025, Löwe et al., 2020).
• If $m$9, the limiting variance is a sum: $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$0 (Leung et al., 2024).

Berry–Esseen type bounds—quantitative central limit theorems—are established under minimal finite moment assumptions, with rates depending on $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$1 and $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$2 (Leung, 2024, Leung et al., 2024, Miglioli et al., 23 Oct 2025). Explicit expressions quantify the error in normal approximation for studentized statistics, including settings with high-dimensional and possibly degenerate kernels.

3. Concentration Inequalities and Non-Asymptotic Guarantees

Sharp deviation inequalities, including Bernstein- and exponential-type inequalities, are available for incomplete U-statistics:

• The probability $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$3 decays at the optimal rate in $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$4 and $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$5 (Maurer, 2022, Duembgen et al., 2022, Giraudo, 2024, Giraudo, 2024).
• Constants depend on the kernel's variance and sensitivity, design geometry (e.g., overlap counts, equireplication), and moment or tail properties.
• For Banach- and Hilbert-valued kernels, moment and exponential inequalities generalize, with deviation rates governed by degeneracy order and the underlying space's smoothness (Giraudo, 2024, Giraudo, 2024, Duembgen et al., 2022).

For random sampling designs, as soon as the number of sampled terms $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$6 reaches $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$7, incomplete U-statistics achieve the same order concentration as the complete U-statistic, justifying their usage in large-scale scenarios (Maurer, 2022).

4. Design Strategies: Randomized, Deterministic, and Structured Sampling

Several construction paradigms enable balancing statistical efficiency and computational tractability:

• Bernoulli or uniform sampling: Draws $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$8 terms at random, with or without replacement. Admits transparent variance formulas and is robust to kernel degeneracy (Leung, 2024, Maurer, 2022, Chen et al., 2017).
• Equireplicate/balanced incomplete block designs: Deterministic designs where each data index appears precisely $$U_n = \binom{n}{m}^{-1}\sum_{1\leq i_1<\cdots<i_m\leq n} h(X_{i_1},\dots,X_{i_m}).$$9 times, and overlap structure is controlled. This minimizes variance for fixed budget and supports exact finite-sample Berry–Esseen bounds (Miglioli et al., 23 Oct 2025, Duembgen et al., 2022, Kong et al., 2020).
• Orthogonal Array and hypergraph-based designs: Provides higher-order balance in sampled tuples, permitting asymptotically efficient estimation with dramatically fewer terms—e.g., $h$0 for non-degenerate kernels, improving on earlier $h$1 requirements (Kong et al., 2020).
• Pruning or conditional selection: Subset selection based on auxiliary covariates, as in conditional dependence testing, to enhance power or avoid matrix inversions (Cabrera et al., 2024).
• Balanced-incomplete or cyclic designs: Used in scatter estimation and M-estimation, offering nearly complete efficiency with modest $h$2 (Duembgen et al., 2022).

The choice of design directly impacts bias, variance, tail behavior, and computational cost, and optimal strategies are context-dependent.

5. Applications in Statistical Inference and Machine Learning

Large-Scale Empirical Risk Minimization and Ensemble Methods

Incomplete U-statistics underlie scalable ERM for risk functionals that are U-statistics of order $h$3 (ranking, clustering, metric learning) (Clémençon et al., 2015, Song et al., 2019).

• Uniform deviation theorems guarantee $h$4 learning rates as soon as $h$5, under standard VC or entropy assumptions.
• In stochastic gradient descent, incomplete-U-based mini-batching achieves lower variance and faster convergence than sub-sampling followed by complete U-statistics (Clémençon et al., 2015).

Hypothesis Testing under Constraints

Incomplete U-statistics are central to testing polynomial or semialgebraic constraints (e.g., covariance structure, phylogenetic hypotheses):

• The SDL framework (Barnhill et al., 17 Jul 2025, Sturma et al., 2022) uses incomplete U-statistics with Gaussian multiplier bootstrap to maintain validity even near boundaries or singularities, where classical Wald tests fail.
• Variant constructions enable uniform type I error control with high-dimensional and possibly mixed-degenerate kernels (Sturma et al., 2022).

Kernel Methods and Nonparametric Testing

U-statistics estimate quantities such as Maximum Mean Discrepancy (MMD), Hilbert-Schmidt Independence Criterion (HSIC), and Kernel Stein Discrepancy. Incomplete U-statistic analogs provide linear- or subquadratic-time procedures with provable minimax-optimality and scalable bootstrap calibration (Schrab et al., 2022, Miglioli et al., 23 Oct 2025, Cabrera et al., 2024).

High-Dimensional and Infinite-Order Scenarios

Statistical inference based on U-statistics with diverging or infinite order is tractable only via incomplete designs. Non-asymptotic Gaussian and bootstrap approximations are available for inference on ensemble predictors (subbagging, random forests) or network methods-of-moments (Song et al., 2019, Shao et al., 2023).

6. Robustness, Degeneracy, and Singularities

Incomplete U-statistics offer robustness advantages over complete U-statistics:

• In degenerate cases (where leading Hoeffding projections vanish), CLTs and Berry–Esseen bounds for complete U-statistics degenerate or fail, while incomplete U-statistics retain strictly positive asymptotic variance and Gaussian limits across all regimes (Leung et al., 2024, Leung, 2024, Miglioli et al., 23 Oct 2025).
• Singularity-agnostic normal approximations hold uniformly as the model approaches singular points—crucial for polynomial constraint testing and boundary inference (Leung et al., 2024).
• Consistency is secured under minimal moment conditions, including in the presence of infinite second moments, with rates adapting to the moment order $h$6 (Dürre et al., 2021).

7. Computational Complexity, Statistical–Computational Trade-Offs, and Practical Guidance

By subsampling $h$7 terms, incomplete U-statistics lower computational cost by orders of magnitude. Trade-offs are quantitatively characterized:

• For prescribed error tolerance $h$8, exponential concentration and normal approximation rates determine how $h$9 scales with $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$0 (Maurer, 2022, Shao et al., 2023).
• Edgeworth expansions quantify higher-order risk control: with $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$1, the error in coverage (or type I) shrinks as $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$2, allowing practitioners to attain full-U accuracy with $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$3 (or lower, depending on degeneracy and design) (Shao et al., 2023).
• Practical implementations recommend $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$4 or $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$5 proportional to $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$6 for balanced trade-offs in uncertainty quantification and computational budget (Leung, 2024, Barnhill et al., 17 Jul 2025, Schrab et al., 2022).
• For deterministic/equireplicate designs, efficient $$S \subseteq \{(i_1,\dots,i_m) : 1 \leq i_1<\dots<i_m \leq n\}$$7 algorithms construct sampling plans with near-minimum variance (Miglioli et al., 23 Oct 2025, Kong et al., 2020).

Incomplete U-statistics are thus indispensable for modern high-dimensional, large-sample, and constraint-based inference, providing a continuum from i.i.d. statistics to the classical U-theory, computational feasibility, and theoretical guarantees across a variety of statistical and machine learning domains.