On Median of Incomplete U-Statistics

Abstract: We establish the finite-sample concentration rate for the Median-of-Incomplete-U-Statistics (MIU), an efficient robust estimator for the expectation of symmetric kernels.

• The paper presents finite-sample concentration bounds demonstrating that the MIU estimator achieves robust sub-Gaussian performance using median-of-means techniques.
• It introduces an efficient incomplete U-statistic approach that reduces the costly O(N^k) computations to a linear cost in M, ensuring scalability for large datasets.
• The method leverages empirical variance calibration to control estimation error, proving competitive robustness compared to complete U-statistics.

Concentration Inequalities for the Median of Incomplete U-Statistics

Introduction

This work rigorously analyzes the finite-sample concentration behavior of the Median-of-Incomplete-U-Statistics (MIU) estimator, a robust method for estimating expectations of symmetric kernels. The MIU is motivated by computational limitations associated with complete U-statistics, especially as sample size $N$ and kernel order $k$ grow. The paper provides non-asymptotic high probability concentration bounds for MIU, leveraging classical results for median-of-means (MoM) and recent advances in concentration of incomplete U-statistics.

Background and Setting

The median-of-means (MoM) approach partitions data into $T$ independent groups of size $N$; each provides an empirical mean, and their median is taken to mitigate heavy-tailed effects. The paper demonstrates that for a distribution with mean $\mu$ and variance $\sigma^2$, and for $T \geq 8 \ln(1/\delta)$, the MoM estimator $\widehat\mu_\mathrm{MM}$ satisfies a sub-Gaussian-type concentration with high probability:

$$\left|\widehat\mu_\mathrm{MM}-\mu\right| \leq 2\sigma\sqrt{\frac{8\ln(1/\delta)+1}{NT}}$$

with probability at least $1-\delta$. The proof utilizes Chebyshev's and Hoeffding's inequalities to formalize the robust properties of the median-of-means estimator.

For U-statistics, the paper defines the complete U-statistic of order $k$0 as:

$k$1

Here $k$2 is a symmetric kernel and $k$3 is the set of $k$4-combinations of indices. The expectation of $k$5 equals the parameter of interest $k$6.

Using a decoupling argument and symmetrization, it is shown that the complete U-statistic inherits sub-Gaussian concentration properties from i.i.d. means. If $k$7 almost surely, then, for $k$8,

$k$9

with probability at least $T$0.

Incomplete U-Statistics and the MIU Estimator

To mitigate the $T$1 computational cost of complete U-statistics, the incomplete U-statistic $T$2 randomly samples $T$3 $T$4-tuples (with replacement) from $T$5, evaluating $T$6 on each. Repeating this process $T$7 times independently yields $T$8 incomplete estimates, from which the MIU estimator is defined by taking their median:

$T$9

Main Results: Finite Sample Concentration for MIU

A core technical contribution is the finite-sample high-probability bound for the MIU estimator. For any $N$0, $N$1, and $N$2, with probability at least $N$3,

$N$4

This result is obtained via a decomposition of the estimation error into two terms: (i) deviation between MIU and the complete U-statistic, controlled by robust-median-of-means concentration, and (ii) deviation between the complete U-statistic and the population parameter, bounded by U-statistics concentration inequalities.

The proof leverages conditional symmetries and adapts the robust median-of-means machinery to incomplete U-statistics, exploiting empirical variance of the kernel evaluated over all $N$5-tuples in the sample.

Implications and Future Directions

The result establishes the MIU estimator as a computationally tractable robust alternative to the complete U-statistic, delivering concentration guarantees competitive with those of the unbiased estimator, subject to an empirical variance term and the additional approximation error stemming from incomplete subsampling. This provides formal justification for the use of MIU estimators in large-scale and high-order kernel applications, particularly in robust statistics and statistical machine learning scenarios where heavy-tailed data or contamination is plausible.

A critical practical implication is the scalability of high-order $N$6-statistic-based inference, which becomes feasible for large $N$7 and $N$8 due to the linear cost in $N$9 rather than $\mu$0. The empirical variance calibration suggests potential adaptivity, tuning $\mu$1 and $\mu$2 for target confidence/accuracy tradeoffs.

Theoretically, this work adds to concentration-of-measure methodology by extending robust estimators' analysis to functionals of randomly selected dependent tuples (as in incomplete U-statistics). These tools can inform robust estimation under dependence, higher-order interactions, and are promising for applications in distributed/combinatorial statistical learning, high-dimensional ANOVA decompositions, and model assessment.

Future research directions include: (i) extending the MIU analysis to more general dependence structures within the data, (ii) exploring minimax optimal rates for MIU under heavy-tailed or contaminated data, and (iii) applying MIU in contemporary ML settings, e.g., empirical risk minimization under complex loss kernels, where incomplete U-statistics arise naturally.

Conclusion

This paper provides a finite-sample concentration result for the median of incomplete U-statistics estimator, quantifying its estimation error relative to the population parameter as a sum of a “robust, empirical variance” term and a standard U-statistics deviation term. The findings substantiate MIU as an efficient, robust, and scalable estimator for expectation of symmetric kernels when complete U-statistics are computationally infeasible. This bridges robust estimation theory and practical, computationally efficient methodology for large-scale statistical inference.