Serfling’s Inequality: Finite-Sampling Tail Bounds

• Serfling’s inequality is a finite-sampling exponential concentration inequality that extends Hoeffding’s bounds to sampling without replacement.
• It utilizes a telescoping sum and conditional moment generating functions to derive sharp tail bounds incorporating finite-sample corrections.
• The method is crucial in hypergeometric settings and two-sample empirical processes, enhancing statistical inference in nonparametric tests.

Serfling’s inequality is a finite-sampling exponential concentration inequality that extends Hoeffding’s classical bounds for sums of independent bounded random variables to the case of sampling without replacement from a finite population. It provides sharp exponential tail bounds for deviations of sample means from the population mean under sampling fraction corrections, with particular relevance for hypergeometric and empirical process contexts (Greene et al., 2015).

1. Formal Statement and Interpretation

Let $\{c_1,\ldots,c_N\}$ be a finite population (“urn”) with $c_i \in \mathbb{R}$, population mean $\mu_N = \frac{1}{N}\sum_{i=1}^N c_i$, variance $$\sigma_N^2 = \frac{1}{N}\sum_{i=1}^N (c_i - \mu_N)^2$$, minimum $a_N = \min_i c_i$, and maximum $b_N = \max_i c_i$. Consider sampling without replacement $n \le N$ elements, yielding $Y_1,\ldots,Y_n$, with sample mean $\overline Y_n = \frac1n\sum_{i=1}^n Y_i$. Define the finite-sampling fractions $f_n^* = \frac{n-1}{N}$ and $f_n = \frac{n-1}{N-1}$.

Serfling’s inequality states that for any $\lambda > 0$,

$$\Pr\!\Bigl(\sqrt n\,(\overline Y_n-\mu_N)\;\ge\;\lambda\Bigr) \;\le\; \exp\!\Bigl(-\,\frac{2\,\lambda^2}{\,(1-f_n^*)\,(b_N-a_N)^2}\Bigr).$$

A frequently employed specialization is when $b_N - a_N = 1$: $$\Pr\!\bigl(\overline Y_n-\mu_N\ge\epsilon\bigr) \;\le\;\exp\!\bigl(-2\,n\,\epsilon^2/(1-f_n^*)\bigr).$$ These forms quantify the upper tail probabilities of the sample mean deviating from the population mean under sampling without replacement, with explicit finite-sample corrections.

2. Analytical Strategy and Key Proof Elements

The proof adapts Hoeffding’s martingale-based approach for independent variables to the without-replacement regime. The essential steps are:

• Expressing $S = \sum_{i=1}^n Y_i$ as a telescoping sum of conditional expectations.
• Stepwise control of the conditional moment-generating function $M(t) = \mathbb{E}\bigl[e^{t(S-\mathbb{E} S)}\bigr]$, leveraging that at each stage, the remaining population values remain bounded in $[a_N, b_N]$.
• Demonstrating, by induction,

$$M(t) \le \exp\!\Bigl(\frac{t^2}{8}\sum_{i=1}^n (b_N-a_N)^2\left(1-\frac{i-1}{N}\right)\Bigr) = \exp\!\Bigl(\frac{t^2 n (1-f_n^*) (b_N-a_N)^2}{8}\Bigr).$$

• Invoking Markov’s inequality $$\Pr(S - \mathbb{E} S \ge \lambda\sqrt n) \le e^{-t\lambda\sqrt n}M(t)$$ and optimizing $t$ to obtain the exponential rate. The correction factor $(1-f_n^*) = (N-n+1)/N$ directly arises from the diminishing uncertainty after each observed draw, distinguishing the without-replacement scenario from the i.i.d. case (Greene et al., 2015).

3. Relationship to Hoeffding’s Inequality and Refinements

Hoeffding’s classical bound for independent random variables $Z_1,\ldots,Z_n$ with $Z_i\in[a,b]$ is

$$\Pr\!\Bigl(\sum_{i=1}^n Z_i \ge t\Bigr) \le \exp\!\Bigl(-\frac{2t^2}{n(b-a)^2}\Bigr).$$

For sampling with replacement from $\{c_1,\ldots,c_N\}$, substituting $Z_i = Y_i - \mu_N$ yields

$$\Pr\bigl(\overline Y_n-\mu_N\ge\epsilon\bigr) \le \exp\!\bigl(-2n\epsilon^2/(b_N-a_N)^2\bigr).$$

Serfling’s bound, with its $1/(1-f_n^*)$ exponent augmentation, tightens the concentration in the without-replacement regime. Since $1-f_n^* = (N-n+1)/N < 1$, the bound always strictly improves upon the naive i.i.d. Hoeffding bound for sampling without replacement.

A conceivable enhancement is to replace $1-f_n^*$ with the more accurate $1-f_n = (N-n)/(N-1)$, which has been achieved in special cases but remains open in general. Bennett-type refinements, applying Ehm’s representation of the hypergeometric as sums of independent Bernoullis, yield

$$\Pr\!\bigl(\sqrt n(\overline Y_n-\mu_N)\ge\lambda\bigr) \le \exp\Bigl(-\frac{\lambda^2}{2\,\sigma_N^2(1-f_n)}\psi\left(\frac{\lambda}{\sqrt n\,\sigma_N^2(1-f_n)}\right)\Bigr),$$

where $\psi(y) = 2y^{-2}h(1+y)$, $h(u) = u(\ln u-1)+1$. For binary populations ($\sigma_N^2 \le 1/4$), an explicit Hoeffding-style bound follows (Greene et al., 2015).

4. Hypergeometric Specialization

Consider the population made up of $D$ ones and $N-D$ zeros. Here, $\overline Y_n$ is the sample proportion of ones; $\mu_N = D/N$; $b_N - a_N = 1$. Serfling’s bound yields

$$\Pr\!\bigl(\sqrt n(\overline Y_n-D/N)\ge\lambda\bigr) \le \exp\Bigl(-\frac{2\lambda^2}{1-f_n^*}\Bigr).$$

Specifically, for $X \sim \mathrm{Hypergeometric}(N,D,n)$,

$$\Pr\Bigl(\sqrt n\,\bigl(n^{-1}X-D/N\bigr)\ge\lambda\Bigr) \le \exp\Bigl(-\frac{2\lambda^2}{1-(n-1)/N}\Bigr).$$

Such hypergeometric tail bounds are central in settings where binary attributes are counted under finite sampling, for example, in quality control and resampling inference (Greene et al., 2015).

5. Finite-Sampling Correction Terms and Open Questions

The “primitive” finite-sampling correction in Serfling’s exponent is $1-f_n^* = (N-n+1)/N$. The “true” correction $1-f_n = (N-n)/(N-1)$, corresponding to the classical finite-population variance reduction, appears in recent Bennett-type and Hoeffding-type inequalities but has not been universally established for Serfling’s bound in general settings. The proximity of these correction factors is essential for maximal sharpness in empirical process and finite-population inferential theory. Existing results and evidence suggest such refinement is plausible and likely achievable in further generalizations, especially under independence approximations (Greene et al., 2015).

6. Applications to Two-Sample Empirical Process Statistics

Serfling’s inequality forms the backbone of corrected exponential tail bounds for two-sample Kolmogorov–Smirnov (K–S) statistics. For independent samples $X_1,\ldots,X_m$ and $Y_1,\ldots,Y_n$ from the same continuous distribution $F$, with empirical CDFs $F_m$ and $G_n$, the two-sample one-sided K–S statistic is

$$D_{m,n}^+ = \sqrt{\frac{mn}{m+n}}\;\sup_x (F_m(x) - G_n(x)).$$

Viewing the pooled empirical CDF $H_{m+n}$ as the population, each empirical CDF is a sample without replacement, and Serfling’s bound applies. In the balanced case $m = n$, with $N=2n$,

$$\Pr(D_{n,n}^+ \ge t) \le \exp\Bigl(-2\,\frac{N-1}{N}\, t^2\Bigr),$$

with the two-sided statistic

$$\Pr(D_{n,n} \ge t) \le 2 \,\exp\Bigl(-2\,\frac{N-1}{N}\, t^2\Bigr).$$

The finite-sampling correction $\frac{N-1}{N}$ adjusts the exponent of the classical Dvoretzky–Kiefer–Wolfowitz–Massart inequality $\exp(-2t^2)$. For unbalanced samples ($m\neq n$), conjecturally, similar exponential bounds hold: $$\Pr(D_{m,n}^+ \ge t) \le \exp\left(-2\frac{N-1}{N} t^2\right), \quad \Pr(D_{m,n} \ge t) \le 2 \exp\left(-2 \frac{N-1}{N} t^2\right).$$ These corrections have significance for the tightness and calibration of empirical process-based inference, notably in nonparametric hypothesis testing and distributional comparison (Greene et al., 2015).