An Elementary Proof of the Dvoretzky--Kiefer--Wolfowitz--Massart Inequality

Abstract: The Dvoretzky--Kiefer--Wolfowitz--Massart inequality gives an upper bound on the probability that the empirical distribution function of a finite sequence of independent random variables deviates from its theoretical value. It is widely used in statistics in tests and in the production of confidence intervals. The original proof by Massart, later reformulated by Dudley, is very long and technical. Recently, Reeve slightly extends Dvoretzky--Kiefer--Wolfowitz--Massart inequality and presents an alternative, much shorter proof, that uses a continuous time martingale. We present a simpler and less technical proof of the original Dvoretzky--Kiefer--Wolfowitz--Massart inequality, based on a discrete-time martingale and Sion's minimax theorem.

• The paper presents a new direct proof of the DKWM inequality using discrete-time martingales that recovers the optimal exponent 2nε².
• It employs minimax duality and Sion's theorem to simplify traditional, technical proofs, thereby reducing analytical complexity.
• The results offer sharp, non-asymptotic exponential concentration bounds crucial for empirical process theory and nonparametric inference.

An Elementary Proof of the Dvoretzky–Kiefer–Wolfowitz–Massart Inequality

Overview

The Dvoretzky–Kiefer–Wolfowitz–Massart (DKWM) inequality provides sharp, non-asymptotic exponential concentration bounds for the uniform deviation between the empirical distribution function (EDF) and the true cumulative distribution function (CDF) of independent and identically distributed (i.i.d.) samples. The DKWM inequality is foundational in probability theory and statistics, underpinning the finite-sample validity of nonparametric inference, uniform empirical process theory, and confidence bands.

This paper delivers a new, direct, and elementary proof of the DKWM inequality. In contrast to Massart's original, highly technical arguments and Reeve's recent martingale-centric refinements that utilize continuous time, this work employs a discrete-time martingale framework along with Sion's minimax theorem. The result is an explicit, concise, and accessible argument that recovers the optimal constants and generality of Massart's theorem without the analytical complexity typical of previous proofs (2607.04387).

Statement of Results

Given $n$ i.i.d. real random variables $(X_i)_{i=1}^n$ with distribution function $F$ and empirical distribution function $\hat{F}_n(x)$, the DKWM bounds are:

• One-sided:

$$P\left(\sup_{x \in \mathbb{R}} (\hat{F}_n(x) - F(x)) > \varepsilon \right) \leq e^{-2n \varepsilon^2}, \quad \forall \varepsilon > 0$$

• Two-sided:

$$P\left(\sup_{x \in \mathbb{R}} |\hat{F}_n(x) - F(x)| > \varepsilon \right) \leq 2 e^{-2n \varepsilon^2}, \quad \forall \varepsilon > 0$$

The sharpness in the exponent $2n \varepsilon^2$ is preserved.

Proof Strategy

Reduction to Canonical Setting

The proof begins by observing that the supremum deviation is preserved under probability integral transform. Thus, it suffices to establish the bound for $n$ i.i.d. $U[0,1]$ random variables. This reduction exploits the fact that the CDF transforms any sample into a uniform distribution, allowing the EDF–CDF deviation for general distributions to be controlled by the uniform case.

Discrete-Time Martingale Construction

Let $(U_i)_{i=1}^n$ be i.i.d. $(X_i)_{i=1}^n$0. The supremum is translated into the event that the $(X_i)_{i=1}^n$1-th order statistic $(X_i)_{i=1}^n$2 drops beneath $(X_i)_{i=1}^n$3 for some $(X_i)_{i=1}^n$4. Introducing $(X_i)_{i=1}^n$5, the event $(X_i)_{i=1}^n$6 is equivalent to $(X_i)_{i=1}^n$7.

A reverse martingale sequence $(X_i)_{i=1}^n$8 is constructed, with $(X_i)_{i=1}^n$9 a function of $F$0 designed so that $F$1 upper bounds the desired probability via Doob’s reverse martingale maximal inequality. The explicit link to the binomial tail probability sets the stage for a sharp, exponential bound.

Minimax Optimization and Sion's Theorem

Optimizing the martingale-based upper bound over an exponential generating parameter $F$2 and using the explicit structure of the binomial likelihood, the argument proceeds by bounding the probability as:

$F$3

The function $F$4 is tailored to encapsulate the exponential moment method. Crucially, it is shown to possess quasi-concavity in $F$5 and quasi-convexity in $F$6, allowing Sion’s minimax theorem to be applied for saddle-point exchange.

Explicit Bound via Binary Relative Entropy

The minimax evaluation leads to an explicit representation in terms of the binary Kullback–Leibler divergence:

$F$7

Here, $F$8 denotes the binary KL divergence. Applying Pinsker's inequality, $F$9, recovers the canonical $\hat{F}_n(x)$0 bound.

Numerical Sharpness and Claims

The proof precisely recovers the optimal DKWM constants as established by Massart and removes additional technical constraints on $\hat{F}_n(x)$1. The approach is fully explicit and avoids reliance on measure-theoretic underpinnings or advanced empirical process theory.

Key strong claim: The exponent $\hat{F}_n(x)$2 is achieved by a direct martingale and minimax argument, exposing the tightness of the DKWM inequality without the intricacy of previous treatments involving either symmetrization or continuous time limits.

Implications and Theoretical Impact

This proof substantially lowers the access barrier for the application and teaching of concentration inequalities for empirical processes. The explicit martingale construction demystifies the role of order statistics and binomial deviations, potentially facilitating generalizations to other settings such as Markov-dependent samples, sub-exponential tails, or high-dimensional function-indexed processes. The method may also be adapted to derive nonasymptotic bounds for other functionals of empirical measures with sharp constants.

The direct connection between martingale inequalities, saddle point duality, and information-theoretic divergence supports a unifying methodological template for future empirical process concentration results.

Prospects for Future Research

The discrete martingale and minimax approach advocated here may inspire several follow-ups:

• Extensions to non-i.i.d. settings: Adaptation to dependent data via coupling or block martingales.
• Sharper constants for small $\hat{F}_n(x)$3: Fine tuning the argument for finite-sample improvements, possibly leveraging exact binomial tail behavior.
• High-dimensional analogues: Application to multivariate empirical distributions and processes on function spaces.
• Algorithmic certificate generation: Programmatic generation of nonasymptotic confidence bands and online estimation protocols with proved guarantees.

Conclusion

This paper provides an elementary, transparent, and optimal proof of the Dvoretzky–Kiefer–Wolfowitz–Massart inequality. Employing discrete-time martingale constructions and minimax duality, it offers both pedagogical clarity and practical power, yielding sharp deviation bounds fundamental to nonparametric statistics and empirical process theory (2607.04387). The methodology opens new directions for streamlined proofs in probability and statistics, and sets a foundation for further theoretical developments and practical algorithms in finite-sample statistical control.