Bounding the expectation of the supremum of empirical processes indexed by Hölder classes

Abstract: In this note, we provide upper bounds on the expectation of the supremum of empirical processes indexed by H\"older classes of any smoothness and for any distribution supported on a bounded set in $\mathbb R^d$. These results can be alternatively seen as non-asymptotic risk bounds, when the unknown distribution is estimated by its empirical counterpart, based on $n$ independent observations, and the error of estimation is quantified by the integral probability metrics (IPM). In particular, the IPM indexed by a H\"older class is considered and the corresponding rates are derived. These results interpolate between the two well-known extreme cases: the rate $n^{-1/d}$ corresponding to the Wassertein-1 distance (the least smooth case) and the fast rate $n^{-1/2}$ corresponding to very smooth functions (for instance, functions from an RKHS defined by a bounded kernel).