Distribution-robust mean estimation via smoothed random perturbations (1906.10300v1)

Abstract: We consider the problem of mean estimation assuming only finite variance. We study a new class of mean estimators constructed by integrating over random noise applied to a soft-truncated empirical mean estimator. For appropriate choices of noise, we show that this can be computed in closed form, and utilizing relative entropy inequalities, these estimators enjoy deviations with exponential tails controlled by the second moment of the underlying distribution. We consider both additive and multiplicative noise, and several noise distribution families in our analysis. Furthermore, we empirically investigate the sensitivity to the mean-standard deviation ratio for numerous concrete manifestations of the estimator class of interest. Our main take-away is that an inexpensive new estimator can achieve nearly sub-Gaussian performance for a wide variety of data distributions.