Mean Estimation with Sub-Gaussian Rates in Polynomial Time

Abstract: We study polynomial time algorithms for estimating the mean of a heavy-tailed multivariate random vector. We assume only that the random vector $X$ has finite mean and covariance. In this setting, the radius of confidence intervals achieved by the empirical mean are large compared to the case that $X$ is Gaussian or sub-Gaussian. We offer the first polynomial time algorithm to estimate the mean with sub-Gaussian-size confidence intervals under such mild assumptions. Our algorithm is based on a new semidefinite programming relaxation of a high-dimensional median. Previous estimators which assumed only existence of finitely-many moments of $X$ either sacrifice sub-Gaussian performance or are only known to be computable via brute-force search procedures requiring time exponential in the dimension.

• The paper introduces the first polynomial-time algorithm to estimate multivariate means with sub-Gaussian confidence intervals under minimal assumptions.
• It leverages semidefinite programming and the sum of squares framework to efficiently approximate high-dimensional medians.
• The method enables robust mean estimation for heavy-tailed distributions, matching the tight confidence bounds of the Gaussian setting.

Mean Estimation with Sub-Gaussian Rates in Polynomial Time

The paper "Mean estimation with sub-Gaussian rates in polynomial time" by Samuel B. Hopkins presents a significant advancement in the field of statistics and computer science by introducing the first polynomial-time algorithm capable of efficiently estimating the mean of a multivariate distribution with heavy tails. This endeavor addresses one of the more challenging problems in high-dimensional statistics: achieving confidence intervals similar to those in the Gaussian setting under minimal assumptions of finite mean and covariance.

The key contribution of the paper is an algorithm that offers confidence intervals of sub-Gaussian size in polynomial time. The existing methods, prior to this work, either required exponential computation time or could not achieve such tight confidence bounds under minimal assumptions. The author provides a method based on semidefinite programming (SDP) and specifically leverages the sum of squares (SoS) technique to design the estimator. These tools are employed to create a semidefinite programming relaxation that reliably approximates a high-dimensional median, not by directly computing the mean but by finding a point in space that lies within a median-appropriate range in every directional projection.

The problem context for the algorithm is where the random vector $X$ has finite mean and covariance, but may exhibit heavy-tailed behavior where traditional Gaussian assumptions break down. In standard scenarios, large confidence intervals are typically expected when empirical means are used for estimation; however, for sub-Gaussian distributions, smaller intervals are possible. The paper makes bold progress by matching these results even for distributions with possibly heavy tails.

The technical strength lies in the SDP and the SoS framework, which not only facilitates the construction of the estimator but also ensures it can be computed efficiently in polynomial time. Prior works could only assure performance through computationally intensive, brute-force search procedures across dimensions.

Key results include a demonstration that the population mean $\mu$ is $(r, p)$-central with respect to i.i.d. samples with a high probability, supporting the tight bound claim for the confidence interval achievable by the proposed method. The implications for practical algorithms and theoretical bounds present novel pathways for dealing with high-dimensional data where heavy-tailed distributions are present. This work strengthens the arsenal available for robust estimation under relaxed assumptions, pertinent to many applications in machine learning, data science, and econometrics.

In conclusion, this paper paves the way for further research into practical implementations of algorithms that were largely considered computationally infeasible before the introduction of this approach. Future developments could focus on enhancing the runtime efficiency of the solution or extending the technique to other statistical estimation problems. The integration of SoS methods with statistical estimations demonstrates a promising symbiosis between computational capabilities and statistical rigor.