Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem (1905.11882v2)

Abstract: We prove several fundamental statistical bounds for entropic OT with the squared Euclidean cost between subgaussian probability measures in arbitrary dimension. First, through a new sample complexity result we establish the rate of convergence of entropic OT for empirical measures. Our analysis improves exponentially on the bound of Genevay et al. (2019) and extends their work to unbounded measures. Second, we establish a central limit theorem for entropic OT, based on techniques developed by Del Barrio and Loubes (2019). Previously, such a result was only known for finite metric spaces. As an application of our results, we develop and analyze a new technique for estimating the entropy of a random variable corrupted by gaussian noise.

• The paper presents exponentially improved sample complexity bounds for entropic optimal transport, demonstrating 1/√n convergence for empirical measures even in high dimensions.
• A central limit theorem is established for entropic optimal transport, extending its use in statistical inference for subgaussian distributions in arbitrary dimensions.
• The developed theory is applied to propose a novel approach for estimating the entropy of Gaussian-corrupted random variables.

Statistical Bounds for Entropic Optimal Transport: Sample Complexity and the Central Limit Theorem

The paper "Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem" by Gonzalo Mena and Jonathan Weed focuses on establishing statistical bounds for entropic optimal transport (OT) through analytical techniques. This advancement enables better understanding and application of entropic OT in various high-dimensional statistical and machine learning contexts.

Key Contributions

1. Sample Complexity: The authors present exponentially improved sample complexity bounds for the entropic OT, a refinement from previous work by Genevay et al. (2019). These bounds allow entropic OT to overcome obstacles like the curse of dimensionality, especially for unbounded measures with subgaussian characteristics. The authors demonstrate that the entropic OT converges at the rate of $1/\sqrt{n}$ for empirical measures, even in high-dimensional spaces.
2. Central Limit Theorem (CLT): A significant contribution of this paper is the establishment of a CLT for entropic OT. By applying methodologies from Del Barrio and Loubes (2019), the paper extends this result to subgaussian probability distributions in arbitrary dimensions. This opens new avenues for using entropic OT in statistical inference, providing asymptotic distributional limits that were previously only known for finitely supported probabilities.
3. Entropy Estimation: As an application of the developed theory, the paper introduces a novel approach to estimate the entropy of a random variable corrupted by Gaussian noise. The results suggest that entropic OT can be leveraged for effective entropy estimation, thus offering insight into theoretical frameworks like the Information Bottleneck Principle in deep learning.

Implications and Future Directions

The developments presented in this paper can profoundly impact theoretical and practical applications in AI and statistics:

• Theoretical Implication: By enabling statistical guarantees in OT, researchers can more reliably apply OT in areas requiring precise statistical measures and complex probability distributions, such as generative modeling and domain adaptation.
• Practical Utilization: The bounded sample complexity and established CLTs enhance the reliability of entropic OT in dynamic environments where empirical measures are studied, simplifying tasks like image recognition and data generation.
• Future Research: Beyond immediate applications, these findings invite further exploration into the optimization of entropic OT algorithms for large-scale datasets. There's a potential for adapting these results to other types of regularized transport costs and elucidating their behavior in non-Euclidean spaces.

Conclusion

This paper presents a crucial step forward in the statistical analysis of entropic OT, providing robust sample complexity bounds and a CLT for OT over subgaussian measures. These advancements are not only significant in overcoming computational challenges associated with high-dimensional data analysis but also in providing theoretical foundations for applied machine learning techniques. This work lays groundwork for extensive future exploration across statistical and machine learning methodologies.