Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On the potential benefits of entropic regularization for smoothing Wasserstein estimators (2210.06934v3)

Published 13 Oct 2022 in stat.ML, stat.AP, and stat.ME

Abstract: This paper is focused on the study of entropic regularization in optimal transport as a smoothing method for Wasserstein estimators, through the prism of the classical tradeoff between approximation and estimation errors in statistics. Wasserstein estimators are defined as solutions of variational problems whose objective function involves the use of an optimal transport cost between probability measures. Such estimators can be regularized by replacing the optimal transport cost by its regularized version using an entropy penalty on the transport plan. The use of such a regularization has a potentially significant smoothing effect on the resulting estimators. In this work, we investigate its potential benefits on the approximation and estimation properties of regularized Wasserstein estimators. Our main contribution is to discuss how entropic regularization may reach, at a lower computational cost, statistical performances that are comparable to those of un-regularized Wasserstein estimators in statistical learning problems involving distributional data analysis. To this end, we present new theoretical results on the convergence of regularized Wasserstein estimators. We also study their numerical performances using simulated and real data in the supervised learning problem of proportions estimation in mixture models using optimal transport.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Critical assessment of automated flow cytometry data analysis techniques. Nature methods, 10(3):228, 2013.
  2. Near-linear time approximation algorithms for optimal transport via sinkhorn iteration. Advances in neural information processing systems, 30, 2017.
  3. Wasserstein generative adversarial networks. In International Conference on Machine Learning, pages 214–223, 2017.
  4. Stochastic optimization for regularized wasserstein estimators. In International Conference on Machine Learning, pages 602–612. PMLR, 2020.
  5. B. Bercu and J. Bigot. Asymptotic distribution and convergence rates of stochastic algorithms for entropic optimal transportation between probability measures. The Annals of Statistics, 49(2):968 – 987, 2021.
  6. On parameter estimation with the Wasserstein distance. Information and Inference: A Journal of the IMA, 8(4):657–676, 2019.
  7. J. Bigot. Statistical data analysis in the Wasserstein space. ESAIM: ProcS, 68:1–19, 2020.
  8. Data-driven regularization of Wasserstein barycenters with an application to multivariate density registration. Information and Inference: A Journal of the IMA, 8, 04 2018.
  9. Central limit theorems for entropy-regularized optimal transport on finite spaces and statistical applications. Electronic Journal of Statistics, 13(2):5120 – 5150, 2019. doi: 10.1214/19-EJS1637. URL https://doi.org/10.1214/19-EJS1637.
  10. Supplement to "on the potential benefits of entropic regularization for smoothing wasserstein estimators", 2022.
  11. Computational resources for high-dimensional immune analysis from the human immunology project consortium. Nature biotechnology, 32(2):146, 2014.
  12. Current trends in flow cytometry automated data analysis software. Cytometry Part A, 99(10):1007–1021, 2021.
  13. An entropic generalization of Caffarelli’s contraction theorem via covariance inequalities. arXiv preprint arXiv:2203.04954, 2022.
  14. Faster Wasserstein Distance Estimation with the Sinkhorn Divergence. In Proc. NeurIPS’20, 2020.
  15. Lénaïc Chizat. Doubly Regularized Entropic Wasserstein Barycenters, 2023.
  16. M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in neural information processing systems, pages 2292–2300, 2013.
  17. Computational optimal transport: Complexity by accelerated gradient descent is better than by sinkhorn algorithm. In International conference on machine learning, pages 1367–1376. PMLR, 2018.
  18. Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2681–2690, 2019.
  19. N. Fournier and A. Guillin. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3):707–738, 2015.
  20. Cytopt: Optimal transport with domain adaptation for interpreting flow cytometry data. The Annals of Applied Statistics, 17(2):1086–1104, 2023.
  21. G. Biau and M. Sangnier and U. Tanielian. Some Theoretical Insights into Wasserstein GANs. Journal of Machine Learning Research, 22(119):1–45, 2021. URL http://jmlr.org/papers/v22/20-553.html.
  22. Stochastic optimization for large-scale optimal transport. In Advances in neural information processing systems, pages 3440–3448, 2016.
  23. Learning generative models with sinkhorn divergences. In Amos Storkey and Fernando Perez-Cruz, editors, Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, volume 84 of Proceedings of Machine Learning Research, pages 1608–1617. PMLR, 2018.
  24. Sample complexity of sinkhorn divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1574–1583. PMLR, 2019.
  25. Michael Hardy. Combinatorics of Partial Derivatives. The Electronic Journal of Combinatorics, pages R1–R1, 2006. ISSN 1077-8926. doi: 10.37236/1027.
  26. Optimal transport improves cell-cell similarity inference in single-cell omics data. bioRxiv, 2021.
  27. Entropy-regularized optimal transport generative models. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3532–3536. IEEE, 2019.
  28. T. Manole and J Niles-Weed. Sharp convergence rates for empirical optimal transport with smooth costs. arXiv preprint arXiv:2106.13181, 2021.
  29. G. Mena and J. Niles-Weed. Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. Advances in Neural Information Processing Systems, 32, 2019.
  30. V. M. Panaretos and Y. Zemel. Statistical Aspects of Wasserstein Distances. Annual Reviews of Statistics and its Applications, 6:405–431, 2018.
  31. Fast and robust earth mover’s distances. In 2009 IEEE 12th international conference on computer vision, pages 460–467. IEEE, 2009.
  32. Modeling Probability Density Functions as Data Objects. Econometrics and Statistics, 21(C):159–178, 2022.
  33. Computational optimal transport. Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019.
  34. On the convergence and robustness of training gans with regularized optimal transport. In Advances in Neural Information Processing Systems, pages 7091–7101, 2018.
  35. F. Santambrogio. Optimal transport for applied mathematicians. Birkäuser, NY, 55(58-63):94, 2015.
  36. Weak Convergence and Empirical Processes. With Applications to Statistics. New York: Springer, 1996.
  37. R. van Handel. Probability in High Dimension. Princeton University, 2016.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now