Papers
Topics
Authors
Recent
Search
2000 character limit reached

Wasserstein Proximal Coordinate Gradient Algorithms

Published 7 May 2024 in math.OC, math.ST, and stat.TH | (2405.04628v2)

Abstract: Motivated by approximation Bayesian computation using mean-field variational approximation and the computation of equilibrium in multi-species systems with cross-interaction, this paper investigates the composite geodesically convex optimization problem over multiple distributions. The objective functional under consideration is composed of a convex potential energy on a product of Wasserstein spaces and a sum of convex self-interaction and internal energies associated with each distribution. To efficiently solve this problem, we introduce the Wasserstein Proximal Coordinate Gradient (WPCG) algorithms with parallel, sequential, and random update schemes. Under a quadratic growth (QG) condition that is weaker than the usual strong convexity requirement on the objective functional, we show that WPCG converges exponentially fast to the unique global optimum. In the absence of the QG condition, WPCG is still demonstrated to converge to the global optimal solution, albeit at a slower polynomial rate. Numerical results for both motivating examples are consistent with our theoretical findings.

Authors (3)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Gradient flows of probability measures. Handbook of differential equations: evolutionary equations, 3:1–136, 2007.
  2. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2005.
  3. Input convex neural networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 146–155. PMLR, 06–11 Aug 2017. URL https://proceedings.mlr.press/v70/amos17b.html.
  4. Donald G Aronson. The porous medium equation. Nonlinear Diffusion Problems: Lectures given at the 2nd 1985 Session of the Centro Internazionale Matermatico Estivo (CIME) held at Montecatini Terme, Italy June 10–June 18, 1985, pages 1–46, 2006.
  5. Mirror sinkhorn: Fast online optimization on transport polytopes. arXiv preprint arXiv:2211.10420, 2022.
  6. Amir Beck. First-order methods in optimization. SIAM, 2017.
  7. Pattern recognition and machine learning, volume 4. Springer, 2006.
  8. Existence and uniqueness of equilibrium for a spatial model of social interactions. International Economic Review, 57(1):31–60, 2016.
  9. François Bolley. Separability and completeness for the wasserstein distance. In Séminaire de probabilités XLI, pages 371–377. Springer, 2008.
  10. Yann Brenier. Décomposition polaire et réarrangement monotone des champs de vecteurs. CR Acad. Sci. Paris Sér. I Math., 305:805–808, 1987.
  11. Numerical solution for an aggregation equation with degenerate diffusion. Applied Mathematics and Computation, 377:125145, 2020.
  12. New efficient algorithm for solving thermodynamic chemistry. AIChE Journal, 48(4):894–904, 2002.
  13. Zoology of a nonlocal cross-diffusion model for two species. SIAM Journal on Applied Mathematics, 78(2):1078–1104, 2018.
  14. Aggregation-diffusion equations: dynamics, asymptotics, and singular limits. Active Particles, Volume 2: Advances in Theory, Models, and Applications, pages 65–108, 2019.
  15. Gradient descent algorithms for bures-wasserstein barycenters. In Conference on Learning Theory, pages 1276–1304. PMLR, 2020.
  16. Random-batch method for multi-species stochastic interacting particle systems. Journal of Computational Physics, 463:111220, 2022.
  17. Central limit theorems for empirical transportation cost in general dimension. The Annals of Probability, 47(2):926–951, 2019.
  18. Measure solutions for non-local interaction pdes with two species. Nonlinearity, 26(10):2777, 2013.
  19. Nonasymptotic convergence analysis for the unadjusted langevin algorithm. 2017.
  20. Equilibria for an aggregation model with two species. SIAM Journal on Applied Dynamical Systems, 16(4):2287–2338, 2017.
  21. Gerald B Folland. Real analysis: modern techniques and their applications, volume 40. John Wiley & Sons, 1999.
  22. On representations of mean-field variational inference. arXiv preprint arXiv:2210.11385, 2022.
  23. Online learning to transport via the minimal selection principle. In Conference on Learning Theory, pages 4085–4109. PMLR, 2022.
  24. Computing equilibrium measures with power law kernels. Mathematics of Computation, 91(337):2247–2281, 2022.
  25. Iteration complexity of a block coordinate gradient descent method for convex optimization. SIAM Journal on Optimization, 25(3):1298–1313, 2015.
  26. Random batch methods (rbm) for interacting particle systems. Journal of Computational Physics, 400:108877, 2020.
  27. The variational formulation of the fokker–planck equation. SIAM journal on mathematical analysis, 29(1):1–17, 1998.
  28. Linear convergence of gradient and proximal-gradient methods under the polyak-łojasiewicz condition. In Joint European conference on machine learning and knowledge discovery in databases, pages 795–811. Springer, 2016.
  29. Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. The Annals of Mathematical Statistics, pages 887–906, 1956.
  30. Variational inference via wasserstein gradient flows. arXiv preprint arXiv:2205.15902, 2022.
  31. Towards a mathematical theory of trajectory inference. arXiv preprint arXiv:2102.09204, 2021.
  32. Wellposedness and regularity of solutions of an aggregation equation. Revista Matemática Iberoamericana, 26(1):261–294, 2010.
  33. On faster convergence of cyclic block coordinate descent-type methods for strongly convex minimization. The Journal of Machine Learning Research, 18(1):6741–6764, 2017.
  34. The landscape of empirical risk for non-convex losses. arXiv preprint arXiv:1607.06534, 2016.
  35. Large-scale wasserstein gradient flows. Advances in Neural Information Processing Systems, 34:15243–15256, 2021.
  36. Felix Otto. The geometry of dissipative evolution equations: the porous medium equation. 2001.
  37. Computational study of multi-species fractional reaction-diffusion system with abc operator. Chaos, Solitons & Fractals, 128:280–289, 2019.
  38. Computing multi-species chemical equilibrium with an algorithm based on the reaction extents. Computers & chemical engineering, 58:135–143, 2013.
  39. Zehra Pinar. The reaction–cross-diffusion models for tissue growth. Mathematical Methods in the Applied Sciences, 44(18):13805–13811, 2021.
  40. R Tyrrell Rockafellar. Convex analysis, volume 11. Princeton university press, 1997.
  41. The wasserstein proximal gradient algorithm. Advances in Neural Information Processing Systems, 33:12356–12366, 2020.
  42. Filippo Santambrogio. Optimal transport for applied mathematicians. Birkäuser, NY, 55(58-63):94, 2015.
  43. Using the adap learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the annual symposium on computer application in medical care, page 261. American Medical Informatics Association, 1988.
  44. Worst-case complexity of cyclic coordinate descent: o⁢(n2)𝑜superscript𝑛2o(n^{2})italic_o ( italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ) gap with randomized version. Mathematical Programming, 185:487–520, 2021.
  45. The bayesian update: variational formulations and gradient flows. Bayesian Analysis, 15(1):29–56, 2020.
  46. Juan Luis Vázquez. The porous medium equation: mathematical theory. Oxford University Press on Demand, 2007.
  47. Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
  48. Cédric Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021.
  49. Andre Wibisono. Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem. In Conference on Learning Theory, pages 2093–3027. PMLR, 2018.
  50. Stephen J Wright. Coordinate descent algorithms. Mathematical Programming, 151(1):3–34, 2015.
  51. Optimization for data analysis. Cambridge University Press, 2022.
  52. Learning gaussian mixtures using the wasserstein-fisher-rao gradient flow. arXiv preprint arXiv:2301.01766, 2023.
  53. Mean field variational inference via wasserstein gradient flow. arXiv preprint arXiv:2207.08074, 2022.
  54. Minimizing convex functionals over space of probability measures via kl divergence gradient flow. arXiv preprint arXiv:2311.00894, 2023.
  55. Analysis of degenerate cross-diffusion population models with volume filling. In Annales de l’Institut Henri Poincaré C, Analyse Non Linéaire, volume 34, pages 1–29. Elsevier, 2017.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 7 likes about this paper.

Sign Up for Free