Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

An inexact Bregman proximal point method and its acceleration version for unbalanced optimal transport (2402.16978v1)

Published 26 Feb 2024 in math.OC and cs.LG

Abstract: The Unbalanced Optimal Transport (UOT) problem plays increasingly important roles in computational biology, computational imaging and deep learning. Scaling algorithm is widely used to solve UOT due to its convenience and good convergence properties. However, this algorithm has lower accuracy for large regularization parameters, and due to stability issues, small regularization parameters can easily lead to numerical overflow. We address this challenge by developing an inexact Bregman proximal point method for solving UOT. This algorithm approximates the proximal operator using the Scaling algorithm at each iteration. The algorithm (1) converges to the true solution of UOT, (2) has theoretical guarantees and robust regularization parameter selection, (3) mitigates numerical stability issues, and (4) can achieve comparable computational complexity to the Scaling algorithm in specific practice. Building upon this, we develop an accelerated version of inexact Bregman proximal point method for solving UOT by using acceleration techniques of Bregman proximal point method and provide theoretical guarantees and experimental validation of convergence and acceleration.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. Cédric Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021.
  2. Yann Brenier. Polar factorization and monotone rearrangement of vector-valued functions. Communications on pure and applied mathematics, 44(4):375–417, 1991.
  3. Convex color image segmentation with optimal transport distances. In Scale Space and Variational Methods in Computer Vision: 5th International Conference, SSVM 2015, Lège-Cap Ferret, France, May 31-June 4, 2015, Proceedings 5, pages 256–269. Springer, 2015.
  4. The earth mover’s distance as a metric for image retrieval. International journal of computer vision, 40:99–121, 2000.
  5. Wasserstein propagation for semi-supervised learning. In International Conference on Machine Learning, pages 306–314. PMLR, 2014.
  6. Learning with a wasserstein loss. Advances in neural information processing systems, 28, 2015.
  7. Optimal-transport analysis of single-cell gene expression identifies developmental trajectories in reprogramming. Cell, 176(4):928–943, 2019.
  8. Parallel unbalanced optimal transport regularization for large scale imaging problems. arXiv preprint arXiv:1909.00149, 2019.
  9. Application of an unbalanced optimal transport distance and a mixed l1/wasserstein distance to full waveform inversion. Geophysical Journal International, 230(2):1338–1357, 2022.
  10. Scalable unbalanced optimal transport using generative adversarial networks. arXiv preprint arXiv:1810.11447, 2018.
  11. Spatio-temporal alignments: Optimal transport through space and time. In International Conference on Artificial Intelligence and Statistics, pages 1695–1704. PMLR, 2020.
  12. Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
  13. Scaling algorithms for unbalanced optimal transport problems. Mathematics of Computation, 87(314):2563–2609, 2018.
  14. Dykstras algorithm with bregman projections: A convergence proof. Optimization, 48(4):409–427, 2000.
  15. Iterative bregman projections for regularized transportation problems. SIAM Journal on Scientific Computing, 37(2):A1111–A1138, 2015.
  16. Near-linear time approximation algorithms for optimal transport via sinkhorn iteration. Advances in neural information processing systems, 30, 2017.
  17. On unbalanced optimal transport: An analysis of sinkhorn algorithm. In International Conference on Machine Learning, pages 7673–7682. PMLR, 2020.
  18. A fast proximal point method for computing exact wasserstein distance. In Uncertainty in artificial intelligence, pages 433–453. PMLR, 2020.
  19. Bregman proximal point algorithm revisited: A new inexact version and its inertial variant. SIAM Journal on Optimization, 32(3):1523–1554, 2022.
  20. Yurii Nesterov. On an approach to the construction of optimal methods of minimization of smooth convex functions. Ekonomika i Mateaticheskie Metody, 24(3):509–517, 1988.
  21. Yurii Evgen’evich Nesterov. A method of solving a convex programming problem with convergence rate o\\\backslash\bigl(k^2\\\backslash\bigr). In Doklady Akademii Nauk, volume 269, pages 543–547. Russian Academy of Sciences, 1983.
  22. Osman Güler. New proximal point algorithms for convex minimization. SIAM Journal on Optimization, 2(4):649–664, 1992.
  23. Bregman augmented lagrangian and its acceleration. arXiv preprint arXiv:2002.06315, 2020.
  24. Accelerated bregman proximal gradient methods for relatively smooth convex optimization. Computational Optimization and Applications, 79:405–440, 2021.
  25. An efficient implementable inexact entropic proximal point algorithm for a class of linear programming problems. Computational Optimization and Applications, 85(1):107–146, 2023.
  26. B Lemaire. On the convergence of some iterative methods for convex minimization. In Recent Developments in Optimization: Seventh French-German Conference on Optimization, pages 252–268. Springer, 1995.
  27. Krzysztof C Kiwiel. Proximal minimization methods with generalized bregman functions. SIAM journal on control and optimization, 35(4):1142–1168, 1997.
  28. Marc Teboulle. Convergence of proximal-like algorithms. SIAM Journal on Optimization, 7(4):1069–1083, 1997.
  29. A unified framework for some inexact proximal point algorithms. Numerical functional analysis and optimization, 22(7-8):1013–1035, 2001.
  30. Convergence rates of inexact proximal-gradient methods for convex optimization. Advances in neural information processing systems, 24, 2011.
  31. Unbalanced optimal transport through non-negative penalized linear regression. Advances in Neural Information Processing Systems, 34:23270–23282, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Xiang Chen (344 papers)
  2. Faqiang Wang (10 papers)
  3. Jun Liu (606 papers)
  4. Li Cui (14 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now