Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Sinkhorn-type Algorithm for Constrained Optimal Transport (2403.05054v1)

Published 8 Mar 2024 in math.OC and cs.LG

Abstract: Entropic optimal transport (OT) and the Sinkhorn algorithm have made it practical for machine learning practitioners to perform the fundamental task of calculating transport distance between statistical distributions. In this work, we focus on a general class of OT problems under a combination of equality and inequality constraints. We derive the corresponding entropy regularization formulation and introduce a Sinkhorn-type algorithm for such constrained OT problems supported by theoretical guarantees. We first bound the approximation error when solving the problem through entropic regularization, which reduces exponentially with the increase of the regularization parameter. Furthermore, we prove a sublinear first-order convergence rate of the proposed Sinkhorn-type algorithm in the dual space by characterizing the optimization procedure with a Lyapunov function. To achieve fast and higher-order convergence under weak entropy regularization, we augment the Sinkhorn-type algorithm with dynamic regularization scheduling and second-order acceleration. Overall, this work systematically combines recent theoretical and numerical advances in entropic optimal transport with the constrained case, allowing practitioners to derive approximate transport plans in complex scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. David J Aldous. The ζ𝜁\zetaitalic_ζ (2) limit in the random assignment problem. Random Structures & Algorithms, 18(4):381–418, 2001.
  2. Near-linear time approximation algorithms for optimal transport via sinkhorn iteration. Advances in neural information processing systems, 30, 2017.
  3. Massively scalable sinkhorn distances via the nyström method. Advances in neural information processing systems, 32, 2019.
  4. Structured optimal transport. In International conference on artificial intelligence and statistics, pages 1771–1780. PMLR, 2018.
  5. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214–223. PMLR, 2017.
  6. Model-independent bounds for option prices—a mass transport approach. Finance and Stochastics, 17:477–501, 2013.
  7. Iterative bregman projections for regularized transportation problems. SIAM Journal on Scientific Computing, 37(2):A1111–A1138, 2015.
  8. Wasserstein barycentric coordinates: histogram regression using optimal transport. ACM Trans. Graph., 35(4):71–1, 2016.
  9. From optimal transport to generative modeling: the vegan cookbook. arXiv preprint arXiv:1705.07642, 2017.
  10. Convex optimization. Cambridge university press, 2004.
  11. Optimal-transport formulation of electronic density-functional theory. Physical Review A, 85(6):062502, 2012.
  12. Exponential convergence of sinkhorn under regularization scheduling. In SIAM Conference on Applied and Computational Discrete Algorithms (ACDA23), pages 180–188. SIAM, 2023.
  13. Graph optimal transport for cross-domain alignment. In International Conference on Machine Learning, pages 1542–1553. PMLR, 2020.
  14. Joint distribution optimal transportation for domain adaptation. Advances in neural information processing systems, 30, 2017.
  15. Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
  16. Rachid Deriche. Recursively implementating the Gaussian and its derivatives. PhD thesis, INRIA, 1993.
  17. Max-sliced wasserstein distance and its use for gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10648–10656, 2019.
  18. Martingale optimal transport and robust hedging in continuous time. Probability Theory and Related Fields, 160(1-2):391–427, 2014.
  19. Computational optimal transport: Complexity by accelerated gradient descent is better than by sinkhorn’s algorithm. In International conference on machine learning, pages 1367–1376. PMLR, 2018.
  20. S C Fang. An unconstrained convex programming view of linear programming. Zeitschrift für Operations Research, 36:149–161, 1992.
  21. Unbalanced minibatch optimal transport; applications to domain adaptation. In International Conference on Machine Learning, pages 3186–3197. PMLR, 2021.
  22. Unsupervised visual domain adaptation using subspace alignment. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2013.
  23. A stochastic control approach to no-arbitrage bounds given marginals, with an application to lookback options. The Annals of Applied Probability, 24(1):312 – 336, 2014.
  24. Optimal maps for the multidimensional monge-kantorovich problem. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 51(1):23–45, 1998.
  25. Stochastic optimization for large-scale optimal transport. Advances in neural information processing systems, 29, 2016.
  26. Gan and vae from an optimal transport point of view. arXiv preprint arXiv:1706.01807, 2017.
  27. Learning generative models with sinkhorn divergences. In International Conference on Artificial Intelligence and Statistics, pages 1608–1617. PMLR, 2018.
  28. Sample complexity of sinkhorn divergences. In The 22nd international conference on artificial intelligence and statistics, pages 1574–1583. PMLR, 2019.
  29. Matrix computations. JHU press, 2013.
  30. Computational methods for martingale optimal transport problems. The Annals of Applied Probability, 29(6):3311–3347, 2019.
  31. Geodesic sinkhorn for fast and accurate optimal transport on manifolds. ArXiv, 2023.
  32. Otlda: A geometry-aware optimal transport approach for topic modeling. Advances in Neural Information Processing Systems, 33:18573–18582, 2020.
  33. Interpretable distribution features with maximum testing power. Advances in Neural Information Processing Systems, 29, 2016.
  34. Efficient and accurate optimal transport with mirror descent and conjugate gradients. arXiv preprint arXiv:2307.08507, 2023.
  35. Semidefinite relaxation of multimarginal optimal transport for strictly correlated electrons in second quantization. SIAM Journal on Scientific Computing, 42(6):B1462–B1489, 2020.
  36. Optimal mass transport: Signal processing and machine-learning applications. IEEE signal processing magazine, 34(4):43–59, 2017.
  37. Optimal transportation with capacity constraints. Transactions of the American Mathematical Society, 367(3):1501–1521, 2015.
  38. Insights into capacity-constrained optimal transport. Proceedings of the National Academy of Sciences, 110(25):10064–10067, 2013.
  39. Nonequispaced fast fourier transform boost for the sinkhorn algorithm. arXiv preprint arXiv:2201.07524, 2022.
  40. A geometric view of optimal transportation and generative model. Computer Aided Geometric Design, 68:1–21, 2019.
  41. Importance sparsification for sinkhorn algorithm. arXiv preprint arXiv:2306.06581, 2023.
  42. On efficient optimal transport: An analysis of greedy and accelerated mirror descent algorithms. In International Conference on Machine Learning, pages 3982–3991. PMLR, 2019.
  43. A deterministic strongly polynomial algorithm for matrix scaling and approximate permanents. In Proceedings of the thirtieth annual ACM symposium on Theory of computing, pages 644–652, 1998.
  44. Tie-Yan Liu et al. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval, 3(3):225–331, 2009.
  45. Linear and nonlinear programming, volume 2. Springer, 1984.
  46. Differential properties of sinkhorn approximation for learning with wasserstein distance. Advances in Neural Information Processing Systems, 31, 2018.
  47. Christopher D Manning. An introduction to information retrieval. Cambridge university press, 2009.
  48. Learning latent permutations with gumbel-sinkhorn networks. arXiv preprint arXiv:1802.08665, 2018.
  49. On the solution of the random link matching problems. Journal de Physique, 48(9):1451–1459, 1987.
  50. Yurii Evgen’evich Nesterov. A method of solving a convex programming problem with convergence rate o\\\backslash\bigl(k^2\\\backslash\bigr). In Doklady Akademii Nauk, volume 269, pages 543–547. Russian Academy of Sciences, 1983.
  51. Improving mini-batch optimal transport via partial transportation. In International Conference on Machine Learning, pages 16656–16690. PMLR, 2022.
  52. Most: Multi-source domain adaptation via optimal transport for student-teacher learning. In Uncertainty in Artificial Intelligence, pages 225–235. PMLR, 2021.
  53. Exploiting mmd and sinkhorn divergences for fair and transferable representation learning. Advances in Neural Information Processing Systems, 33:15360–15370, 2020.
  54. Ot-flow: Fast and accurate continuous normalizing flows via optimal transport. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9223–9232, 2021.
  55. Brendan Pass. Multi-marginal optimal transport: theory and applications. ESAIM: Mathematical Modelling and Numerical Analysis-Modélisation Mathématique et Analyse Numérique, 49(6):1771–1790, 2015.
  56. Sinkhorn autoencoders. In Uncertainty in Artificial Intelligence, pages 733–743. PMLR, 2020.
  57. Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019.
  58. Theoretical analysis of domain adaptation with optimal transport. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part II 10, pages 737–753. Springer, 2017.
  59. Improving gans using optimal transport. arXiv preprint arXiv:1803.05573, 2018.
  60. Nonnegative matrix factorization with earth mover’s distance metric for image analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8):1590–1602, 2011.
  61. On the convergence and robustness of training gans with regularized optimal transport. Advances in Neural Information Processing Systems, 31, 2018.
  62. Linear time sinkhorn divergences using positive features. Advances in Neural Information Processing Systems, 33:13468–13480, 2020.
  63. Richard Sinkhorn. A relationship between arbitrary positive matrices and doubly stochastic matrices. The annals of mathematical statistics, 35(2):876–879, 1964.
  64. Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. ACM Transactions on Graphics (ToG), 34(4):1–11, 2015.
  65. J Michael Steele. Probability theory and combinatorial optimization. SIAM, 1997.
  66. Optimal transportation under controlled stochastic dynamics. The Annals of Probability, 41(5):3201 – 3240, 2013.
  67. Accelerating sinkhorn algorithm with sparse newton iterations. To appear in ICLR 2024, 2024.
  68. Multi-source domain adaptation via weighted joint distributions optimal transport. In Uncertainty in Artificial Intelligence, pages 1970–1980. PMLR, 2022.
  69. Optimal transport for structured data with application on graphs. arXiv preprint arXiv:1805.09114, 2018.
  70. Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009.
  71. Jonathan Weed. An explicit analysis of the entropic penalty in linear programming. In Conference On Learning Theory, pages 1841–1855. PMLR, 2018.
  72. Unsupervised domain adaptation via deep hierarchical optimal transport. arXiv preprint arXiv:2211.11424, 2022.
  73. G Udny Yule. On the methods of measuring association between two attributes. Journal of the Royal Statistical Society, 75(6):579–652, 1912.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets