Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Schrödinger's control and estimation paradigm with spatio-temporal distributions on graphs (2312.05679v2)

Published 9 Dec 2023 in eess.SY, cs.SY, math.OC, and math.PR

Abstract: The problem of reconciling a prior probability law on paths with data was introduced by E. Schr\"odinger in 1931/32. It represents an early formulation of a maximum likelihood problem. This specific formulation can also be seen as the control problem to modify the law of a diffusion process so as to match specifications on marginal distributions at given times. Thereby, in recent years, this so-called Schr\"odinger's bridge problem has been at the center of the uncertainty control development. However, an understudied facet of this program has been to address uncertainty in space (state) and time, modeling the effect of tasks being completed contingent on meeting a certain condition at some random time instead of imposing specifications at fixed times. The present work is a study to extend Schr\"odinger's paradigm on such an issue, and herein, it is tackled in the context of random walks on directed graphs. Specifically, we study the case where one marginal is the initial probability distribution on a Markov chain, while others are marginals of stopping (first-arrival) times at absorbing states, signifying completion of tasks. We show when the prior law on paths is Markov, a Markov policy is once again optimal to satisfy those marginal constraints with respect to a likelihood cost following Schr\"odinger's dictum. Based on this, we present the mathematical formulation involving a Sinkhorn-type iteration to construct the optimal probability law on paths matching the spatio-temporal marginals.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. E. Schrödinger, “Über die Umkehrung der Naturgesetze,” Sitzungsberichte der Preuss Akad. Wissen. Phys. Math. Klasse, Sonderausgabe, vol. IX, pp. 144–153, 1931.
  2. E. Schrödinger, “Sur la théorie relativiste de l’électron et l’interprétation de la mécanique quantique,” in Annales de l’institut Henri Poincaré, vol. 2(4), pp. 269–310, Presses universitaires de France, 1932.
  3. H. Cramér, “Sur un nouveau théoreme-limite de la théorie des probabilités,” Actual. Sci. Ind., vol. 736, pp. 5–23, 1938.
  4. I. N. Sanov, On the probability of large deviations of random variables. United States Air Force, Office of Scientific Research, 1958.
  5. Y. Chen, T. T. Georgiou, and M. Pavon, “Stochastic control liaisons: Richard Sinkhorn meets Gaspard Monge on a Schrödinger bridge,” Siam Review, vol. 63, no. 2, pp. 249–313, 2021.
  6. B. Jamison, “Reciprocal processes,” Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 30, pp. 65–86, 1974.
  7. B. Jamison, “The Markov processes of Schrödinger,” Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, vol. 32, no. 4, pp. 323–331, 1975.
  8. H. Föllmer, “Random fields and diffusion processes,” Ecole d’Ete de Probabilites de Saint-Flour XV-XVII, 1985-87, 1988.
  9. H. Föllmer and S. Orey, “Large deviations for the empirical field of a Gibbs measure,” The Annals of Probability, pp. 961–977, 1988.
  10. I. Csiszár, “The method of Types,” IEEE Transactions on Information Theory, vol. 44, no. 6, pp. 2505–2523, 1998.
  11. P. Dai Pra, “A stochastic control approach to reciprocal diffusion processes,” Applied Mathematics and Optimization, vol. 23, no. 1, pp. 313–329, 1991.
  12. E. Todorov, “Efficient computation of optimal actions,” Proceedings of the national academy of sciences, vol. 106, no. 28, pp. 11478–11483, 2009.
  13. I. G. Vladimirov and I. R. Petersen, “State distributions and minimum relative entropy noise sequences in uncertain stochastic systems: The discrete-time case,” SIAM Journal on Control and Optimization, vol. 53, no. 3, pp. 1107–1153, 2015.
  14. Y. Chen, T. T. Georgiou, and M. Pavon, “Optimal steering of a linear stochastic system to a final probability distribution, Part I,” IEEE Transactions on Automatic Control, vol. 61, no. 5, pp. 1158–1169, 2015.
  15. Y. Chen, T. Georgiou, M. Pavon, and A. Tannenbaum, “Robust transport over networks,” IEEE transactions on automatic control, vol. 62, no. 9, pp. 4675–4682, 2016.
  16. Y. Chen, T. T. Georgiou, and M. Pavon, “Controlling uncertainty,” IEEE Control Systems Magazine, vol. 41, no. 4, pp. 82–94, 2021.
  17. A. Wang and H. Wang, “Survey on stochastic distribution systems: A full probability density function control theory with potential applications,” Optimal Control Applications and Methods, vol. 42, no. 6, pp. 1812–1839, 2021.
  18. S. Redner, A guide to first-passage processes. Cambridge university press, 2001.
  19. M. Morse, J. Bell, G. Li, and J. X. Tang, “Flagellar motor switching in caulobacter crescentus obeys first passage time statistics,” Physical review letters, vol. 115, no. 19, p. 198103, 2015.
  20. I. Neri, É. Roldán, and F. Jülicher, “Statistics of infima and stopping times of entropy production and applications to active molecular processes,” Physical Review X, vol. 7, no. 1, p. 011019, 2017.
  21. G. Grimmett and D. Stirzaker, Probability and random processes. Oxford university press, 2020.
  22. S. Kullback and R. A. Leibler, “On information and sufficiency,” The annals of mathematical statistics, vol. 22, no. 1, pp. 79–86, 1951.
  23. T. M. Cover, Elements of information theory. John Wiley & Sons, 1999.
  24. Springer Science & Business Media, 2009.
  25. Y. Chen, T. T. Georgiou, and M. Pavon, “Optimal transport in systems and control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, pp. 89–113, 2021.
  26. T. T. Georgiou and M. Pavon, “Positive contraction mappings for classical and quantum Schrödinger systems,” Journal of Mathematical Physics, vol. 56, no. 3, 2015.
  27. G. Peyré, M. Cuturi, et al., “Computational optimal transport: With applications to data science,” Foundations and Trends® in Machine Learning, vol. 11, no. 5-6, pp. 355–607, 2019.
  28. Y. Chen, T. Georgiou, and M. Pavon, “Entropic and displacement interpolation: a computational approach using the Hilbert metric,” SIAM Journal on Applied Mathematics, vol. 76, no. 6, pp. 2375–2396, 2016.
  29. Y. Chen, T. T. Georgiou, M. Pavon, and A. Tannenbaum, “Relaxed Schrödinger bridges and robust network routing,” IEEE transactions on control of network systems, vol. 7, no. 2, pp. 923–931, 2019.
  30. Y. Chen, T. Georgiou, and M. Pavon, “Optimal steering of inertial particles diffusing anisotropically with losses,” in 2015 American Control Conference (ACC), pp. 1252–1257, IEEE, 2015.
  31. K. Okamoto, M. Goldshtein, and P. Tsiotras, “Optimal covariance control for stochastic systems under chance constraints,” IEEE Control Systems Letters, vol. 2, no. 2, pp. 266–271, 2018.
  32. E. Bakolas, “Finite-horizon covariance control for discrete-time stochastic linear systems subject to input constraints,” Automatica, vol. 91, pp. 61–68, 2018.
  33. K. F. Caluya and A. Halder, “Wasserstein proximal algorithms for the Schrödinger bridge problem: Density control with nonlinear drift,” IEEE Transactions on Automatic Control, vol. 67, no. 3, pp. 1163–1178, 2021.
  34. Y. Chen, T. T. Georgiou, and M. Pavon, “The most likely evolution of diffusing and vanishing particles: Schrödinger bridges with unbalanced marginals,” SIAM Journal on Control and Optimization, vol. 60, no. 4, pp. 2016–2039, 2022.
  35. M. Pavon, G. Trigila, and E. G. Tabak, “The data-driven Schrödinger bridge,” Communications on Pure and Applied Mathematics, vol. 74, no. 7, pp. 1545–1573, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets