Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 134 tok/s
Gemini 2.5 Pro 41 tok/s Pro
GPT-5 Medium 28 tok/s Pro
GPT-5 High 39 tok/s Pro
GPT-4o 101 tok/s Pro
Kimi K2 191 tok/s Pro
GPT OSS 120B 428 tok/s Pro
Claude Sonnet 4.5 37 tok/s Pro
2000 character limit reached

Discrete-Time Maximum Likelihood Neural Distribution Steering (2409.02272v1)

Published 3 Sep 2024 in eess.SY and cs.SY

Abstract: This paper studies the problem of steering the distribution of a discrete-time dynamical system from an initial distribution to a target distribution in finite time. The formulation is fully nonlinear, allowing the use of general control policies, parametrized by neural networks. Although similar solutions have been explored in the continuous-time context, extending these techniques to systems with discrete dynamics is not trivial. The proposed algorithm results in a regularized maximum likelihood optimization problem, which is solved using machine learning techniques. After presenting the algorithm, we provide several numerical examples that illustrate the capabilities of the proposed method. We start from a simple problem that admits a solution through semidefinite programming, serving as a benchmark for the proposed approach. Then, we employ the framework in more general problems that cannot be solved using existing techniques, such as problems with non-Gaussian boundary distributions and non-linear dynamics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. L. Ruthotto and E. Haber, “An introduction to deep generative modeling,” GAMM-Mitteilungen, vol. 44, no. 2, p. e202100008, 2021.
  2. K. F. Caluya and A. Halder, “Wasserstein proximal algorithms for the Schrödinger bridge problem: Density control with nonlinear drift,” Transactions on Automatic Control, vol. 67, no. 3, pp. 1163–1178, 2021.
  3. Y. Chen, T. T. Georgiou, and M. Pavon, “Controlling uncertainty,” Control Systems Magazine, vol. 41, no. 4, pp. 82–94, 2021.
  4. A. D. Saravanos, Y. Li, and E. A. Theodorou, “Distributed hierarchical distribution control for very-large-scale clustered multi-agent systems,” in Robotics: Science and Systems XIX, (Daegu, Republic of Korea), July 2023.
  5. A. D. Saravanos, A. Tsolovikos, E. Bakolas, and E. Theodorou, “Distributed Covariance Steering with Consensus ADMM for Stochastic Multi-Agent Systems,” in Proceedings of Robotics: Science and Systems, (Virtual), July 2021.
  6. L. Ruthotto, S. J. Osher, W. Li, L. Nurbekyan, and S. W. Fung, “A machine learning framework for solving high-dimensional mean field game and mean field control problems,” Proceedings of the National Academy of Sciences, vol. 117, no. 17, pp. 9183–9193, 2020.
  7. Y. Chen, “Density control of interacting agent systems,” Transactions on Automatic Control, vol. 69, no. 1, pp. 246–260, 2024.
  8. G.-H. Liu, T. Chen, O. So, and E. Theodorou, “Deep generalized Schrödinger bridge,” in Advances in Neural Information Processing Systems, vol. 35, (Louisiana, LA), pp. 9374–9388, Curran Associates, Inc., 2022.
  9. J. Knaup, K. Okamoto, and P. Tsiotras, “Safe high-performance autonomous off-road driving using covariance steering stochastic model predictive control,” Transactions on Control Systems Technology, vol. 31, no. 5, pp. 2066–2081, 2023.
  10. A. D. Saravanos, I. M. Balci, E. Bakolas, and E. A. Theodorou, “Distributed model predictive covariance steering,” arXiv preprint arXiv:2212.00398, 2022.
  11. Y. Chen, T. T. Georgiou, and M. Pavon, “Optimal transport over a linear dynamical system,” Transactions on Automatic Control, vol. 62, no. 5, pp. 2137–2152, 2016.
  12. E. Bakolas, “Finite-horizon covariance control for discrete-time stochastic linear systems subject to input constraints,” Automatica, vol. 91, pp. 61–68, 2018.
  13. F. Liu, G. Rapakoulias, and P. Tsiotras, “Optimal covariance steering for discrete-time linear stochastic systems,” arXiv preprint arXiv:2211.00618, 2022.
  14. I. M. Balci and E. Bakolas, “Exact SDP formulation for discrete-time covariance steering with Wasserstein terminal cost,” arXiv preprint arXiv:2205.10740, 2022.
  15. Y. Chen, T. T. Georgiou, and M. Pavon, “Optimal steering of a linear stochastic system to a final probability distribution, part I,” Transactions on Automatic Control, vol. 61, no. 5, pp. 1158–1169, 2015.
  16. Y. Chen, T. T. Georgiou, and M. Pavon, “Optimal steering of a linear stochastic system to a final probability distribution, part II,” Transactions on Automatic Control, vol. 61, no. 5, pp. 1170–1180, 2015.
  17. G. Rapakoulias and P. Tsiotras, “Discrete-time optimal covariance steering via semidefinite programming,” in 62nd Conference on Decision and Control, (Singapore), pp. 1802–1807, 2023.
  18. J. Ridderhof, J. Pilipovsky, and P. Tsiotras, “Chance-constrained covariance control for low-thrust minimum-fuel trajectory optimization,” in 2020 AAS/AIAA Astrodynamics Specialist Conference, pp. 9–13, 2020.
  19. V. Sivaramakrishnan, J. Pilipovsky, M. Oishi, and P. Tsiotras, “Distribution steering for discrete-time linear systems with general disturbances using characteristic functions,” in American Control Conference, (Atlanta, GA), pp. 4183–4190, 2022.
  20. I. Balci and E. Bakolas, “Density steering of gaussian mixture models for discrete-time linear systems,” arXiv preprint arXiv:2311.08500, 2023.
  21. R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” in Advances in Neural Information Processing Systems, vol. 31, (Montreal, Canada), pp. 6571–6584, Curran Associates, Inc., 2018.
  22. D. Onken, S. W. Fung, X. Li, and L. Ruthotto, “OT-flow: Fast and accurate continuous normalizing flows via optimal transport,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, (held virtually), pp. 9223–9232, 2021.
  23. Y. Chen, T. T. Georgiou, and M. Pavon, “On the relation between optimal transport and Schrödinger bridges: A stochastic control viewpoint,” Journal of Optimization Theory and Applications, vol. 169, pp. 671–691, 2016.
  24. T. Chen, G.-H. Liu, and E. A. Theodorou, “Likelihood training of Schrödinger bridge using forward-backward SDEs theory,” in International Conference on Learning Representations, (held virtually), April 2022.
  25. K. F. Caluya and A. Halder, “Finite horizon density steering for multi-input state feedback linearizable systems,” in American Control Conference, pp. 3577–3582, IEEE, 2020.
  26. J. Behrmann, W. Grathwohl, R. T. Chen, D. Duvenaud, and J.-H. Jacobsen, “Invertible residual networks,” in International Conference on Machine Learning, pp. 573–582, PMLR, 2019.
  27. C.-W. Huang, R. T. Chen, C. Tsirigotis, and A. Courville, “Convex potential flows: Universal probability distributions with optimal transport and convex optimization,” in International Conference on Learning Representations, (held virtually), April 2020.
  28. R. Van Den Berg, L. Hasenclever, J. M. Tomczak, and M. Welling, “Sylvester normalizing flows for variational inference,” in 34th Conference on Uncertainty in Artificial Intelligence, (Monterey, CA), pp. 393–402, Association For Uncertainty in Artificial Intelligence (AUAI), 2018.
  29. E. Haber and L. Ruthotto, “Stable architectures for deep neural networks,” Inverse Problems, vol. 34, no. 1, p. 014004, 2017.
  30. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016.
  31. J. Pilipovsky, V. Sivaramakrishnan, M. Oishi, and P. Tsiotras, “Probabilistic verification of relu neural networks via characteristic functions,” in Learning for Dynamics and Control Conference, (Philadelphia, PA), pp. 966–979, PMLR, 2023.
  32. G. Peyré and M. Cuturi, Computational Optimal Transport: With Applications to Data Science. Foundations and Trends in Machine Learning, 2019.
  33. G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, “Normalizing flows for probabilistic modeling and inference,” The Journal of Machine Learning Research, vol. 22, no. 1, pp. 2617–2680, 2021.
  34. J. Duchi, “Derivations for linear algebra and optimization,” Berkeley, California, vol. 3, no. 1, pp. 2325–5870, 2007.
  35. M. ApS, “Mosek modeling cookbook,” 2020.
  36. Princeton University Press, 2009.
  37. T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normalization for generative adversarial networks,” in International Conference on Learning Representations, (Vancouver, Canada), April 2018.
  38. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations, (Vancouver, Canada), April 2018.
  39. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., “Pytorch: An imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems, vol. 32, (Vancouver, Canada), 2019.
  40. D. Onken, L. Nurbekyan, X. Li, S. W. Fung, S. Osher, and L. Ruthotto, “A neural network approach for high-dimensional optimal control applied to multiagent path finding,” Transactions on Control Systems Technology, vol. 31, no. 1, pp. 235–251, 2022.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up