Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Stable and Passive Neural Differential Equations (2404.12554v2)

Published 19 Apr 2024 in eess.SY, cs.LG, and cs.SY

Abstract: In this paper, we introduce a novel class of neural differential equation, which are intrinsically Lyapunov stable, exponentially stable or passive. We take a recently proposed Polyak Lojasiewicz network (PLNet) as an Lyapunov function and then parameterize the vector field as the descent directions of the Lyapunov function. The resulting models have a same structure as the general Hamiltonian dynamics, where the Hamiltonian is lower- and upper-bounded by quadratic functions. Moreover, it is also positive definite w.r.t. either a known or learnable equilibrium. We illustrate the effectiveness of the proposed model on a damped double pendulum system.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural networks, vol. 2, no. 5, pp. 359–366, 1989.
  2. R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” Advances in neural information processing systems, vol. 31, 2018.
  3. M. M. Tobenkin, I. R. Manchester, J. Wang, A. Megretski, and R. Tedrake, “Convex optimization in identification of stable non-linear state space models,” in 49th IEEE Conference on Decision and Control (CDC), pp. 7232–7237, IEEE, 2010.
  4. S. M. Khansari-Zadeh and A. Billard, “Learning stable nonlinear dynamical systems with gaussian mixture models,” IEEE Transactions on Robotics, vol. 27, no. 5, pp. 943–957, 2011.
  5. M. M. Tobenkin, I. R. Manchester, and A. Megretski, “Convex parameterizations and fidelity bounds for nonlinear identification and reduced-order modelling,” IEEE Transactions on Automatic Control, vol. 62, no. 7, pp. 3679–3686, 2017.
  6. J. Umenberger, J. Wågberg, I. R. Manchester, and T. B. Schön, “Maximum likelihood identification of stable linear dynamical systems,” Automatica, vol. 96, pp. 280–292, 2018.
  7. W. Lohmiller and J.-J. E. Slotine, “On contraction analysis for non-linear systems,” Automatica, vol. 34, no. 6, pp. 683–696, 1998.
  8. M. Revay and I. Manchester, “Contracting implicit recurrent neural networks: Stable models with improved trainability,” in Learning for Dynamics and Control, pp. 393–403, PMLR, 2020.
  9. M. Revay, R. Wang, and I. R. Manchester, “Recurrent equilibrium networks: Flexible dynamic models with guaranteed stability and robustness,” IEEE Transactions on Automatic Control, 2023.
  10. F. Fan, B. Yi, D. Rye, G. Shi, and I. R. Manchester, “Learning stable koopman embeddings for identification and control,” arXiv preprint arXiv:2401.08153, 2024.
  11. D. Martinelli, C. L. Galimberti, I. R. Manchester, L. Furieri, and G. Ferrari-Trecate, “Unconstrained parametrization of dissipative and contracting neural ordinary differential equations,” in 2023 62nd IEEE Conference on Decision and Control (CDC), pp. 3043–3048, IEEE, 2023.
  12. F. Wilson Jr, “The structure of the level surfaces of a lyapunov function,” 1967.
  13. M. T. Anderson, “Geometrization of 3-manifolds via the ricci flow,” Notices AMS, vol. 51, pp. 184–193, 2004.
  14. S. M. Richards, F. Berkenkamp, and A. Krause, “The lyapunov neural network: Adaptive stability certification for safe learning of dynamical systems,” in Conference on Robot Learning, pp. 466–476, PMLR, 2018.
  15. J. Z. Kolter and G. Manek, “Learning stable deep dynamics models,” Advances in neural information processing systems, vol. 32, 2019.
  16. B. Amos, L. Xu, and J. Z. Kolter, “Input convex neural networks,” in International Conference on Machine Learning (ICML), pp. 146–155, PMLR, 2017.
  17. C. Dawson, S. Gao, and C. Fan, “Safe control with learned certificates: A survey of neural lyapunov, barrier, and contraction methods for robotics and control,” IEEE Transactions on Robotics, 2023.
  18. R. Wang, K. Dvijotham, and I. R. Manchester, “Monotone, bi-Lipschitz, and Polyak-Lojasiewicz networks,” arXiv preprint arXiv:2402.01344, 2024.
  19. A. Van Der Schaft, D. Jeltsema, et al., “Port-hamiltonian systems theory: An introductory overview,” Foundations and Trends® in Systems and Control, vol. 1, no. 2-3, pp. 173–378, 2014.
  20. Springer Science & Business Media, 2009.
  21. H. K. Khalil, Nonlinear systems. Prentice Hall, New York, NY, 2002.
  22. R. T. Chen, J. Behrmann, D. K. Duvenaud, and J.-H. Jacobsen, “Residual flows for invertible generative modeling,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  23. J. Behrmann, W. Grathwohl, R. T. Chen, D. Duvenaud, and J.-H. Jacobsen, “Invertible residual networks,” in International conference on machine learning (ICML), pp. 573–582, PMLR, 2019.
  24. C. Lu, J. Chen, C. Li, Q. Wang, and J. Zhu, “Implicit normalizing flows,” in International Conference on Learning Representations (ICLR), 2021.
  25. B. Ahn, C. Kim, Y. Hong, and H. J. Kim, “Invertible monotone operators for normalizing flows,” Advances in Neural Information Processing Systems, vol. 35, pp. 16836–16848, 2022.
  26. B. Polyak, “Gradient methods for minimizing functionals (in russian),” USSR Computational Mathematics and Mathematical Physics, vol. 3, no. 4, pp. 643–653, 1963.
  27. S. Lojasiewicz, “A topological property of real analytic subsets,” Coll. du CNRS, Les équations aux dérivées partielles, vol. 117, no. 87-89, p. 2, 1963.
  28. Springer Science & Business Media, 2007.
  29. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR), 2015.
Citations (1)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets