2000 character limit reached
Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees (2404.07373v1)
Published 10 Apr 2024 in eess.SY, cs.LG, and cs.SY
Abstract: In this paper, a method is presented to synthesize neural network controllers such that the feedback system of plant and controller is dissipative, certifying performance requirements such as L2 gain bounds. The class of plants considered is that of linear time-invariant (LTI) systems interconnected with an uncertainty, including nonlinearities treated as an uncertainty for convenience of analysis. The uncertainty of the plant and the nonlinearities of the neural network are both described using integral quadratic constraints (IQCs). First, a dissipativity condition is derived for uncertain LTI systems. Second, this condition is used to construct a linear matrix inequality (LMI) which can be used to synthesize neural network controllers. Finally, this convex condition is used in a projection-based training method to synthesize neural network controllers with dissipativity guarantees. Numerical examples on an inverted pendulum and a flexible rod on a cart are provided to demonstrate the effectiveness of this approach.
- Deep equilibrium models. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
- Stephen P. Boyd, editor. Linear Matrix Inequalities in System and Control Theory. Number vol. 15 in SIAM Studies in Applied Mathematics. Society for Industrial and Applied Mathematics, 1994.
- Linear System Theory. Springer Texts in Electrical Engineering. Springer, 1991.
- Implicit deep learning. SIAM Journal on Mathematics of Data Science, 3(3):930–958, 2021. Publisher: Society for Industrial and Applied Mathematics.
- Safety verification and robustness analysis of neural networks via quadratic constraints and semidefinite programming. IEEE Transactions on Automatic Control, 67(1):1–15, 2022.
- Efficient and accurate estimation of Lipschitz constants for deep neural networks. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
- Recurrent neural network controllers synthesis with stability guarantees for partially observed systems. Proceedings of the AAAI Conference on Artificial Intelligence, 36(5):5385–5394, 2022.
- Certifying incremental quadratic constraints for neural networks via convex optimization. In Proceedings of the 3rd Conference on Learning for Dynamics and Control, pages 842–853. PMLR, 2021.
- Synthesis of stabilizing recurrent equilibrium network controllers. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 7449–7454, 2022. ISSN: 2576-2370.
- Continuous control with deep reinforcement learning. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
- A. Megretski and A. Rantzer. System analysis via integral quadratic constraints. IEEE Transactions on Automatic Control, 42(6):819–830, 1997.
- Neural network training under semidefinite constraints. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 2731–2736, 2022. ISSN: 2576-2370.
- Linear systems with neural network nonlinearities: Improved stability analysis via acausal Zames-Falb multipliers. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 3611–3618, 2021.
- Training robust neural networks using Lipschitz bounds. IEEE Control Systems Letters, 6:121–126, 2022.
- Recurrent equilibrium networks: Flexible dynamic models with guaranteed stability and robustness. IEEE Transactions on Automatic Control, pages 1–16, 2023.
- Multiobjective output-feedback control via LMI optimization. IEEE Transactions on Automatic Control, 42(7):896–911, 1997.
- Carsten W. Scherer. Dissipativity and Integral Quadratic Constraints: Tailored Computational Robustness Tests for Complex Interconnections. 42(3):115–139.
- Peter Seiler. Stability analysis with dissipation inequalities and integral quadratic constraints. IEEE Transactions on Automatic Control, 60(6):1704–1709, 2015. Conference Name: IEEE Transactions on Automatic Control.
- Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning. The MIT Press, second edition, 2020.
- MDP homomorphic networks: Group symmetries in reinforcement learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 4199–4210. Curran Associates, Inc., 2020.
- IQC-Synthesis with general dynamic multipliers. International Journal of Robust and Nonlinear Control, 24(17):3027–3056, 2014.
- Robust stability and performance analysis based on integral quadratic constraints. European Journal of Control, 31:1–32, 2016.
- SO-2 Equivariant Reinforcement Learning. In International Conference on Learning Representations, 2022.
- Learning over all stabilizing nonlinear controllers for a partially-observed linear system. IEEE Control Systems Letters, 7:91–96, 2023.
- Robust synthesis for linear parameter varying systems using integral quadratic constraints. Automatica, 68:111–118, 2016.
- Jan C. Willems. Dissipative dynamical systems part I: General theory. Archive for Rational Mechanics and Analysis, 45(5):321–351, 1972.
- Jan C. Willems. Dissipative dynamical systems part II: Linear systems with quadratic supply rates. Archive for Rational Mechanics and Analysis, 45(5):352–393, 1972.
- Stability analysis using quadratic constraints for systems with neural network controllers. IEEE Transactions on Automatic Control, 67(4):1980–1987, 2022.