2000 character limit reached
Data-Driven Stable Neural Feedback Loop Design (2405.02100v2)
Published 3 May 2024 in math.OC
Abstract: This paper proposes a data-driven approach to design a feedforward Neural Network (NN) controller with a stability guarantee for plants with unknown dynamics. We first introduce data-driven representations of stability conditions for Neural Feedback Loops (NFLs) with linear plants, which can be formulated into a semidefinite program (SDP). Subsequently, this SDP constraint is integrated into the NN training process to ensure stability of the feedback loop. The whole NN controller design problem can be solved by an iterative algorithm. Finally, we illustrate the effectiveness of the proposed method compared to model-based methods via numerical examples.
- A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE transactions on systems, man, and cybernetics, no. 5, pp. 834–846, 1983.
- C. W. Anderson, “Learning to control an inverted pendulum with connectionist networks,” in 1988 American Control Conference, 1988, Conference Proceedings, pp. 2294–2298.
- W. Li and J. J. E. Slotine, “Neural network control of unknown nonlinear systems,” in 1989 American Control Conference, 1989, Conference Proceedings, pp. 1136–1141.
- Y. Chen, Y. Shi, and B. Zhang, “Optimal control via neural networks: A convex approach,” arXiv preprint arXiv:1805.11835, 2018.
- F. Fabiani and P. J. Goulart, “Reliably-stabilizing piecewise-affine neural network controllers,” IEEE Transactions on Automatic Control, vol. 68, no. 9, pp. 5201–5215, 2023.
- E. Kaufmann, L. Bauersfeld, A. Loquercio, M. Müller, V. Koltun, and D. Scaramuzza, “Champion-level drone racing using deep reinforcement learning,” Nature, vol. 620, no. 7976, pp. 982–987, 2023.
- S. Gowal, K. D. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, R. Arandjelović, T. Mann, and P. Kohli, “On the effectiveness of interval bound propagation for training verifiably robust models,” arXiv preprint, 2019.
- M. Fazlyab, M. Morari, and G. J. Pappas, “Safety verification and robustness analysis of neural networks via quadratic constraints and semidefinite programming,” IEEE Transactions on Automatic Control, vol. 67, no. 1, pp. 1–15, 2022.
- M. Newton and A. Papachristodoulou, “Neural network verification using polynomial optimisation,” in 2021 60th IEEE Conference on Decision and Control (CDC), 2021, Conference Proceedings, pp. 5092–5097.
- M. Fazlyab, A. Robey, H. Hassani, M. Morari, and G. Pappas, “Efficient and accurate estimation of lipschitz constants for deep neural networks,” Advances in Neural Information Processing Systems, vol. 32, 2019.
- P. Pauli, N. Funcke, D. Gramlich, M. A. Msalmi, and F. Allgöwer, “Neural network training under semidefinite constraints,” in 2022 IEEE 61st Conference on Decision and Control (CDC), 2022, Conference Proceedings, pp. 2731–2736.
- P. Pauli, A. Koch, J. Berberich, P. Kohler, and F. Allgöwer, “Training robust neural networks using lipschitz bounds,” IEEE Control Systems Letters, vol. 6, pp. 121–126, 2022.
- H. Yin, P. Seiler, M. Jin, and M. Arcak, “Imitation learning with stability and safety guarantees,” IEEE Control Systems Letters, vol. 6, pp. 409–414, 2022.
- F. Gu, H. Yin, L. E. Ghaoui, M. Arcak, P. Seiler, and M. Jin, “Recurrent neural network controllers synthesis with stability guarantees for partially observed systems,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 5, pp. 5385–5394, 2022. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/20476
- N. Junnarkar, H. Yin, F. Gu, M. Arcak, and P. Seiler, “Synthesis of stabilizing recurrent equilibrium network controllers,” in 2022 IEEE 61st Conference on Decision and Control (CDC), 2022, Conference Proceedings, pp. 7449–7454.
- R. Sepulchre, “Data-driven control: Part one of two [about this issue],” IEEE Control Systems Magazine, vol. 43, no. 5, pp. 4–7, 2023.
- C. D. Persis and P. Tesi, “Formulas for data-driven control: Stabilization, optimality, and robustness,” IEEE Transactions on Automatic Control, vol. 65, no. 3, pp. 909–924, 2020.
- M. Guo, C. D. Persis, and P. Tesi, “Data-driven stabilization of nonlinear polynomial systems with noisy data,” IEEE Transactions on Automatic Control, vol. 67, no. 8, pp. 4210–4217, 2022.
- S. Prajna, A. Papachristodoulou, and W. Fen, “Nonlinear control synthesis by sum of squares optimization: a lyapunov-based approach,” in 2004 5th Asian Control Conference (IEEE Cat. No.04EX904), vol. 1, 2004, Conference Proceedings, pp. 157–165 Vol.1.
- H. Wang, Z. Xiong, L. Zhao, and A. Papachristodoulou, “Model-free verification for neural network controlled systems,” arXiv preprint arXiv:2312.08293, 2023.
- H. Yin, P. Seiler, and M. Arcak, “Stability analysis using quadratic constraints for systems with neural network controllers,” IEEE Transactions on Automatic Control, vol. 67, no. 4, pp. 1980–1987, 2022.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.