A PAC-Bayesian Framework for Optimal Control with Stability Guarantees (2403.17790v3)
Abstract: Stochastic Nonlinear Optimal Control (SNOC) involves minimizing a cost function that averages out the random uncertainties affecting the dynamics of nonlinear systems. For tractability reasons, this problem is typically addressed by minimizing an empirical cost, which represents the average cost across a finite dataset of sampled disturbances. However, this approach raises the challenge of quantifying the control performance against out-of-sample uncertainties. Particularly, in scenarios where the training dataset is small, SNOC policies are prone to overfitting, resulting in significant discrepancies between the empirical cost and the true cost, i.e., the average SNOC cost incurred during control deployment. Therefore, establishing generalization bounds on the true cost is crucial for ensuring reliability in real-world applications. In this paper, we introduce a novel approach that leverages PAC-Bayes theory to provide rigorous generalization bounds for SNOC. Based on these bounds, we propose a new method for designing optimal controllers, offering a principled way to incorporate prior knowledge into the synthesis process, which aids in improving the control policy and mitigating overfitting. Furthermore, by leveraging recent parametrizations of stabilizing controllers for nonlinear systems, our framework inherently ensures closed-loop stability. The effectiveness of our proposed method in incorporating prior knowledge and combating overfitting is shown by designing neural network controllers for tasks in cooperative robotics.
- P. Alquier, “User-friendly introduction to PAC-Bayes bounds,” Foundations and Trends® in Machine Learning, vol. 17, no. 2, pp. 174–303, 2024.
- A. Majumdar, A. Farid, and A. Sonar, “PAC-Bayes control: learning policies that provably generalize to novel environments,” The International Journal of Robotics Research, vol. 40, no. 2-3, pp. 574–593, 2021.
- M. Fard and J. Pineau, “PAC-Bayesian model selection for reinforcement learning,” in Advances in Neural Information Processing Systems, vol. 23, 2010.
- M. Fard, J. Pineau, and C. Szepesvári, “PAC-Bayesian policy evaluation for reinforcement learning,” Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, p. 195 – 202, 2011.
- C. Dawson, S. Gao, and C. Fan, “Safe control with learned certificates: A survey of neural Lyapunov, barrier, and contraction methods for robotics and control,” IEEE Transactions on Robotics, vol. 39, no. 3, pp. 1749–1767, 2023.
- R. Schwan, C. N. Jones, and D. Kuhn, “Stability verification of neural network controllers using mixed-integer programming,” IEEE Transactions on Automatic Control, 2023.
- L. Furieri, C. L. Galimberti, and G. Ferrari-Trecate, “Neural system level synthesis: Learning over all stabilizing policies for nonlinear systems,” in 61st Conference on Decision and Control (CDC), 2022, pp. 2765–2770.
- R. Wang, N. H. Barbara, M. Revay, and I. Manchester, “Learning over all stabilizing nonlinear controllers for a partially-observed linear system,” IEEE Control Systems Letters, vol. 7, pp. 91–96, 2022.
- L. Furieri, C. L. Galimberti, M. Zakwan, and G. Ferrari-Trecate, “Distributed neural network control with dependability guarantees: a compositional port-Hamiltonian approach,” in Learning for Dynamics and Control Conference. PMLR, 2022, pp. 571–583.
- F. Gu, H. Yin, L. El Ghaoui, M. Arcak, P. Seiler, and M. Jin, “Recurrent neural network controllers synthesis with stability guarantees for partially observed systems,” in AAAI, 2022, pp. 5385–5394.
- P. Pauli, J. Köhler, J. Berberich, A. Koch, and F. Allgöwer, “Offset-free setpoint tracking using neural network controllers,” in Learning for Dynamics and Control. PMLR, 2021, pp. 992–1003.
- F. Bonassi, M. Farina, J. Xie, and R. Scattolini, “On recurrent neural networks for learning-based control: recent results and ideas for future developments,” Journal of Process Control, vol. 114, pp. 92–104, 2022.
- A. Abate, D. Ahmed, M. Giacobbe, and A. Peruffo, “Formal synthesis of Lyapunov neural networks,” IEEE Control Systems Letters, vol. 5, no. 3, pp. 773–778, 2020.
- F. Berkenkamp, M. Turchetta, A. P. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” Advances in Neural Information Processing Systems, pp. 909–919, 2018.
- M. Revay, R. Wang, and I. R. Manchester, “Recurrent equilibrium networks: Flexible dynamic models with guaranteed stability and robustness,” IEEE Transactions on Automatic Control, 2023.
- Q. Liu and D. Wang, “Stein variational gradient descent: A general purpose Bayesian inference algorithm,” in Advances in Neural Information Processing Systems, vol. 29, 2016.
- O. Catoni, “PAC-Bayesian supervised classification: The thermodynamics of statistical learning,” IMS Lecture Notes Monograph Series, vol. 56, pp. 1–163, 2007.
- C. G. Economou, M. Morari, and B. O. Palsson, “Internal model control: extension to nonlinear system,” Industrial & Engineering Chemistry Process Design and Development, vol. 25, no. 2, pp. 403–411, 1986.
- P. J. Green, K. Łatuszyński, M. Pereyra, and C. P. Robert, “Bayesian computation: a perspective on the current state, and sampling backward and forwards,” Stat. Comput, vol. 25, no. 4, pp. 835–862, 2015.
- P. Alquier, J. Ridgway, and N. Chopin, “On the properties of variational approximations of Gibbs posteriors,” Journal of Machine Learning Research, vol. 17, no. 236, pp. 1–41, 2016.
- J. Rothfuss, V. Fortuin, M. Josifoski, and A. Krause, “PACOH: Bayes-optimal meta-learning with PAC-guarantees,” in International Conference on Machine Learning. PMLR, 2021, pp. 9116–9126.
- M. G. Boroujeni, A. Krause, and G. Ferrari-Trecate, “Personalized federated learning of probabilistic models: A PAC-Bayesian approach,” arXiv preprint arXiv:2401.08351, 2024.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
