Stability-informed Bayesian Optimization for MPC Cost Function Learning (2404.12187v1)
Abstract: Designing predictive controllers towards optimal closed-loop performance while maintaining safety and stability is challenging. This work explores closed-loop learning for predictive control parameters under imperfect information while considering closed-loop stability. We employ constrained Bayesian optimization to learn a model predictive controller's (MPC) cost function parametrized as a feedforward neural network, optimizing closed-loop behavior as well as minimizing model-plant mismatch. Doing so offers a high degree of freedom and, thus, the opportunity for efficient and global optimization towards the desired and optimal closed-loop behavior. We extend this framework by stability constraints on the learned controller parameters, exploiting the optimal value function of the underlying MPC as a Lyapunov candidate. The effectiveness of the proposed approach is underlined in simulations, highlighting its performance and safety capabilities.
- A painless deterministic policy gradient method for learning-based MPC. In European Control Conference, 1–7.
- Safe learning of regions of attraction for uncertain, nonlinear systems with Gaussian processes. In Conference on Decision and Control, 4661–4666.
- Neural Lyapunov control. In Advances in Neural Information Processing Systems, volume 32.
- An introduction to nonlinear model predictive control. In 21st Benelux meeting on systems and control, volume 11, 119–141. Veldhoven.
- Garnett, R. (2023). Bayesian Optimization. Cambridge University Press.
- Gevers, M. (1993). Towards a joint design of identification and control? In Essays on Control: Perspectives in the Theory and its Applications, 111–151. Springer.
- Cautious model predictive control using Gaussian process regression. IEEE Transactions on Control Systems Technology, 28(6), 2736–2743.
- Reinforcement learning for MPC: Fundamentals and current challenges. In IFAC-PapersOnLine, volume 56, 5773–5780.
- Krishnamoorthy, D. (2023). On tuning parameterized control policies online for safety-critical systems – Applied to biomedical systems. In IFAC-PapersOnLine, volume 56, 5781–5786.
- Online learning-based model predictive control with Gaussian process models and stability guarantees. International Journal of Robust and Nonlinear Control, 31(18), 8785–8812.
- Performance-oriented model learning for control via multi-objective Bayesian optimization. Computers & Chemical Engineering, 162, 107770.
- A tutorial on derivative-free policy learning methods for interpretable controller representations. In American Control Conference, 1295–1306.
- Performance-oriented model learning for data-driven MPC design. IEEE Control Systems Letters, 3(3), 577–582.
- Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA.
- Model Predictive Control: Theory, Computation, and Design. Nob Hill Publishing, Madison, Wisconsin, 2nd edition.
- Convex neural network-based cost modifications for learning model predictive control. IEEE Open Journal of Control Systems, 1, 366–379.
- Safe reinforcement learning using robust MPC. IEEE Transactions on Automatic Control, 66(8), 3638–3652.