On Building Myopic MPC Policies using Supervised Learning (2401.12546v2)
Abstract: The application of supervised learning techniques in combination with model predictive control (MPC) has recently generated significant interest, particularly in the area of approximate explicit MPC, where function approximators like deep neural networks are used to learn the MPC policy via optimal state-action pairs generated offline. While the aim of approximate explicit MPC is to closely replicate the MPC policy, substituting online optimization with a trained neural network, the performance guarantees that come with solving the online optimization problem are typically lost. This paper considers an alternative strategy, where supervised learning is used to learn the optimal value function offline instead of learning the optimal policy. This can then be used as the cost-to-go function in a myopic MPC with a very short prediction horizon, such that the online computation burden reduces significantly without affecting the controller performance. This approach differs from existing work on value function approximations in the sense that it learns the cost-to-go function by using offline-collected state-value pairs, rather than closed-loop performance data. The cost of generating the state-value pairs used for training is addressed using a sensitivity-based data augmentation scheme.
- A neural network model predictive controller. Journal of Process Control, 16(9), 937–946.
- The explicit linear quadratic regulator for constrained systems. Automatica, 38(1), 3–20.
- Bertsekas, D.P. (2019). Reinforcement learning and optimal control. Athena Scientific Belmont, MA.
- Toward safe dose delivery in plasma medicine using projected neural network-based fast approximate NMPC. IFAC-PapersOnLine, 53(2), 5279–5285.
- Deep neural network approximation of nonlinear model predictive control. IFAC-PapersOnLine, 53(2), 11319–11324.
- Approximating explicit model predictive control using constrained neural networks. In 2018 Annual American control conference (ACC), 1520–1527. IEEE.
- Large scale model predictive control with neural networks and primal active sets. Automatica, 135, 109947.
- Approximate model predictive building control via machine learning. Applied Energy, 218, 199–216.
- Behavioural cloning in control of a dynamic system. In 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century, volume 3, 2904–2909. IEEE.
- Fiacco, A.V. (1976). Sensitivity analysis for nonlinear programming using penalty methods. Mathematical programming, 10(1), 287–311.
- Learning an approximate model predictive controller with guarantees. IEEE Control Systems Letters, 2(3), 543–548.
- Efficient representation and approximation of model predictive control laws via deep learning. IEEE Transactions on Cybernetics.
- Imputing a convex objective function. In 2011 IEEE international symposium on intelligent control, 613–619. IEEE.
- Krishnamoorthy, D. (2022). A sensitivity-based data augmentation framework for model predictive control policy approximation. IEEE Transactions on Automatic Control, 67, 6090 – 6097.
- Krishnamoorthy, D. (2023). An improved data augmentation scheme for model predictive control policy approximation. IEEE Control Systems Letters, 7, 1867 – 1872.
- An adaptive correction scheme for offset-free asymptotic performance in deep learning-based economic MPC. IFAC-PapersOnLine, 54(3), 584–589.
- Industrial, large-scale model predictive control with structured neural networks. Computers & Chemical Engineering, 150, 107291.
- Maciejowski, J.M. (2002). Predictive control: with constraints. Pearson education.
- Fusion of machine learning and mpc under uncertainty: What advances are on the horizon? In 2022 American Control Conference (ACC), 342–357. IEEE.
- Safe and fast tracking on a robot manipulator: Robust mpc and neural network control. IEEE Robotics and Automation Letters, 5(2), 3050–3057.
- A receding-horizon regulator for nonlinear systems and a neural approximation. Automatica, 31(10), 1443–1451.
- Approximate closed-loop robust model predictive control with guaranteed stability and constraint satisfaction. IEEE Control Systems Letters.
- Approximate model predictive control with recurrent neural network for autonomous driving vehicles. In 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), 1076–1081. IEEE.
- Learning the optimal state-feedback using deep networks. In 2016 IEEE Symposium Series on Computational Intelligence (SSCI), 1–8. IEEE.
- A predictive safety filter for learning-based control of constrained nonlinear dynamical systems. Automatica, 129, 109597.
- Safe and near-optimal policy learning for model predictive control using primal-dual neural networks. In 2019 American Control Conference (ACC), 354–359. IEEE.
- Near-optimal rapid mpc using neural networks: A primal-dual policy learning framework. IEEE Transactions on Control Systems Technology, 29(5), 2102–2114.