2000 character limit reached
Learning to Satisfy Unknown Constraints in Iterative MPC
Published 9 Jun 2020 in eess.SY, cs.SY, and stat.ML | (2006.05054v3)
Abstract: We propose a control design method for linear time-invariant systems that iteratively learns to satisfy unknown polyhedral state constraints. At each iteration of a repetitive task, the method constructs an estimate of the unknown environment constraints using collected closed-loop trajectory data. This estimated constraint set is improved iteratively upon collection of additional data. An MPC controller is then designed to robustly satisfy the estimated constraint set. This paper presents the details of the proposed approach, and provides robust and probabilistic guarantees of constraint satisfaction as a function of the number of executed task iterations. We demonstrate the safety of the proposed framework and explore the safety vs. performance trade-off in a detailed numerical example.
- M. Tanaskovic, L. Fagiano, C. Novara, and M. Morari, “Data-driven control of nonlinear systems: An on-line direct approach,” Automatica, vol. 75, pp. 1–10, 2017.
- B. Recht, “A tour of reinforcement learning: The view from continuous control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 2, pp. 253–279, 2019.
- U. Rosolia, X. Zhang, and F. Borrelli, “Data-driven predictive control for autonomous systems,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, pp. 259–286, 2018.
- L. Hewing, K. P. Wabersich, M. Menner, and M. N. Zeilinger, “Learning-based model predictive control: Toward safe learning in control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, 2019.
- F. Pourbabaee, “Robust experimentation in the continuous time bandit problem,” Economic Theory, pp. 1–31, 2020.
- A. Liniger, A. Domahidi, and M. Morari, “Optimization-based autonomous racing of 1: 43 scale RC cars,” Optimal Control Applications and Methods, vol. 36, no. 5, pp. 628–647, 2015.
- D. Jain, A. Li, S. Singhal, A. Rajeswaran, V. Kumar, and E. Todorov, “Learning deep visuomotor policies for dexterous hand manipulation,” in International Conference on Robotics and Automation (ICRA), May 2019, pp. 3636–3643.
- F. Berkenkamp and A. P. Schoellig, “Safe and robust learning control with gaussian processes,” in 2015 European Control Conference (ECC), July 2015, pp. 2496–2501.
- D. P. Losey, M. Li, J. Bohg, and D. Sadigh, “Learning from my partner’s actions: Roles in decentralized robot teams,” in Conference on Robot Learning. PMLR, 2020, pp. 752–765.
- H. Yin, A. Packard, M. Arcak, and P. Seiler, “Finite horizon backward reachability analysis and control synthesis for uncertain nonlinear systems,” in American Control Conference (ACC), July 2019, pp. 5020–5026.
- A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier function based quadratic programs for safety critical systems,” IEEE Transactions on Automatic Control, vol. 62, no. 8, pp. 3861–3876, 2016.
- S. L. Herbert, M. Chen, S. Han, S. Bansal, J. F. Fisac, and C. J. Tomlin, “FaSTrack: A modular framework for fast and guaranteed safe motion planning,” in Conference on Decision and Control (CDC), Dec 2017, pp. 1517–1522.
- M. Bujarbaruah, X. Zhang, M. Tanaskovic, and F. Borrelli, “Adaptive stochastic MPC under time varying uncertainty,” IEEE Transactions on Automatic Control, pp. 1–1, 2020.
- J. Köhler, M. A. Müller, and F. Allgöwer, “Nonlinear reference tracking: An economic model predictive control perspective,” IEEE Transactions on Automatic Control, vol. 64, no. 1, pp. 254–269, Jan 2019.
- S. Singh, A. Majumdar, J. Slotine, and M. Pavone, “Robust online motion planning via contraction theory and convex optimization,” in International Conference on Robotics and Automation (ICRA), 2017, pp. 5883–5890.
- F. Berkenkamp, M. Turchetta, A. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” in Advances in neural information processing systems, 2017, pp. 908–918.
- L. Hewing, J. Kabzan, and M. N. Zeilinger, “Cautious model predictive control using gaussian process regression,” IEEE Transactions on Control Systems Technology, vol. 28, no. 6, pp. 2736–2743, 2019.
- R. Soloperto, M. A. Müller, S. Trimpe, and F. Allgöwer, “Learning-based robust model predictive control with state-dependent uncertainty,” in IFAC Conference on Nonlinear Model Predictive Control, Madison, Wisconsin, USA, Aug. 2018.
- T. Koller, F. Berkenkamp, M. Turchetta, and A. Krause, “Learning-based model predictive control for safe exploration,” in Conference on Decision and Control (CDC), Dec 2018, pp. 6059–6066.
- L. Armesto, J. Bosga, V. Ivan, and S. Vijayakumar, “Efficient learning of constraints and generic null space policies,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 1520–1526.
- C. Pérez-D’Arpino and J. A. Shah, “C-LEARN: Learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy,” in International Conference on Robotics and Automation (ICRA), May 2017, pp. 4058–4065.
- G. Chou, D. Berenson, and N. Ozay, “Learning constraints from demonstrations,” in International Workshop on the Algorithmic Foundations of Robotics. Springer, 2018, pp. 228–245.
- P. J. Goulart, E. C. Kerrigan, and J. M. Maciejowski, “Optimization over state feedback policies for robust control with constraints,” Automatica, vol. 42, no. 4, pp. 523–533, 2006.
- S. Gupta, G. Joshi, and O. Yağan, “Active distribution learning from indirect samples,” in 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2018, pp. 1012–1019.
- S. Gupta, S. Chaudhari, S. Mukherjee, G. Joshi, and O. Yağan, “A unified approach to translate classical bandit algorithms to the structured bandit setting,” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 3, pp. 840–853, 2020.
- S. Gupta, G. Joshi, and O. Yağan, “Correlated multi-armed bandits with a latent random source,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 3572–3576.
- ——, “Best-arm identification in correlated multi-armed bandits,” IEEE Journal on Selected Areas in Information Theory, vol. 2, no. 2, pp. 549–563, 2021.
- S. Gupta, S. Chaudhari, G. Joshi, and O. Yağan, “Multi-armed bandits with correlated arms,” IEEE Transactions on Information Theory, vol. 67, no. 10, pp. 6711–6732, 2021.
- S. Gupta, “Structured and correlated multi-armed bandits: Algorithms, theory and applications,” Ph.D. dissertation, Carnegie Mellon University, 2022.
- Y. J. Cho, S. Gupta, G. Joshi, and O. Yağan, “Bandit-based communication-efficient client selection strategies for federated learning,” in 2020 54th Asilomar Conference on Signals, Systems, and Computers. IEEE, 2020, pp. 1066–1069.
- U. Rosolia and F. Borrelli, “Learning model predictive control for iterative tasks: A computationally efficient approach for linear system,” IFAC-PapersOnLine, vol. 50, no. 1, pp. 3142–3147, 2017.
- K. P. Wabersich and M. N. Zeilinger, “Linear model predictive safety certification for learning-based control,” in Conference on Decision and Control (CDC), Dec 2018, pp. 7130–7135.
- A. Girard and G. J. Pappas, “Approximation metrics for discrete and continuous systems,” IEEE Transactions on Automatic Control, vol. 52, no. 5, pp. 782–798, 2007.
- A. B. Kurzhanski and P. Varaiya, “Ellipsoidal techniques for reachability analysis,” in International Workshop on Hybrid Systems: Computation and Control. Springer, 2000, pp. 202–214.
- G. C. Calafiore and M. C. Campi, “The scenario approach to robust control design,” IEEE Transactions on automatic control, vol. 51, no. 5, pp. 742–753, 2006.
- X. Zhang, K. Margellos, P. Goulart, and J. Lygeros, “Stochastic model predictive control using a combination of randomized and robust optimization,” in Conference on Decision and Control (CDC), Florence, Italy, 2013.
- M. Bujarbaruah, A. Shetty, K. Poolla, and F. Borrelli, “Learning robustness with bounded failure: An iterative MPC approach,” IFAC-PapersOnLine, vol. 53, pp. 7085–7090, 2020.
- J. Löfberg, “YALMIP: A toolbox for modeling and optimization in MATLAB,” in IEEE International Symposium on Computer Aided Control Systems Design, 2004, pp. 284–289.
- R. Tempo, E. W. Bai, and F. Dabbene, “Probabilistic robustness analysis: explicit bounds for the minimum number of samples,” in Conference on Decision and Control (CDC), vol. 3, Dec 1996, pp. 3424–3428.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.