Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Innovation-triggered Learning with Application to Data-driven Predictive Control (2401.15824v2)

Published 29 Jan 2024 in eess.SY and cs.SY

Abstract: Data-driven control has attracted lots of attention in recent years, especially for plants that are difficult to model based on first principles. In particular, a key issue in data-driven approaches is how to make efficient use of data as the abundance of data becomes overwhelming. To address this issue, this work proposes an innovation-triggered learning framework and a corresponding data-driven controller design approach with guaranteed stability. Specifically, we consider a linear time-invariant system with unknown dynamics. A set-membership approach is introduced to learn a parametric uncertainty set for the unknown dynamics. Then, a data selection mechanism is proposed by online evaluating the innovation contained in the sampled data, wherein the innovation is quantified by its effect of shrinking the parametric uncertainty set. Next, after introducing a stability criterion using the set-membership estimate of the system dynamics, a robust data-driven predictive controller is designed by minimizing a worst-case cost function. The closed-loop stability of the data-driven predictive controller equipped with the innovation-triggered learning protocol is discussed within a high probability framework. Finally, comparative numerical experiments are performed to verify the validity of the proposed approach, and the characteristics and the design principle of the learning hyper-parameter are also discussed.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. A. Grancharova, J. Kocijan, and T. A. Johansen, “Explicit stochastic predictive control of combustion plants based on Gaussian process models,” Automatica, vol. 44, no. 6, pp. 1621–1631, 2008.
  2. X. Zhang, M. Bujarbaruah, and F. Borrelli, “Safe and near-optimal policy learning for model predictive control using primal-dual neural networks,” in 2019 American Control Conference (ACC), (Philadelphia, PA, USA), pp. 354–359, IEEE, 2019.
  3. P. Scharnhorst, E. T. Maddalena, Y. Jiang, and C. N. Jones, “Robust uncertainty bounds in reproducing kernel hilbert spaces: A convex optimization approach,” IEEE Transactions on Automatic Control, vol. 68, no. 5, pp. 2848–2861, 2023.
  4. I. Markovsky and F. Dörfler, “Behavioral systems theory in data-driven analysis, signal processing, and control,” Annual Reviews in Control, vol. 52, pp. 42–64, 2021.
  5. H. J. van Waarde, M. K. Camlibel, and M. Mesbahi, “From noisy data to feedback controllers: Nonconservative design via a matrix S-lemma,” IEEE Transactions on Automatic Control, vol. 67, no. 1, pp. 162–175, 2020.
  6. Z. Hou and S. Jin, “Data-driven model-free adaptive control for a class of mimo nonlinear discrete-time systems,” IEEE transactions on neural networks, vol. 22, no. 12, pp. 2173–2188, 2011.
  7. A. L. Strehl, L. Li, E. Wiewiora, J. Langford, and M. L. Littman, “Pac model-free reinforcement learning,” in Proceedings of the 23rd international conference on Machine learning, pp. 881–888, 2006.
  8. D. Baumann, F. Solowjow, K. H. Johansson, and S. Trimpe, “Event-triggered pulse control with model learning (if necessary),” in 2019 American Control Conference (ACC), pp. 792–797, IEEE, 2019.
  9. I. Abraham and T. D. Murphey, “Active learning of dynamics for data-driven control using Koopman operators,” IEEE Transactions on Robotics, vol. 35, no. 5, pp. 1071–1083, 2019.
  10. S. Wang, X. Jin, S. Mao, A. V. Vasilakos, and Y. Tang, “Model-free event-triggered optimal consensus control of multiple Euler-Lagrange systems via reinforcement learning,” IEEE Transactions on Network Science and Engineering, vol. 8, no. 1, pp. 246–258, 2020.
  11. H. Schluter, F. Solowjow, and S. Trimpe, “Event-triggered learning for linear quadratic control,” IEEE Transactions on Automatic Control, pp. 4485–4498, 2020.
  12. T. Lew, A. Sharma, J. Harrison, A. Bylard, and M. Pavone, “Safe active dynamics learning and control: A sequential exploration–exploitation framework,” IEEE Transactions on Robotics, 2022.
  13. J. Jiao, A. Capone, and S. Hirche, “Backstepping tracking control using gaussian processes with event-triggered online learning,” IEEE Control Systems Letters, vol. 6, pp. 3176–3181, 2022.
  14. J. Umlauft and S. Hirche, “Feedback linearization based on Gaussian processes with event-triggered online learning,” IEEE Transactions on Automatic Control, vol. 65, no. 10, pp. 4154–4169, 2019.
  15. F. Solowjow, D. Baumann, J. Garcke, and S. Trimpe, “Event-triggered learning for resource-efficient networked control,” in 2018 Annual American Control Conference (ACC), (Milwaukee, WI, USA), pp. 6506–6512, IEEE, 2018.
  16. J. Umlauft, T. Beckers, A. Capone, A. Lederer, and S. Hirche, “Smart forgetting for safe online learning with Gaussian processes,” in Proceedings of the 2nd Conference on Learning for Dynamics and Control, vol. 120, (Zürich, Switzerland), 2020.
  17. A. Lederer, A. J. O. Conejo, K. A. Maier, W. Xiao, J. Umlauft, and S. Hirche, “Gaussian process-based real-time learning for safety critical applications,” in International Conference on Machine Learning, pp. 6055–6064, PMLR, 2021.
  18. K. He, Y. Deng, G. Wang, X. Sun, Y. Sun, and Z. Chen, “Learning-based trajectory tracking and balance control for bicycle robots with a pendulum: A Gaussian process approach,” IEEE/ASME Transactions on Mechatronics, vol. 27, no. 2, pp. 634–644, 2022.
  19. K. Zheng, D. Shi, Y. Shi, and J. Wang, “Non-parametric event-triggered learning with applications to adaptive model predictive control,” IEEE Transactions on Automatic Control, 2022.
  20. J.-P. Calliess, S. J. Roberts, C. E. Rasmussen, and J. Maciejowski, “Lazily adapted constant kinky inference for nonparametric regression and model-reference adaptive control,” Automatica, vol. 122, p. 109216, 2020.
  21. J.-D. Diao, J. Guo, and C.-Y. Sun, “Event-triggered identification of FIR systems with binary-valued output observations,” Automatica, vol. 98, pp. 95–102, 2018.
  22. J. Guo and J.-D. Diao, “Prediction-based event-triggered identification of quantized input FIR systems with quantized output observations,” Science China Information Sciences, vol. 63, no. 1, pp. 1–12, 2020.
  23. F. Solowjow and S. Trimpe, “Event-triggered learning,” Automatica, vol. 117, p. 109009, 2020.
  24. Y. Zheng, S. Li, R. Wan, and Y. Wang, “Economic Lyapunov-based model predictive control with event-triggered parametric identification,” International Journal of Robust and Nonlinear Control, vol. 32, no. 1, pp. 205–226, 2022.
  25. J. Beuchert, F. Solowjow, J. Raisch, S. Trimpe, and T. Seel, “Hierarchical event-triggered learning for cyclically excited systems with application to wireless sensor networks,” IEEE Control Systems Letters, vol. 4, no. 1, pp. 103–108, 2020.
  26. B. Recht, “A tour of reinforcement learning: The view from continuous control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 2, pp. 253–279, 2019.
  27. N. Matni, A. Proutiere, A. Rantzer, and S. Tu, “From self-tuning regulators to reinforcement learning and back again,” in 2019 IEEE 58th Conference on Decision and Control (CDC), pp. 3724–3740, IEEE, 2019.
  28. S. Dean, H. Mania, N. Matni, B. Recht, and S. Tu, “On the sample complexity of the linear quadratic regulator,” Foundations of Computational Mathematics, vol. 20, no. 4, pp. 633–679, 2020.
  29. J. C. Willems, P. Rapisarda, I. Markovsky, and B. L. De Moor, “A note on persistency of excitation,” Systems & Control Letters, vol. 54, no. 4, pp. 325–329, 2005.
  30. H. J. van Waarde, M. K. Camlibel, J. Eising, and H. L. Trentelman, “Quadratic matrix inequalities with applications to data-based control,” SIAM Journal on Control and Optimization, vol. 61, no. 4, pp. 2251–2281, 2023.
  31. H. J. van Waarde, C. De Persis, M. K. Camlibel, and P. Tesi, “Willems’ fundamental lemma for state-space systems and its extension to multiple datasets,” IEEE Control Systems Letters, vol. 4, no. 3, pp. 602–607, 2020.
  32. P. Schmitz, T. Faulwasser, and K. Worthmann, “Willems’ fundamental lemma for linear descriptor systems and its use for data-driven output-feedback MPC,” IEEE Control Systems Letters, vol. 6, pp. 2443–2448, 2022.
  33. Y. Lian, J. Shi, M. Koch, and C. N. Jones, “Adaptive robust data-driven building control via bilevel reformulation: An experimental result,” IEEE Transactions on Control Systems Technology, pp. 1–17, 2023.
  34. C. De Persis and P. Tesi, “Formulas for data-driven control: stabilization, optimality, and robustness,” IEEE Transactions on Automatic Control, vol. 65, no. 3, pp. 909–924, 2019.
  35. T. R. Steentjes, M. Lazar, and P. M. Van den Hof, “On data-driven control: informativity of noisy input-output data with cross-covariance bounds,” IEEE Control Systems Letters, vol. 6, pp. 2192–2197, 2021.
  36. J. Berberich, C. W. Scherer, and F. Allgöwer, “Combining prior knowledge and data for robust controller design,” IEEE Transactions on Automatic Control, pp. 1–16, 2022.
  37. R. Ahlswede and A. Winter, “Strong converse for identification via quantum channels,” IEEE Transactions on Information Theory, vol. 48, no. 3, pp. 569–579, 2002.
  38. J. Navarro, “A very simple proof of the multivariate chebyshev’s inequality,” Communications in Statistics - Theory and Methods, vol. 45, no. 12, pp. 3458–3463, 2016.
  39. Birkhäuser Boston, MA, 1997.
  40. H. J. Van Waarde, J. Eising, H. L. Trentelman, and M. K. Camlibel, “Data informativity: a new perspective on data-driven analysis and control,” IEEE Transactions on Automatic Control, vol. 65, no. 11, pp. 4753–4768, 2020.
  41. J. A. Rossiter, B. Kouvaritakis, and M. Rice, “A numerically robust state-space approach to stable-predictive control strategies,” Automatica, vol. 34, no. 1, pp. 65–73, 1998.
  42. D. Mayne, “Robust and stochastic MPC: are we going in the right direction?,” IFAC-PapersOnLine, vol. 48, no. 23, pp. 1–8, 2015.
  43. F. A. Bayer, F. D. Brunner, M. Lazar, M. Wijnand, and F. Allgöwer, “A tube-based approach to nonlinear explicit MPC,” in 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 4059–4064, IEEE, 2016.
  44. R. D. McAllister and J. B. Rawlings, “The stochastic robustness of nominal and stochastic model predictive control,” IEEE Transactions on Automatic Control, pp. 1–13, 2022.
  45. K. Zhang, C. Liu, and Y. Shi, “Self-triggered adaptive model predictive control of constrained nonlinear systems: a min–max approach,” Automatica, vol. 142, p. 110424, 2022.
  46. Nob Hill Publishing, LLC, 2019.
  47. D. O. A. Brian and B. M. John, Optimal filtering. Prentice-Hall, Inc, 1979.
  48. L. Ljung, System Identification: Theory for the User. Pearson, 1998.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now