Papers
Topics
Authors
Recent
Search
2000 character limit reached

SIMBa: System Identification Methods leveraging Backpropagation

Published 23 Nov 2023 in eess.SY and cs.SY | (2311.13889v3)

Abstract: This manuscript details and extends the SIMBa toolbox (System Identification Methods leveraging Backpropagation) presented in previous work, which uses well-established Machine Learning tools for discrete-time linear multi-step-ahead state-space System Identification (SI). SIMBa leverages linear-matrix-inequality-based free parametrizations of Schur matrices to guarantee the stability of the identified model by design. In this paper, backed up by novel free parametrizations of Schur matrices, we extend the toolbox to show how SIMBa can incorporate known sparsity patterns or true values of the state-space matrices to identify without jeopardizing stability. We extensively investigate SIMBa's behavior when identifying diverse systems with various properties from both simulated and real-world data. Overall, we find it consistently outperforms traditional stable subspace identification methods, and sometimes significantly, especially when enforcing desired model properties. These results hint at the potential of SIMBa to pave the way for generic structured nonlinear SI. The toolbox is open-sourced on https://github.com/Cemempamoi/simba.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. W. Rawat and Z. Wang, “Deep convolutional neural networks for image classification: A comprehensive review,” Neural computation, vol. 29, no. 9, pp. 2352–2449, 2017.
  2. L. Ljung, C. Andersson, K. Tiels, and T. B. Schön, “Deep learning and system identification,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 1175–1181, 2020.
  3. M. Forgione and D. Piga, “Continuous-time system identification with neural networks: Model structures and fitting criteria,” European Journal of Control, vol. 59, pp. 69–81, 2021.
  4. L. Ljung, “System identification,” in Signal analysis and prediction.   Springer, 1998, pp. 163–173.
  5. D. Gedon, N. Wahlström, T. B. Schön, and L. Ljung, “Deep state space models for nonlinear system identification,” IFAC-PapersOnLine, vol. 54, no. 7, pp. 481–486, 2021.
  6. L. Di Natale, M. Zakwan, B. Svetozarevic, P. Heer, G. F. Trecate, and C. N. Jones, “Stable Linear Subspace Identification: A Machine Learning Approach,” arXiv preprint arXiv:2311.03197, 2023.
  7. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems 32.   Curran Associates, Inc., 2019, pp. 8024–8035. [Online]. Available: http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
  8. G. Armenise, M. Vaccari, R. B. Di Capaci, and G. Pannocchia, “An open-source system identification package for multivariable processes,” in 2018 UKACC 12th International Conference on Control (CONTROL).   IEEE, 2018, pp. 152–157.
  9. S. J. Qin, “An overview of subspace identification,” Computers & chemical engineering, vol. 30, no. 10-12, pp. 1502–1513, 2006.
  10. P. Van Overschee and B. De Moor, “N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems,” Automatica, vol. 30, no. 1, pp. 75–93, 1994.
  11. M. Verhaegen and P. Dewilde, “Subspace model identification part 2. Analysis of the elementary output-error state-space model identification algorithm,” International journal of control, vol. 56, no. 5, pp. 1211–1241, 1992.
  12. W. E. Larimore, “Canonical variate analysis in identification, filtering, and adaptive control,” in 29th IEEE Conference on Decision and control.   IEEE, 1990, pp. 596–604.
  13. P. Van Overschee and B. De Moor, “A unifying theorem for three subspace system identification algorithms,” Automatica, vol. 31, no. 12, pp. 1853–1864, 1995.
  14. S. J. Qin, W. Lin, and L. Ljung, “A novel subspace identification approach with enforced causal models,” Automatica, vol. 41, no. 12, pp. 2043–2053, 2005.
  15. S. J. Qin and L. Ljung, “Parallel QR implementation of subspace identification with parsimonious models,” IFAC Proceedings Volumes, vol. 36, no. 16, pp. 1591–1596, 2003.
  16. G. Pannocchia and M. Calosi, “A predictor form PARSIMonious algorithm for closed-loop subspace identification,” Journal of Process Control, vol. 20, no. 4, pp. 517–524, 2010.
  17. J. M. Maciejowski, “Guaranteed stability with subspace methods,” Systems & Control Letters, vol. 26, no. 2, pp. 153–156, 1995.
  18. B. Boots, G. J. Gordon, and S. Siddiqi, “A constraint generation approach to learning stable linear dynamical systems,” Advances in neural information processing systems, vol. 20, 2007.
  19. S. L. Lacy and D. S. Bernstein, “Subspace identification with guaranteed stability using constrained optimization,” IEEE Transactions on automatic control, vol. 48, no. 7, pp. 1259–1263, 2003.
  20. N. Gillis, M. Karow, and P. Sharma, “A note on approximating the nearest stable discrete-time descriptor systems with fixed rank,” Applied Numerical Mathematics, vol. 148, pp. 131–139, 2020.
  21. W. Jongeneel, T. Sutter, and D. Kuhn, “Efficient learning of a linear dynamical system with stability guarantees,” IEEE Transactions on Automatic Control, 2022.
  22. G. Mamakoukas, I. Abraham, and T. D. Murphey, “Learning stable models for prediction and control,” IEEE Transactions on Robotics, 2023.
  23. A. Haber and M. Verhaegen, “Subspace identification of large-scale interconnected systems,” IEEE Transactions on Automatic Control, vol. 59, no. 10, pp. 2754–2759, 2014.
  24. P. Massioni and M. Verhaegen, “Subspace identification of circulant systems,” Automatica, vol. 44, no. 11, pp. 2825–2833, 2008.
  25. ——, “Subspace identification of distributed, decomposable systems,” in Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.   IEEE, 2009, pp. 3364–3369.
  26. C. Yu and M. Verhaegen, “Subspace identification of distributed clusters of homogeneous systems,” IEEE Transactions on Automatic Control, vol. 62, no. 1, pp. 463–468, 2016.
  27. Y. Li, Z. O’Neill, L. Zhang, J. Chen, P. Im, and J. DeGraw, “Grey-box modeling and application for building energy simulations-A critical review,” Renewable and Sustainable Energy Reviews, vol. 146, p. 111174, 2021.
  28. F. Zapf and T. Wallek, “Gray-box surrogate models for flash, distillation and compression units of chemical processes,” Computers & Chemical Engineering, vol. 155, p. 107510, 2021.
  29. O. O. Obadina, M. A. Thaha, Z. Mohamed, and M. H. Shaheed, “Grey-box modelling and fuzzy logic control of a Leader–Follower robot manipulator system: A hybrid Grey Wolf–Whale Optimisation approach,” ISA transactions, vol. 129, pp. 572–593, 2022.
  30. C. Yu, L. Ljung, A. Wills, and M. Verhaegen, “Constrained subspace method for the identification of structured state-space models (COSMOS),” IEEE Transactions on Automatic Control, vol. 65, no. 10, pp. 4201–4214, 2019.
  31. M. Schoukens, “Improved initialization of state-space artificial neural networks,” in 2021 European Control Conference (ECC).   IEEE, 2021, pp. 1913–1918.
  32. Y. N. Dauphin and S. Schoenholz, “Metainit: Initializing learning by learning to initialize,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  33. M. Zakwan, L. Di Natale, B. Svetozarevic, P. Heer, C. N. Jones, and G. F. Trecate, “Physically consistent neural ODEs for learning multi-physics systems,” arXiv preprint arXiv:2211.06130, 2022.
  34. L. Di Natale, B. Svetozarevic, P. Heer, and C. N. Jones, “Towards scalable physically consistent neural networks: An application to data-driven multi-zone thermal building models,” Applied Energy, vol. 340, p. 121071, 2023.
  35. K. Loya, J. Buzhardt, and P. Tallapragada, “Koopman operator based predictive control with a data archive of observables,” ASME Letters in Dynamic Systems and Control, pp. 1–7, 2023.
  36. J. C. Schulze and A. Mitsos, “Data-Driven Nonlinear Model Reduction Using Koopman Theory: Integrated Control Form and NMPC Case Study,” IEEE Control Systems Letters, vol. 6, pp. 2978–2983, 2022.
  37. J. C. Schulze, D. T. Doncevic, and A. Mitsos, “Identification of MIMO Wiener-type Koopman models for data-driven model reduction using deep learning,” Computers & Chemical Engineering, vol. 161, p. 107781, 2022.
  38. H. Choi, V. De Angelis, and Y. Preger, “Data-driven Battery Modeling based on Koopman Operator Approximation using Neural Network,” in 2023 IEEE Power & Energy Society General Meeting (PESGM).   IEEE, 2023, pp. 1–5.
  39. M. A. Lones, “How to avoid machine learning pitfalls: a guide for academic researchers,” arXiv preprint arXiv:2108.02497, 2021.
  40. M. C. De Oliveira, J. Bernussou, and J. C. Geromel, “A new discrete-time robust stability condition,” Systems & control letters, vol. 37, no. 4, pp. 261–265, 1999.
  41. M. Revay, R. Wang, and I. R. Manchester, “Recurrent equilibrium networks: Flexible dynamic models with guaranteed stability and robustness,” IEEE Transactions on Automatic Control, 2023.
  42. J. Z. Kolter and G. Manek, “Learning stable deep dynamics models,” Advances in neural information processing systems, vol. 32, 2019.
  43. D. Martinelli, C. L. Galimberti, I. R. Manchester, L. Furieri, and G. Ferrari-Trecate, “Unconstrained Parametrization of Dissipative and Contracting Neural Ordinary Differential Equations,” arXiv preprint arXiv:2304.02976, 2023.
  44. G. Mamakoukas, O. Xherija, and T. Murphey, “Memory-efficient learning of stable linear dynamical systems for prediction and control,” Advances in Neural Information Processing Systems, vol. 33, pp. 13 527–13 538, 2020.
  45. S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer, “Scheduled sampling for sequence prediction with recurrent neural networks,” Advances in neural information processing systems, vol. 28, 2015.
  46. C. Yu, L. Ljung, and M. Verhaegen, “Identification of structured state-space models,” Automatica, vol. 90, pp. 54–61, 2018.
  47. S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proceedings of the national academy of sciences, vol. 113, no. 15, pp. 3932–3937, 2016.
  48. M. Chilali and P. Gahinet, “H/sub/spl infin//design with pole placement constraints: an lmi approach,” IEEE Transactions on automatic control, vol. 41, no. 3, pp. 358–367, 1996.
Citations (5)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

GitHub

  1. GitHub - Cemempamoi/simba: SIMBa (14 stars)