Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Finite Sample Analysis for a Class of Subspace Identification Methods (2404.17331v1)

Published 26 Apr 2024 in eess.SY and cs.SY

Abstract: While subspace identification methods (SIMs) are appealing due to their simple parameterization for MIMO systems and robust numerical realizations, a comprehensive statistical analysis of SIMs remains an open problem, especially in the non-asymptotic regime. In this work, we provide a finite sample analysis for a class of SIMs, which reveals that the convergence rates for estimating Markov parameters and system matrices are $\mathcal{O}(1/\sqrt{N})$, in line with classical asymptotic results. Based on the observation that the model format in classical SIMs becomes non-causal because of a projection step, we choose a parsimonious SIM that bypasses the projection step and strictly enforces a causal model to facilitate the analysis, where a bank of ARX models are estimated in parallel. Leveraging recent results from finite sample analysis of an individual ARX model, we obtain an overall error bound of an array of ARX models and proceed to derive error bounds for system matrices via robustness results for the singular value decomposition.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. B. L. Ho and R. E. Kálmán, “Effective construction of linear state-variable models from input/output functions,” Automatisierungstechnik, vol. 14, no. 1-12, pp. 545–548, 1966.
  2. S. J. Qin and L. Ljung, “Closed-loop subspace identification with innovation estimation,” in Proc. 13th IFAC Symp. Syst. Identification, Netherlands, 2003.
  3. M. Jansson, “Subspace identification and ARX modeling,” in Proc. 13th IFAC Symp. Syst. Identification, Netherlands, 2003.
  4. L. Ljung and T. McKelvey, “Subspace identification from closed loop data,” Signal Process., vol. 52, no. 2, pp. 209–215, 1996.
  5. A. Chiuso and G. Picci, “Consistency analysis of some closed-loop subspace identification methods,” Automatica, vol. 41, no. 3, pp. 377–391, 2005.
  6. S. J. Qin, “An overview of subspace identification,” Comput. Chem. Eng., vol. 30, no. 10-12, pp. 1502–1513, 2006.
  7. G. Van der Veen, J.-W. van Wingerden, M. Bergamasco, M. Lovera, and M. Verhaegen, “Closed-loop subspace identification methods: An overview,” IET Control Theory Appl., vol. 7, no. 10, pp. 1339–1358, 2013.
  8. M. Jansson and B. Wahlberg, “On consistency of subspace methods for system identification,” Automatica, vol. 34, no. 12, pp. 1507–1519, 1998.
  9. T. Gustafsson, “Subspace-based system identification: weighting and pre-filtering of instruments,” Automatica, vol. 38, no. 3, pp. 433–443, 2002.
  10. M. Jansson, “Asymptotic variance analysis of subspace identification methods,” Proc. 12th IFAC Symp. Syst. Identification, pp. 91–96, 2000.
  11. D. Bauer, “Asymptotic properties of subspace estimators,” Automatica, vol. 41, no. 3, pp. 359–376, 2005.
  12. A. Chiuso and G. Picci, “The asymptotic variance of subspace estimates,” J. Econom., vol. 118, no. 1-2, pp. 257–291, 2004.
  13. A. Chiuso, “On the relation between CCA and predictor-based subspace identification,” IEEE Trans. Autom. Control, vol. 52, no. 10, pp. 1795–1812, 2007.
  14. A. Chiuso, “The role of vector autoregressive modeling in predictor-based subspace identification,” Automatica, vol. 43, no. 6, pp. 1034–1048, 2007.
  15. A. Chiuso, “Some insights on the choice of the future horizon in closed-loop CCA-type subspace algorithms,” in 2007 Am. Control Conf., 2007, pp. 840–845.
  16. W. E. Larimore, “Statistical optimality and canonical variate analysis system identification,” Signal Process., vol. 52, no. 2, pp. 131–144, 1996.
  17. D. Bauer and L. Ljung, “Some facts about the choice of the weighting matrices in Larimore type of subspace algorithms,” Automatica, vol. 38, no. 5, pp. 763–773, 2002.
  18. M. C. Campi and E. Weyer, “Finite sample properties of system identification methods,” IEEE Trans. Autom. Control, vol. 47, no. 8, pp. 1329–1334, 2002.
  19. E. Weyer, R. C. Williamson, and I. M. Mareels, “Finite sample properties of linear model identification,” IEEE Trans. Autom. Control, vol. 44, no. 7, pp. 1370–1383, 1999.
  20. T. Sarkar and A. Rakhlin, “Near optimal finite time identification of arbitrary linear dynamical systems,” in Int. Conf. on Machine Learn., 2019, pp. 5610–5618.
  21. M. Simchowitz, H. Mania, S. Tu, M. I. Jordan, and B. Recht, “Learning without mixing: Towards a sharp analysis of linear system identification,” in Conf. On Learn. Theory, 2018.
  22. Y. Jedra and A. Proutiere, “Finite-time identification of linear systems: Fundamental limits and optimal algorithms,” IEEE Trans. Autom. Control, vol. 68, no. 5, pp. 2805–2820, 2022.
  23. A. Tsiamis and G. J. Pappas, “Finite sample analysis of stochastic system identification,” in IEEE Conf. Decis. Control, 2019, pp. 3648–3654.
  24. S. Oymak and N. Ozay, “Non-asymptotic identification of LTI systems from a single trajectory,” in Am. Control Conf., 2019, pp. 5655–5661.
  25. T. Sarkar, A. Rakhlin, and M. A. Dahleh, “Finite time LTI system identification,” J. Mach. Learn. Res., vol. 22, no. 1, pp. 1186–1246, 2021.
  26. S. Lale, K. Azizzadenesheli, B. Hassibi, and A. Anandkumar, “Finite-time system identification and adaptive control in autoregressive exogenous systems,” in Learn. Dyn. Control.   PMLR, 2021, pp. 967–979.
  27. A. Tsiamis, I. Ziemann, N. Matni, and G. J. Pappas, “Statistical learning theory for control: A finite-sample perspective,” IEEE Control Syst., vol. 43, no. 6, pp. 67–97, 2023.
  28. I. Ziemann, A. Tsiamis, B. Lee, Y. Jedra, N. Matni, and G. J. Pappas, “A tutorial on the non-asymptotic theory of system identification,” in IEEE Conf. Decis. and Control, 2023, pp. 8921–8939.
  29. S. J. Qin, W. Lin, and L. Ljung, “A novel subspace identification approach with enforced causal models,” Automatica, vol. 41, no. 12, pp. 2043–2053, 2005.
  30. P. Van Overschee and B. De Moor, “A unifying theorem for three subspace system identification algorithms,” Automatica, vol. 31, no. 12, pp. 1853–1864, 1995.
  31. I. Ziemann, “A note on the smallest eigenvalue of the empirical covariance of causal gaussian processes,” IEEE Trans. Autom. Control, vol. 69, no. 2, pp. 1372–1376, 2023.
  32. Y. Abbasi-Yadkori, D. Pál, and C. Szepesvári, “Online least squares estimation with self-normalized processes: An application to bandit problems,” arXiv preprint arXiv:1102.2670, 2011.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now