Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
140 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning of Linear Dynamical Systems as a Non-Commutative Polynomial Optimization Problem (2002.01444v6)

Published 4 Feb 2020 in math.OC, cs.LG, cs.SY, eess.SY, and stat.ML

Abstract: There has been much recent progress in forecasting the next observation of a linear dynamical system (LDS), which is known as the improper learning, as well as in the estimation of its system matrices, which is known as the proper learning of LDS. We present an approach to proper learning of LDS, which in spite of the non-convexity of the problem, guarantees global convergence of numerical solutions to a least-squares estimator. We present promising computational results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Some questions in the theory of moments, volume 2. American Mathematical Society, 1962.
  2. Online learning for time series prediction. In COLT 2013 - The 26th Annual Conference on Learning Theory, June 12-14, 2013, Princeton University, NJ, USA, 2013.
  3. Infectious diseases of humans: dynamics and control. Oxford university press, 1992.
  4. A new approach to learning linear dynamical systems. arXiv preprint arXiv:2301.09519, 2023.
  5. A globally convergent approach for quantum control and system identification. Bulletin of the American Physical Society, 2023.
  6. Recovering models of open quantum systems from data via polynomial optimization: Towards globally convergent quantum system identification. arXiv preprint arXiv:2203.17164, 2022.
  7. Facial reduction for a cone-convex programming problem. Journal of the Australian Mathematical Society, 30(3):369–380, 1981.
  8. Optimization of polynomials in non-commuting variables. Springer, 2016.
  9. Finite sample properties of system identification methods. IEEE Transactions on Automatic Control, 47(8):1329–1334, 2002.
  10. Recent developments in parameter estimation and structure identification of biochemical and genomic systems. Mathematical biosciences, 219(2):57–83, 2009.
  11. Jacques Dixmier. Les C*-algèbres et leurs représentations. Gauthier-Villars, Paris, France, 1969. English translation: C*-algebras (North-Holland, 1982).
  12. Finite time identification in unstable linear systems. Automatica, 96:342–353, 2018.
  13. Exploiting sparsity in semidefinite programming via matrix completion i: General framework. SIAM Journal on Optimization, 11(3):647–674, 2001.
  14. Causal inference by identification of vector autoregressive processes with hidden components. In International Conference on Machine Learning, pages 1917–1925, 2015.
  15. On the imbedding of normed rings into the ring of operators in Hilbert space. Rec. Math. [Mat. Sbornik] N.S., 12(2):197–217, 1943.
  16. Gradient descent learns linear dynamical systems. Journal of Machine Learning Research, 19:1–44, 2018.
  17. Spectral filtering for general linear dynamical systems. In Advances in Neural Information Processing Systems, pages 4634–4643, 2018.
  18. Learning linear dynamical systems via spectral filtering. In Advances in Neural Information Processing Systems, pages 6702–6712, 2017.
  19. J William Helton. “Positive” noncommutative polynomials are sums of squares. Annals of Mathematics, 156(2):675–694, 2002.
  20. Detecting global optimality and extracting solutions in gloptipoly. In Positive polynomials in control, pages 293–310. Springer, 2005.
  21. Magnus Jansson. Subspace identification and ARX modeling. In Proceedings of the 13th IFAC SYSID Symposium, pages 1625–1630, 2003.
  22. Finite-time identification of stable linear systems optimality of the least-squares estimator. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 996–1001. IEEE, 2020.
  23. Tohru Katayama. Subspace methods for system identification. Springer Science & Business Media, 2006.
  24. Sparse noncommutative polynomial optimization. Mathematical Programming, 193(2):789–829, 2022.
  25. Minimizer extraction in polynomial optimization is robust. SIAM Journal on Optimization, 28(4):3177–3207, 2018.
  26. On-line learning of linear dynamical systems: Exponential forgetting in Kalman filters. In The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), pages 4098–4105, 2019. arXiv preprint arXiv:1809.05870.
  27. A two-step pre-processing for semidefinite programming. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 384–389. IEEE, 2020.
  28. Optimal data fitting: A moment approach. SIAM Journal on Optimization, 28(4):3127–3144, 2018.
  29. Computability of optimizers. arXiv preprint arXiv:2301.06148, 2023.
  30. Online arima algorithms for time series prediction. In Thirtieth AAAI conference on artificial intelligence, 2016.
  31. Lennart Ljung. Consistency of the least-squares identification method. IEEE Transactions on Automatic Control, 21(5):779–781, 1976.
  32. Lennart Ljung. System Identification: Theory for the User. Pearson Education, 1998.
  33. Recent scalability improvements for semidefinite programming with applications in machine learning, control, and robotics. Annual Review of Control, Robotics, and Autonomous Systems, 3, 2019.
  34. On the statistical treatment of linear stochastic difference equations. Econometrica, Journal of the Econometric Society, pages 173–220, 1943.
  35. Scott McCullough. Factorization of operator-valued polynomials in several non-commuting variables. Linear Algebra and its Applications, 326(1-3):193–203, 2001.
  36. MOSEK, ApS. The MOSEK Optimizer API for Python 9.2. 2020.
  37. Subspace identification for linear systems: Theory—Implementation—Applications. Springer Science & Business Media, 2012.
  38. Non-asymptotic identification of LTI systems from a single trajectory. In 2019 American Control Conference (ACC), pages 5655–5661. IEEE, 2019.
  39. Partial facial reduction: simplified, equivalent sdps via approximations of the psd cone. Mathematical Programming, 171(1-2):1–54, 2018.
  40. Convergent relaxations of polynomial optimization problems with noncommuting variables. SIAM Journal on Optimization, 20(5):2157–2180, 2010.
  41. Near optimal finite time identification of arbitrary linear dynamical systems. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5610–5618, Long Beach, California, USA, 09–15 Jun 2019. PMLR.
  42. Finite time LTI system identification. J. Mach. Learn. Res., 22(1), jul 2022.
  43. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.
  44. Irving E Segal. Irreducible representations of operator algebras. Bulletin of the American Mathematical Society, 53(2):73–88, 1947.
  45. Learning linear dynamical systems with semi-parametric least squares. In Conference on Learning Theory, pages 2714–2802. PMLR, 2019.
  46. Learning without mixing: Towards a sharp analysis of linear system identification. In Conference On Learning Theory, pages 439–473, 2018.
  47. Finite sample system identification: Optimal rates and the role of regularization. In Learning for Dynamics and Control, pages 16–25. PMLR, 2020.
  48. Arun K Tangirala. Principles of system identification: theory and practice. Crc Press, 2014.
  49. Sample complexity of Kalman filtering for unknown systems. In Learning for Dynamics and Control, pages 435–444. PMLR, 2020.
  50. Finite sample analysis of stochastic system identification. In 2019 IEEE 58th Conference on Decision and Control (CDC), pages 3648–3654. IEEE, 2019.
  51. Online learning of the Kalman filter with logarithmic regret. IEEE Transactions on Automatic Control, pages 1–16, 2022.
  52. Levent Tunçel. Potential reduction and primal-dual methods. In Handbook of semidefinite programming, pages 235–265. Springer, 2000.
  53. Sparsejsr: A fast algorithm to compute joint spectral radius via sparse sos decompositions. In 2021 American Control Conference (ACC), pages 2254–2259. IEEE, 2021.
  54. Exploiting term sparsity in noncommutative polynomial optimization. Computational Optimization and Applications, 80:483–521, 2021.
  55. Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension. SIAM Journal on Optimization, 31(1):114–141, 2021.
  56. Tssos: A moment-sos hierarchy that exploits term sparsity. SIAM Journal on Optimization, 31(1):30–58, 2021.
  57. Bayesian Forecasting and Dynamic Models (2nd ed.). Springer-Verlag, Berlin, Heidelberg, 1997.
  58. Peter Wittek. Algorithm 950: Ncpol2sdpa—sparse semidefinite programming relaxations for polynomial optimization problems of noncommuting variables. ACM Transactions on Mathematical Software (TOMS), 41(3):1–12, 2015.
  59. Proper learning of linear dynamical systems as a non-commutative polynomial optimisation problem. arXiv preprint arXiv:2002.01444, 2023. Version 4.
  60. Fairness in forecasting of observations of linear dynamical systems. Journal of Artificial Intelligence Research, 76:1247–1280, 2023.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now