2000 character limit reached
A finite-sample generalization bound for stable LPV systems
Published 16 May 2024 in cs.LG, cs.SY, and eess.SY | (2405.10054v3)
Abstract: One of the main theoretical challenges in learning dynamical systems from data is providing upper bounds on the generalization error, that is, the difference between the expected prediction error and the empirical prediction error measured on some finite sample. In machine learning, a popular class of such bounds are the so-called Probably Approximately Correct (PAC) bounds. In this paper, we derive a PAC bound for stable continuous-time linear parameter-varying (LPV) systems. Our bound depends on the H2 norm of the chosen class of the LPV systems, but does not depend on the time interval for which the signals are considered.
- P. Alquier. User-friendly introduction to pac-bayes bounds. arXiv preprint arXiv:2110.11216, 2021.
- A. C. Antoulas. Approximation of large-scale dynamical systems. SIAM, 2005.
- P. Bilingsley. Probability and measure. Wiley, 1986.
- M. C. Campi and E. Weyer. Finite sample properties of system identification methods. IEEE TACON, 47(8):1329–1334, 2002.
- On generalization bounds of a family of recurrent neural networks. In AISTATS, 2020.
- Neural ordinary differential equations. NeurIPS, 31, 2018.
- Framing rnn as a kernel method: A neural ode approach. NeurIPS, 2021.
- D. Foster and M. Simchowitz. Logarithmic regret for adversarial online control. In ICML, PMLR. PMLR, 2020.
- Reduced-order modeling of lpv systems in the loewner framework. In CDC, 2021.
- A. Gu and T. Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2023.
- Combining recurrent, convolutional, and continuous-time models with linear state space layers. In NeurIPS, 2021.
- How to train your HIPPO: State space models with generalized orthogonal basis projections. In ICLR, 2023.
- J. Hanson and M. Raginsky. Rademacher complexity of neural odes via chen-fliess series. arXiv preprint arXiv:2401.16655, 2024.
- Learning recurrent neural net models of nonlinear systems. In L4DC, 2021.
- A. Isidori. Nonlinear control systems: an introduction. Springer, 1985.
- Generalization error bounds for deep unfolding rnns. In UAI, 2021.
- P. Kidger. On neural differential equations. arXiv preprint arXiv:2202.02435, 2022.
- P. Koiran and E. D. Sontag. Vapnik-chervonenkis dimension of recurrent neural networks. Discrete Applied Mathematics, 86(1):63–79, 1998.
- A. Krener. Bilinear and nonlinear realization of input-output maps. SIAM Journal on Control, 13(4), 1974.
- Learning complexity dimensions for a continuous-time control system. SIAM journal on control and optimization, 43(3):872–898, 2004.
- Logarithmic regret bound in partially observable linear dynamical systems. Advances in NeurIPS, 33:20876–20888, 2020.
- On the curse of memory in recurrent neural networks: Approximation and optimization analysis. In ICLR, 2021.
- L. Ljung. System identification. In Signal analysis and prediction, pages 163–173. Springer, 1998.
- P. Marion. Generalization bounds for neural ordinary differential equations and deep residual networks. arXiv preprint arXiv:2305.06648, 2023.
- Dissecting neural odes. NeurIPS, 2020.
- Resurrecting recurrent neural networks for long sequences. arXiv preprint arXiv:2303.06349, 2023.
- S. Oymak and N. Ozay. Revisiting ho–kalman-based system identification: Robustness and finite-sample analysis. IEEE TACON, 67(4):1914–1928, 4 2022.
- C. W. Scherer. Robust controller synthesis is convex for systems without control channel uncertainties. In Model-Based Control: Bridging Rigorous Theory and Advanced Technology, pages 13–31. Springer, 2009.
- S. Shalev-Shwartz and S. Ben-David. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
- Learning linear dynamical systems with semi-parametric least squares. In COLT, 2019.
- E. D. Sontag. A learning result for continuous-time recurrent neural networks. Systems & control letters, 34(3):151–158, 1998.
- R. Tóth. Modeling and identification of linear parameter-varying systems, volume 403. Springer, 2010.
- Statistical learning theory for control: A finite-sample perspective. IEEE Control Systems Magazine, 43(6):67–97, 2023.
- M. Vidyasagar and R. L. Karandikar. A learning theory approach to system identification and stochastic adaptive control. Probabilistic and randomized methods for design under uncertainty, pages 265–302, 2006.
- S. Wang and B. Xue. State-space models with layer-wise nonlinearity are universal approximators with exponential decaying memory. In NeurIPS, 2023.
- C. Wei and T. Ma. Data-dependent sample complexity of deep neural networks via lipschitz augmentation. In NeurIPS, 2019.
- L. Zhang and J. Lam. On ℋ2subscriptℋ2\mathcal{H}_{2}caligraphic_H start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT model reduction of bilinear systems. Automatica, 38:205–216, 2002.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
