Structured state-space models are deep Wiener models (2312.06211v2)
Abstract: The goal of this paper is to provide a system identification-friendly introduction to the Structured State-space Models (SSMs). These models have become recently popular in the machine learning community since, owing to their parallelizability, they can be efficiently and scalably trained to tackle extremely-long sequence classification and regression problems. Interestingly, SSMs appear as an effective way to learn deep Wiener models, which allows to reframe SSMs as an extension of a model class commonly used in system identification. In order to stimulate a fruitful exchange of ideas between the machine learning and system identification communities, we deem it useful to summarize the recent contributions on the topic in a structured and accessible form. At last, we highlight future research directions for which this community could provide impactful contributions.
- Deep convolutional networks in system identification. In 2019 IEEE 58th Conference on Decision and Control (CDC), 3670–3676. IEEE.
- Deep learning, volume 1. MIT press Massachusetts, USA.
- Recurrent neural networks for short-term load forecasting: an overview and comparative analysis. Springer.
- Blelloch, G.E. (1990). Prefix sums and their applications. In J.H. Reif (ed.), Synthesis of parallel algorithms. Morgan Kaufmann Publishers Inc.
- On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments. Journal of Process Control, 114, 92–104.
- Nonlinear MPC design for incrementally ISS systems with application to GRU networks. Automatica, 159, 111381.
- Hippo: Recurrent memory with optimal polynomial projections. Advances in neural information processing systems, 33, 1474–1487.
- On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems, 35, 35971–35983.
- Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396.
- Diagonal state spaces are as effective as structured state spaces. Advances in Neural Information Processing Systems, 35, 22982–22994.
- Kumar, S.K. (2017). On weight initialization in deep neural networks. arXiv preprint arXiv:1704.08863.
- Recurrent neural network based MPC for process industries. In 2019 18th European Control Conference (ECC), 1005–1010. IEEE.
- Estimation of grey box and black box models for non-linear circuit data. IFAC Proceedings Volumes, 37(13), 399–404.
- Improved initialization for nonlinear state-space modeling. IEEE Transactions on instrumentation and Measurement, 63(4), 972–980.
- Nonlinear system identification using temporal convolutional networks: a silverbox study. IFAC-PapersOnLine, 52(29), 186–191.
- Stable recurrent models. In International Conference on Learning Representations. ArXiv preprint arXiv:1805.10369.
- Resurrecting recurrent neural networks for long sequences. arXiv preprint arXiv:2303.06349.
- System identification: a frequency domain approach. John Wiley & Sons.
- Searching for activation functions. arXiv preprint arXiv:1710.05941.
- Nonlinear system identification: A user-oriented road map. IEEE Control Systems Magazine, 39(6), 28–99.
- Identification of block-oriented nonlinear systems starting from linear approximations: A survey. Automatica, 85, 272–292.
- Simplified state space layers for sequence modeling. In The Eleventh International Conference on Learning Representations.
- Efficient mask attention-based narmax (mab-narmax) model identification. In 2022 27th International Conference on Automation and Computing (ICAC), 1–6. IEEE.
- Learning-based predictive control of the cooling system of a large business centre. Control Engineering Practice, 97, 104348.
- Tiels, K. (2015). Wiener system identification with generalized orthonormal basis functions. PhD thesis, Vrije Universiteit Brussell.
- Three free data sets for development and benchmarking in nonlinear system identification. In 2013 European control conference (ECC), 2933–2938. IEEE.
- Identification of structured state-space models. Automatica, 90, 54–61.