2000 character limit reached
A novel Reservoir Architecture for Periodic Time Series Prediction (2405.10102v1)
Published 16 May 2024 in cs.NE, cs.AI, cs.LG, and eess.AS
Abstract: This paper introduces a novel approach to predicting periodic time series using reservoir computing. The model is tailored to deliver precise forecasts of rhythms, a crucial aspect for tasks such as generating musical rhythm. Leveraging reservoir computing, our proposed method is ultimately oriented towards predicting human perception of rhythm. Our network accurately predicts rhythmic signals within the human frequency perception range. The model architecture incorporates primary and intermediate neurons tasked with capturing and transmitting rhythmic information. Two parameter matrices, denoted as c and k, regulate the reservoir's overall dynamics. We propose a loss function to adapt c post-training and introduce a dynamic selection (DS) mechanism that adjusts $k$ to focus on areas with outstanding contributions. Experimental results on a diverse test set showcase accurate predictions, further improved through real-time tuning of the reservoir via c and k. Comparative assessments highlight its superior performance compared to conventional models.
- E. W. Large, J. A. Herrera, and M. J. Velasco, “Neural networks for beat perception in musical rhythm,” Frontiers in systems neuroscience, vol. 9, p. 159, 2015.
- P. S. Katz, “Evolution of central pattern generators and rhythmic behaviours,” Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 371, no. 1685, p. 20150057, 2016.
- M. Khashei and M. Bijari, “A novel hybridization of artificial neural networks and arima models for time series forecasting,” Applied soft computing, vol. 11, no. 2, pp. 2664–2675, 2011.
- N. I. Sapankevych and R. Sankar, “Time series prediction using support vector machines: a survey,” IEEE computational intelligence magazine, vol. 4, no. 2, pp. 24–38, 2009.
- L. Cai, K. Janowicz, G. Mai, B. Yan, and R. Zhu, “Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting,” Transactions in GIS, vol. 24, no. 3, pp. 736–755, 2020.
- D. Verstraeten, B. Schrauwen, M. d’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural networks, vol. 20, no. 3, pp. 391–403, 2007.
- W. Maass, T. Natschläger, and H. Markram, “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural computation, vol. 14, no. 11, pp. 2531–2560, 2002.
- D. Kleyko, E. P. Frady, M. Kheffache, and E. Osipov, “Integer echo state networks: Efficient reservoir computing for digital hardware,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 4, pp. 1688–1701, 2022.
- M. Lukoševičius, “A practical guide to applying echo state networks,” in Neural Networks: Tricks of the Trade: Second Edition. Springer, 2012, pp. 659–686.
- D. Botteldooren, “Finite-difference time-domain simulation of low-frequency room acoustic problems,” The Journal of the Acoustical Society of America, vol. 98, no. 6, pp. 3302–3308, 1995.
- K. Kowalczyk and M. van Walstijn, “On-line simulation of 2d resonators with reduced dispersion error using compact implicit finite difference methods,” in 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07, vol. 1. IEEE, 2007, pp. I–285.
- P. Janata, J. L. Birk, J. D. Van Horn, M. Leman, B. Tillmann, and J. J. Bharucha, “The cortical topography of tonal structures underlying western music,” science, vol. 298, no. 5601, pp. 2167–2170, 2002.
- M. Schönwiesner and R. J. Zatorre, “Spectro-temporal modulation transfer function of single voxels in the human auditory cortex measured with high-resolution fmri,” Proceedings of the National Academy of Sciences, vol. 106, no. 34, pp. 14 611–14 616, 2009.
- M. Madadi Asl, A. Valizadeh, and P. A. Tass, “Dendritic and axonal propagation delays may shape neuronal networks with plastic synapses,” Frontiers in physiology, vol. 9, p. 1849, 2018.
- A. M. Taberner and M. C. Liberman, “Response properties of single auditory nerve fibers in the mouse,” Journal of neurophysiology, vol. 93, no. 1, pp. 557–569, 2005.
- S. W. Egger, N. M. Le, and M. Jazayeri, “A neural circuit model for human sensorimotor timing,” Nature communications, vol. 11, no. 1, p. 3933, 2020.
- C. Hamzaçebi, D. Akay, and F. Kutay, “Comparison of direct and iterative artificial neural network forecast approaches in multi-periodic time series forecasting,” Expert systems with applications, vol. 36, no. 2, pp. 3839–3844, 2009.
- T. E. Matthews, M. A. Witek, J. L. Thibodeau, P. Vuust, and V. B. Penhune, “Perceived motor synchrony with the beat is more strongly related to groove than measured synchrony,” Music Perception: An Interdisciplinary Journal, vol. 39, no. 5, pp. 423–442, 2022.
- C. Palmer and A. P. Demos, “Are we in time? how predictive coding and dynamical systems explain musical synchrony,” Current Directions in Psychological Science, vol. 31, no. 2, pp. 147–153, 2022.
- P. Vuust, M. J. Dietz, M. Witek, and M. L. Kringelbach, “Now you hear it: A predictive coding model for understanding rhythmic incongruity,” Annals of the New York Academy of Sciences, vol. 1423, no. 1, pp. 19–29, 2018.
- S. Koelsch, P. Vuust, and K. Friston, “Predictive processes and the peculiar case of music,” Trends in cognitive sciences, vol. 23, no. 1, pp. 63–77, 2019.
- O. A. Heggli, I. Konvalinka, M. L. Kringelbach, and P. Vuust, “Musical interaction is influenced by underlying predictive models and musical expertise,” Scientific reports, vol. 9, no. 1, p. 11048, 2019.
- M. T. Elliott, A. M. Wing, and A. E. Welchman, “Moving in time: Bayesian causal inference explains movement coordination to auditory beats,” Proceedings of the Royal Society B: Biological Sciences, vol. 281, no. 1786, p. 20140751, 2014.
- A. Pikovsky, M. Rosenblum, J. Kurths, and A. Synchronization, “A universal concept in nonlinear sciences,” Self, vol. 2, p. 3, 2001.
- I. Tal, E. W. Large, E. Rabinovitch, Y. Wei, C. E. Schroeder, D. Poeppel, and E. Z. Golumbic, “Neural entrainment to the beat: The “missing-pulse” phenomenon,” Journal of Neuroscience, vol. 37, no. 26, pp. 6331–6341, 2017.
- E. W. Large, “Resonating to musical rhythm: theory and experiment,” The psychology of time, pp. 189–231, 2008.
- I. R. Roman, A. Washburn, E. W. Large, C. Chafe, and T. Fujioka, “Delayed feedback embedded in perception-action coordination cycles results in anticipation behavior during synchronized rhythmic action: A dynamical systems approach,” PLoS computational biology, vol. 15, no. 10, p. e1007371, 2019.
- G. Tanaka, T. Matsumori, H. Yoshida, and K. Aihara, “Reservoir computing with diverse timescales for prediction of multiscale dynamics,” Physical Review Research, vol. 4, no. 3, p. L032014, 2022.
- L. Manneschi, M. O. Ellis, G. Gigante, A. C. Lin, P. Del Giudice, and E. Vasilaki, “Exploiting multiple timescales in hierarchical echo state networks,” Frontiers in Applied Mathematics and Statistics, vol. 6, p. 616658, 2021.
- C. Gallicchio, “Euler state networks: Non-dissipative reservoir computing,” arXiv preprint arXiv:2203.09382, 2022.
- A. Rodan and P. Tino, “Minimum complexity echo state network,” IEEE transactions on neural networks, vol. 22, no. 1, pp. 131–144, 2010.
- D. Bacciu and A. Bongiorno, “Concentric esn: assessing the effect of modularity in cycle reservoirs,” in 2018 International Joint Conference on Neural Networks (IJCNN). IEEE, 2018, pp. 1–8.
- C. Gallicchio, A. Micheli, and L. Pedrelli, “Deep reservoir computing: A critical experimental analysis,” Neurocomputing, vol. 268, pp. 87–99, 2017.
- H. Jaeger, “Echo state network,” scholarpedia, vol. 2, no. 9, p. 2330, 2007.
- G. R. Yang, M. R. Joglekar, H. F. Song, W. T. Newsome, and X.-J. Wang, “Task representations in neural networks trained to perform many cognitive tasks,” Nature neuroscience, vol. 22, no. 2, pp. 297–306, 2019.