Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Spatiotemporal Observer Design for Predictive Learning of High-Dimensional Data (2402.15284v1)

Published 23 Feb 2024 in cs.LG, cs.AI, cs.SY, and eess.SY

Abstract: Although deep learning-based methods have shown great success in spatiotemporal predictive learning, the framework of those models is designed mainly by intuition. How to make spatiotemporal forecasting with theoretical guarantees is still a challenging issue. In this work, we tackle this problem by applying domain knowledge from the dynamical system to the framework design of deep learning models. An observer theory-guided deep learning architecture, called Spatiotemporal Observer, is designed for predictive learning of high dimensional data. The characteristics of the proposed framework are twofold: firstly, it provides the generalization error bound and convergence guarantee for spatiotemporal prediction; secondly, dynamical regularization is introduced to enable the model to learn system dynamics better during training. Further experimental results show that this framework could capture the spatiotemporal dynamics and make accurate predictions in both one-step-ahead and multi-step-ahead forecasting scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. G. Atluri, A. Karpatne, and V. Kumar, “Spatio-temporal data mining: A survey of problems and methods,” ACM Computing Surveys (CSUR), vol. 51, no. 4, pp. 1–41, 2018.
  2. I. Lana, J. Del Ser, M. Velez, and E. I. Vlahogianni, “Road traffic forecasting: Recent advances and new challenges,” IEEE Intelligent Transportation Systems Magazine, vol. 10, no. 2, pp. 93–109, 2018.
  3. Z. Jiang, “A survey on spatial and spatiotemporal prediction methods,” arXiv preprint arXiv:2012.13384, 2020.
  4. G. A. Davis and N. L. Nihan, “Nonparametric regression and short-term freeway traffic forecasting,” Journal of Transportation Engineering, vol. 117, no. 2, pp. 178–188, 1991.
  5. W.-C. Hong, “Traffic flow forecasting by seasonal svr with chaotic simulated annealing algorithm,” Neurocomputing, vol. 74, no. 12-13, pp. 2096–2107, 2011.
  6. S. Sarkka and J. Hartikainen, “Infinite-dimensional kalman filtering approach to spatio-temporal gaussian process regression,” in Artificial Intelligence and Statistics.   PMLR, 2012, pp. 993–1001.
  7. J. Zhang, Y. Zheng, D. Qi, R. Li, and X. Yi, “Dnn-based prediction model for spatio-temporal data,” in Proceedings of the 24th ACM SIGSPATIAL international conference on advances in geographic information systems, 2016, pp. 1–4.
  8. J. Zhang, Y. Zheng, and D. Qi, “Deep spatio-temporal residual networks for citywide crowd flows prediction,” in Thirty-first AAAI conference on artificial intelligence, 2017.
  9. S. Guo, Y. Lin, S. Li, Z. Chen, and H. Wan, “Deep spatial–temporal 3d convolutional neural networks for traffic data forecasting,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 10, pp. 3913–3926, 2019.
  10. X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” Advances in neural information processing systems, vol. 28, 2015.
  11. Y. Wang, J. Zhang, H. Zhu, M. Long, J. Wang, and P. S. Yu, “Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9154–9162.
  12. H. Wu, Z. Yao, J. Wang, and M. Long, “Motionrnn: A flexible model for video prediction with spacetime-varying motions,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 15 435–15 444.
  13. N. Wahlström, T. B. Schön, and M. P. Deisenroth, “From pixels to torques: Policy learning with deep dynamical models,” arXiv preprint arXiv:1502.02251, 2015.
  14. S. Wang, J. Cao, and P. Yu, “Deep learning for spatio-temporal data mining: A survey,” IEEE transactions on knowledge and data engineering, 2020.
  15. A. Karpatne, G. Atluri, J. H. Faghmous, M. Steinbach, A. Banerjee, A. Ganguly, S. Shekhar, N. Samatova, and V. Kumar, “Theory-guided data science: A new paradigm for scientific discovery from data,” IEEE Transactions on knowledge and data engineering, vol. 29, no. 10, pp. 2318–2331, 2017.
  16. Z. Gao, C. Tan, L. Wu, and S. Z. Li, “Simvp: Simpler yet better video prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3170–3180.
  17. S. Oprea, P. Martinez-Gonzalez, A. Garcia-Garcia, J. A. Castro-Vargas, S. Orts-Escolano, J. Garcia-Rodriguez, and A. Argyros, “A review on deep learning techniques for video prediction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 6, pp. 2806–2826, 2020.
  18. Z. C. Lipton, J. Berkowitz, and C. Elkan, “A critical review of recurrent neural networks for sequence learning,” arXiv preprint arXiv:1506.00019, 2015.
  19. X. Shi, Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-k. Wong, and W.-c. Woo, “Deep learning for precipitation nowcasting: A benchmark and a new model,” Advances in neural information processing systems, vol. 30, 2017.
  20. Y. Wang, M. Long, J. Wang, Z. Gao, and P. S. Yu, “Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms,” Advances in neural information processing systems, vol. 30, 2017.
  21. Y. Wang, Z. Gao, M. Long, J. Wang, and S. Y. Philip, “Predrnn++: Towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning,” in International Conference on Machine Learning.   PMLR, 2018, pp. 5123–5132.
  22. C. Ling, J. Zhong, and W. Li, “Predictive coding based multiscale network with encoder-decoder lstm for video prediction,” arXiv preprint arXiv:2212.11642, 2022.
  23. Z. Ma, H. Zhang, and J. Liu, “Ms-rnn: A flexible multi-scale framework for spatiotemporal predictive learning,” arXiv preprint arXiv:2206.03010, 2022.
  24. R. Villegas, J. Yang, S. Hong, X. Lin, and H. Lee, “Decomposing motion and content for natural video sequence prediction,” arXiv preprint arXiv:1706.08033, 2017.
  25. Y. Wang, L. Jiang, M.-H. Yang, L.-J. Li, M. Long, and L. Fei-Fei, “Eidetic 3d lstm: A model for video prediction and beyond,” in International conference on learning representations, 2019.
  26. W. Yu, Y. Lu, S. Easterbrook, and S. Fidler, “Efficient and information-preserving future frame prediction and beyond,” in International Conference on Learning Representations, 2019.
  27. V. L. Guen and N. Thome, “Disentangling physical dynamics from unknown factors for unsupervised video prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 474–11 484.
  28. A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis et al., “Deep learning for computer vision: A brief review,” Computational intelligence and neuroscience, vol. 2018, 2018.
  29. J. Oh, X. Guo, H. Lee, R. L. Lewis, and S. Singh, “Action-conditional video prediction using deep networks in atari games,” Advances in neural information processing systems, vol. 28, 2015.
  30. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497.
  31. Z. Liu, R. A. Yeh, X. Tang, Y. Liu, and A. Agarwala, “Video frame synthesis using deep voxel flow,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 4463–4471.
  32. Z. Xu, Y. Wang, M. Long, J. Wang, and M. KLiss, “Predcnn: Predictive learning with cascade convolutions.” in IJCAI, 2018, pp. 2940–2947.
  33. L. Brivadis, V. Andrieu, and U. Serres, “Luenberger observers for discrete-time nonlinear systems,” in 2019 IEEE 58th Conference on Decision and Control (CDC).   IEEE, 2019, pp. 3435–3440.
  34. J. Peralez and M. Nadri, “Deep learning-based luenberger observer design for discrete-time nonlinear systems,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 4370–4375.
  35. L. d. C. Ramos, F. Di Meglio, V. Morgenthaler, L. F. F. da Silva, and P. Bernard, “Numerical design of luenberger observers for nonlinear systems,” in 2020 59th IEEE Conference on Decision and Control (CDC).   IEEE, 2020, pp. 5435–5442.
  36. F. Petropoulos, D. Apiletti, V. Assimakopoulos, M. Z. Babai, D. K. Barrow, S. B. Taieb, C. Bergmeir, R. J. Bessa, J. Bijak, J. E. Boylan et al., “Forecasting: theory and practice,” International Journal of Forecasting, 2022.
  37. T. Teräsvirta, D. Tjøstheim, and C. W. Granger, “Modelling nonlinear economic time series,” 2010.
  38. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
  39. S. Lin and J. Zhang, “Generalization bounds for convolutional neural networks,” arXiv preprint arXiv:1910.01487, 2019.
  40. F. He, T. Liu, and D. Tao, “Why resnet works? residuals generalize,” IEEE transactions on neural networks and learning systems, vol. 31, no. 12, pp. 5349–5362, 2020.
  41. E. J. McShane, “Extension of range of functions,” Bulletin of the American Mathematical Society, vol. 40, no. 12, pp. 837–842, 1934.
  42. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  43. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, 2019.
  44. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
  45. N. Kalchbrenner, A. Oord, K. Simonyan, I. Danihelka, O. Vinyals, A. Graves, and K. Kavukcuoglu, “Video pixel networks,” in International Conference on Machine Learning.   PMLR, 2017, pp. 1771–1779.
  46. M. Oliu, J. Selva, and S. Escalera, “Folded recurrent neural networks for future video prediction,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 716–731.
  47. Y. Wang, L. Jiang, M.-H. Yang, L.-J. Li, M. Long, and L. Fei-Fei, “Eidetic 3d lstm: A model for video prediction and beyond,” in International conference on learning representations, 2018.
  48. N. Srivastava, E. Mansimov, and R. Salakhudinov, “Unsupervised learning of video representations using lstms,” in International conference on machine learning.   PMLR, 2015, pp. 843–852.
  49. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  50. C. Tan, S. Li, Z. Gao, W. Guan, Z. Wang, Z. Liu, L. Wu, and S. Z. Li, “Openstl: A comprehensive benchmark of spatio-temporal predictive learning,” arXiv preprint arXiv:2306.11249, 2023.
  51. X. Jia, B. De Brabandere, T. Tuytelaars, and L. V. Gool, “Dynamic filter networks,” Advances in neural information processing systems, vol. 29, 2016.
  52. Z. Zhang, C. Luo, S. Feng, R. Ye, Y. Ye, and X. Li, “Rap-net: Region attention predictive network for precipitation nowcasting,” Geoscientific Model Development Discussions, pp. 1–19, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now