Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Higher-order Spatio-temporal Physics-incorporated Graph Neural Network for Multivariate Time Series Imputation (2405.10995v2)

Published 16 May 2024 in cs.LG and cs.AI

Abstract: Exploring the missing values is an essential but challenging issue due to the complex latent spatio-temporal correlation and dynamic nature of time series. Owing to the outstanding performance in dealing with structure learning potentials, Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs) are often used to capture such complex spatio-temporal features in multivariate time series. However, these data-driven models often fail to capture the essential spatio-temporal relationships when significant signal corruption occurs. Additionally, calculating the high-order neighbor nodes in these models is of high computational complexity. To address these problems, we propose a novel higher-order spatio-temporal physics-incorporated GNN (HSPGNN). Firstly, the dynamic Laplacian matrix can be obtained by the spatial attention mechanism. Then, the generic inhomogeneous partial differential equation (PDE) of physical dynamic systems is used to construct the dynamic higher-order spatio-temporal GNN to obtain the missing time series values. Moreover, we estimate the missing impact by Normalizing Flows (NF) to evaluate the importance of each node in the graph for better explainability. Experimental results on four benchmark datasets demonstrate the effectiveness of HSPGNN and the superior performance when combining various order neighbor nodes. Also, graph-like optical flow, dynamic graphs, and missing impact can be obtained naturally by HSPGNN, which provides better dynamic analysis and explanation than traditional data-driven models. Our code is available at https://github.com/gorgen2020/HSPGNN.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Filling the g_ap_s: Multivariate time series imputation by graph neural networks. In ICLR 2022, pp.  1–20. 2022.
  2. Graph neural network: A comprehensive review on non-euclidean space. IEEE Access, 9:60588–60606, 2021.
  3. Optical flow based moving object detection and tracking for traffic surveillance. International Journal of Electrical, Computer, Energetic, Electronic and Communication Engineering, 7(9):1252–1256, 2013.
  4. Explainability techniques for graph convolutional networks. arXiv preprint arXiv:1905.13686, 2019.
  5. Toeplitz matrices, asymptotic linear algebra and functional analysis, volume 67. Springer, 2000.
  6. Physics-informed neural networks for heat transfer problems. Journal of Heat Transfer, 143(6):060801, 2021.
  7. Exact matrix completion via convex optimization. Communications of the ACM, 55(6):111–119, 2012.
  8. Brits: Bidirectional recurrent imputation for time series. Advances in neural information processing systems, 31, 2018.
  9. Nhits: Neural hierarchical interpolation for time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.  6989–6997, 2023.
  10. Missing traffic data imputation and pattern discovery with a bayesian augmented tensor factorization model. Transportation Research Part C: Emerging Technologies, 104:66–77, 2019.
  11. Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE transactions on fundamentals of electronics, communications and computer sciences, 92(3):708–721, 2009.
  12. Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1):21–27, 1967.
  13. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022.
  14. Saits: Self-attention-based imputation for time series. Expert Systems with Applications, 219:119619, 2023.
  15. Elman, J. L. Finding structure in time. Cognitive science, 14(2):179–211, 1990.
  16. Fan, J. Multi-mode deep matrix and tensor factorization. In international conference on learning representations, 2022.
  17. Incorporating physics-based models into data driven approaches for air leak detection in city buses. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  438–450. Springer, 2022.
  18. Time series data imputation: A survey on deep learning approaches. arXiv preprint arXiv:2011.11347, 2020.
  19. Farlow, S. J. Partial differential equations for scientists and engineers. Courier Corporation, 1993.
  20. Effective deep memory networks for distant supervised relation extraction. In IJCAI, volume 17, pp.  1–7, 2017.
  21. Dynamic graph convolution network for traffic forecasting based on latent network of laplace matrix estimation. IEEE Transactions on Intelligent Transportation Systems, 23(2):1009–1018, 2020.
  22. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp.  922–929, 2019.
  23. Long short-term memory. Neural computation, 9:1735–80, 12 1997. doi: 10.1162/neco.1997.9.8.1735.
  24. Graphlime: Local interpretable model explanations for graph neural networks. IEEE Transactions on Knowledge and Data Engineering, 2022.
  25. A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection. arXiv preprint arXiv:2307.03759, 2023.
  26. Johnson, C. R. Matrix completion problems: a survey. In Matrix theory and applications, volume 40, pp.  171–198, 1990.
  27. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  28. Klinger, A. The vandermonde matrix. The American Mathematical Monthly, 74(5):571–574, 1967.
  29. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34:26548–26560, 2021.
  30. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926, 2017.
  31. Semantics-aware dynamic graph convolutional network for traffic flow forecasting. IEEE Transactions on Vehicular Technology, 2023.
  32. On kinematic waves ii. a theory of traffic flow on long crowded roads. Proceedings of the royal society of london. series a. mathematical and physical sciences, 229(1178):317–345, 1955.
  33. Low-rank matrix completion: A contemporary survey. IEEE Access, 7:94215–94237, 2019.
  34. knn ensembles with penalized dtw for multivariate time series imputation. In 2016 International Joint Conference on Neural Networks (IJCNN), pp.  2774–2781. IEEE, 2016.
  35. On the wave theory in heat conduction. 1994.
  36. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  10772–10781, 2019.
  37. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  38. Variational inference with normalizing flows. In International conference on machine learning, pp.  1530–1538. PMLR, 2015.
  39. Physics-incorporated convolutional recurrent neural networks for source identification and forecasting of dynamical systems. Neural Networks, 144:359–371, 2021.
  40. Partial differential equations in action: from modelling to theory, volume 147. Springer Nature, 2022.
  41. Interpreting graph neural networks for nlp with differentiable edge masking. In International Conference on Learning Representations, 2020.
  42. Higher-order explanations of graph neural networks via relevant walks. IEEE transactions on pattern analysis and machine intelligence, 44(11):7581–7596, 2021.
  43. A physics-informed deep learning paradigm for traffic state and fundamental diagram estimation. IEEE Transactions on Intelligent Transportation Systems, 23(8):11688–11698, 2021.
  44. Imputation for the analysis of missing values and prediction of time series data. In 2011 international conference on recent trends in information Technology (ICRTIT), pp.  1158–1163. IEEE, 2011.
  45. Strang, G. Introduction to linear algebra. SIAM, 2022.
  46. Tealab, A. Time series forecasting using artificial neural networks methodologies: A systematic review. Future Computing and Informatics Journal, 3(2):334–340, 2018.
  47. Theoretical and numerical modeling of membrane and bending elastic wave propagation in honeycomb thin layers and sandwiches. Journal of Sound and Vibration, 382:100–121, 2016.
  48. Trindade, A. ElectricityLoadDiagrams20112014. UCI Machine Learning Repository, 2015. DOI: https://doi.org/10.24432/C58C86.
  49. Advances in neural information processing systems. Attention is All you Need, 2017.
  50. Multiple imputation using chained equations: issues and guidance for practice. Statistics in medicine, 30(4):377–399, 2011.
  51. Inductive graph neural networks for spatiotemporal kriging. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  4478–4485, 2021.
  52. Hrst-lr: A hessian regularization spatio-temporal low rank algorithm for traffic data imputation. IEEE Transactions on Intelligent Transportation Systems, 2023.
  53. Robust corrupted data recovery and clustering via generalized transformed tensor low-rank representation. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  54. St-mvl: filling missing values in geo-sensory time series data. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016.
  55. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
  56. Xgnn: Towards model-level explanations of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  430–438, 2020.
  57. Explainability in graph neural networks: A taxonomic survey. IEEE transactions on pattern analysis and machine intelligence, 45(5):5782–5799, 2022.
  58. Transformed distribution matching for missing value imputation. In International Conference on Machine Learning, pp.  42159–42186. PMLR, 2023.
  59. Spatio-temporal neural networks for space-time series forecasting and relations discovery. In 2017 IEEE International Conference on Data Mining (ICDM), pp.  705–714. IEEE, 2017.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets