Papers
Topics
Authors
Recent
2000 character limit reached

Enhancing Topological Dependencies in Spatio-Temporal Graphs with Cycle Message Passing Blocks (2401.15894v2)

Published 29 Jan 2024 in cs.LG and cs.AI

Abstract: Graph Neural Networks (GNNs) and Transformer-based models have been increasingly adopted to learn the complex vector representations of spatio-temporal graphs, capturing intricate spatio-temporal dependencies crucial for applications such as traffic datasets. Although many existing methods utilize multi-head attention mechanisms and message-passing neural networks (MPNNs) to capture both spatial and temporal relations, these approaches encode temporal and spatial relations independently, and reflect the graph's topological characteristics in a limited manner. In this work, we introduce the Cycle to Mixer (Cy2Mixer), a novel spatio-temporal GNN based on topological non-trivial invariants of spatio-temporal graphs with gated multi-layer perceptrons (gMLP). The Cy2Mixer is composed of three blocks based on MLPs: A temporal block for capturing temporal properties, a message-passing block for encapsulating spatial information, and a cycle message-passing block for enriching topological information through cyclic subgraphs. We bolster the effectiveness of Cy2Mixer with mathematical evidence emphasizing that our cycle message-passing block is capable of offering differentiated information to the deep learning model compared to the message-passing block. Furthermore, empirical evaluations substantiate the efficacy of the Cy2Mixer, demonstrating state-of-the-art performances across various spatio-temporal benchmark datasets. The source code is available at \url{https://github.com/leemingo/cy2mixer}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Using dynamic time warping to find patterns in time series. In Proceedings of the 3rd international conference on knowledge discovery and data mining, pages 359–370, 1994.
  2. Weisfeiler and lehman go cellular: Cw networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 2625–2640. Curran Associates, Inc., 2021.
  3. Graph neural controlled differential equations for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6367–6374, 2022.
  4. Cycle to clique (cy2c) graph neural network: A sight to see beyond neighborhood aggregation. In The Eleventh International Conference on Learning Representations, 2022.
  5. St-norm: Spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 269–278, 2021.
  6. Dynamic spatial-temporal graph convolutional neural networks for traffic forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 890–897, 2019.
  7. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering, 34(11):5415–5428, 2021.
  8. Robin Hartshorne. Algebraic geometry. Graduate Texts in Mathematics, pages 60–68, 206–224, 1977.
  9. Allen Hatcher. Algebraic Topology. Cambridge University Press, 2002.
  10. Gaussian error linear units (gelus), 2023.
  11. Topological graph neural networks. arXiv preprint arXiv:2102.07835, 2021.
  12. Pdformer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction. arXiv preprint arXiv:2301.07945, 2023.
  13. Spatial-temporal fusion graph neural networks for traffic flow forecasting, 2021.
  14. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926, 2017.
  15. Dynamic spatial-temporal representation learning for traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 22(11):7169–7183, 2020.
  16. Pay attention to mlps. Advances in Neural Information Processing Systems, 34:9204–9215, 2021.
  17. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. arXiv preprint arXiv:2308.10425, 2023.
  18. Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1720–1730, 2019.
  19. The pwlr graph representation: A persistent weisfeiler-lehman scheme with random walks for graph classification. In Topological, Algebraic and Geometric Learning Workshops 2022, pages 287–297. PMLR, 2022.
  20. Keith Paton. An algorithm for finding a fundamental set of cycles of a graph. Communications of the ACM, 12(9):514–518, 1969.
  21. A hybrid model for spatiotemporal forecasting of pm2. 5 based on graph convolutional neural network and long short-term memory. Science of the Total Environment, 664:1–10, 2019.
  22. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 4454–4458, 2022.
  23. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. arXiv preprint arXiv:2206.09112, 2022.
  24. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 914–921, 2020.
  25. Short-term traffic forecasting: Overview of objectives and methods. Transport reviews, 24(5):533–557, 2004.
  26. Pm2. 5-gnn: A domain knowledge enhanced graph neural network for pm2. 5 forecasting. In Proceedings of the 28th international conference on advances in geographic information systems, pages 163–166, 2020.
  27. Traffic flow prediction via spatial temporal graph neural network. In Proceedings of the web conference 2020, pages 1082–1092, 2020.
  28. Libcity: An open library for traffic prediction. In Proceedings of the 29th international conference on advances in geographic information systems, pages 145–148, 2021.
  29. Graph wavenet for deep spatial-temporal graph modeling. arXiv preprint arXiv:1906.00121, 2019.
  30. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 753–763, 2020.
  31. How powerful are graph neural networks?, 2019.
  32. Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908, 2020.
  33. A literature survey on smart cities. Sci. China Inf. Sci., 58(10):1–18, 2015.
  34. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875, 2017.
  35. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1234–1241, 2020.
  36. A correlation information-based spatiotemporal network for traffic flow forecasting. Neural Computing and Applications, pages 1–19, 2023.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Slide Deck Streamline Icon: https://streamlinehq.com

Whiteboard

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up