Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts (2403.02600v1)

Published 5 Mar 2024 in cs.LG and cs.SI

Abstract: Accurate traffic forecasting is challenging due to the complex dependency on road networks, various types of roads, and the abrupt speed change due to the events. Recent works mainly focus on dynamic spatial modeling with adaptive graph embedding or graph attention having less consideration for temporal characteristics and in-situ modeling. In this paper, we propose a novel deep learning model named TESTAM, which individually models recurring and non-recurring traffic patterns by a mixture-of-experts model with three experts on temporal modeling, spatio-temporal modeling with static graph, and dynamic spatio-temporal dependency modeling with dynamic graph. By introducing different experts and properly routing them, TESTAM could better model various circumstances, including spatially isolated nodes, highly related nodes, and recurring and non-recurring events. For the proper routing, we reformulate a gating problem into a classification problem with pseudo labels. Experimental results on three public traffic network datasets, METR-LA, PEMS-BAY, and EXPY-TKY, demonstrate that TESTAM achieves a better indication and modeling of recurring and non-recurring traffic. We published the official code at https://github.com/HyunWookL/TESTAM

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Adaptive graph convolutional recurrent network for traffic forecasting. In Advances in Neural Information Processing Systems, volume 33, 2020.
  2. Spectral temporal graph neural network for multivariate time-series forecasting. In Advances in Neural Information Processing Systems, volume 33, 2020.
  3. Spatial mixture-of-experts. In Advances in Neural Information Processing Systems, volume 35, 2022.
  4. Learning factored representations in a deep mixture of experts. In International Conference on Learning Representations, 2014.
  5. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23:120:1–120:39, 2022.
  6. Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):3656–3663, 2019.
  7. Hypernetworks. In International Conference on Learning Representations, 2017.
  8. Spatio-temporal meta-graph learning for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.  8078–8086, 2023.
  9. A visual analytics system for improving attention-based traffic forecasting models. IEE Transaction on Visualization and Computer Graphics, 29(1):1102–1112, 2023.
  10. Time2vec: Learning a vector representation of time. CoRR, abs/1907.05321, 2019. URL http://arxiv.org/abs/1907.05321.
  11. A visual analytics system for exploring, monitoring, and forecasting road traffic congestion. IEEE Transactions on Visualization and Computer Graphics, 26(11):3133–3146, 2020.
  12. Learning to remember patterns: Pattern matching memory networks for traffic forecasting. In International Conference on Learning Representations, 2022.
  13. A brief overview of machine learning methods for short-term traffic forecasting and future directions. SIGSPATIAL Special, 10(1):3–9, 2018.
  14. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In International Conference on Learning Representations, 2018.
  15. Sgdr: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2017.
  16. Deciding how to decide: Dynamic routing in artificial neural networks. In Proceedings of the International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp.  2363–2372, 2017.
  17. ST-GRAT: A novel spatio-temporal graph attention networks for accurately forecasting dynamically changing road speed. In CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020, pp.  1215–1224. ACM, 2020.
  18. Scaling vision with sparse mixture of experts. In Advances in Neural Information Processing Systems, pp.  8583–8595, 2021.
  19. Routing networks: Adaptive selection of non-linear functions for multi-task learning. In International Conference on Learning Representations, 2018.
  20. At a glance: Pixel approximate entropy as a measure of line chart complexity. IEEE Transactions on Visualization and Computer Graphics, 25(01):872–881, 2019. ISSN 1941-0506. doi: 10.1109/TVCG.2018.2865264.
  21. Discrete graph structure learning for forecasting multiple time series. In International Conference on Learning Representations, 2021.
  22. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations, 2017.
  23. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017.
  24. Short-term traffic forecasting: Where we are and where we’re going. Transportation Research Part C: Emerging Technologies, 43:3–19, 2014. Special Issue on Short-term Traffic Flow Forecasting.
  25. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the International Joint Conference on Artificial Intelligence, pp.  1907–1913, 2019.
  26. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
  27. Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908, 2020.
  28. Coupled layer-wise graph convolution for transportation demand prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 35(5):4617–4625, 2021.
  29. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the International Joint Conference on Artificial Intelligence, pp.  3634–3640, 2018.
  30. Dnn-based prediction model for spatio-temporal data. In Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, SIGSPACIAL ’16, 2016.
  31. Spatio-temporal graph structure learning for traffic forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01):1177–1185, 2020.
  32. GMAN: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.  1234–1241, 2020.
  33. Mixture-of-experts with expert choice routing. In Advances in Neural Information Processing Systems, volume 35, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Hyunwook Lee (10 papers)
  2. Sungahn Ko (16 papers)
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now