Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction (2405.17898v1)

Published 28 May 2024 in cs.LG, cs.AI, and cs.CY

Abstract: The objective of traffic prediction is to accurately forecast and analyze the dynamics of transportation patterns, considering both space and time. However, the presence of distribution shift poses a significant challenge in this field, as existing models struggle to generalize well when faced with test data that significantly differs from the training distribution. To tackle this issue, this paper introduces a simple and universal spatio-temporal prompt-tuning framework-FlashST, which adapts pre-trained models to the specific characteristics of diverse downstream datasets, improving generalization in diverse traffic prediction scenarios. Specifically, the FlashST framework employs a lightweight spatio-temporal prompt network for in-context learning, capturing spatio-temporal invariant knowledge and facilitating effective adaptation to diverse scenarios. Additionally, we incorporate a distribution mapping mechanism to align the data distributions of pre-training and downstream data, facilitating effective knowledge transfer in spatio-temporal forecasting. Empirical evaluations demonstrate the effectiveness of our FlashST across different spatio-temporal prediction tasks using diverse urban datasets. Code is available at https://github.com/HKUDS/FlashST.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Adaptive graph convolutional recurrent network for traffic forecasting. In NeurIPS, pp.  17804–17815, 2020.
  2. Language models are few-shot learners. In NeurIPS, 2020.
  3. Debiased contrastive learning. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), NeurIPS, pp.  8765–8775, 2020.
  4. Towards spatio- temporal aware traffic time series forecasting. In ICDE, pp.  2900–2913, 2022.
  5. Spatial-temporal graph ode networks for traffic flow forecasting. In KDD, pp.  364–373, 2021.
  6. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In AAAI, pp.  922–929, 2019.
  7. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. In KDD, pp.  547–555, 2021.
  8. Pdformer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction. In AAAI, pp.  4365–4373, 2023.
  9. Spatial-temporal fusion graph neural networks for traffic flow forecasting. In AAAI, pp.  4189–4196, 2021.
  10. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In ICLR, 2018.
  11. Spatial-temporal hypergraph self-supervised learning for crime prediction. In ICDE, pp.  2984–2996, 2022.
  12. Gpt-st: Generative pre-training of spatio-temporal graph neural networks. In NeurIPS, 2023.
  13. Urbangpt: Spatio-temporal large language models. arXiv preprint arXiv:2403.00813, 2024.
  14. Msdr: Multi-step dependency relation networks for spatial temporal forecasting. In KDD, pp.  1042–1050, 2022.
  15. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, pp.  1–35, 2023a.
  16. Largest: A benchmark dataset for large-scale traffic forecasting. arXiv preprint arXiv:2306.08259, 2023b.
  17. Spatio-temporal graph few-shot learning with cross-city knowledge transfer. In KDD, pp.  1162–1172, 2022.
  18. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07676, 2020.
  19. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. In CIKM, pp.  4454–4458, 2022a.
  20. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In KDD, pp.  1567–1577, 2022b.
  21. Repository-level prompt generation for large language models of code. In ICML, pp.  31693–31715. PMLR, 2023.
  22. Visual prompt tuning for generative transfer learning. In CVPR, pp.  19840–19851, 2023.
  23. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In AAAI, pp.  914–921, 2020.
  24. Understanding the behaviour of contrastive loss. In CVPR, pp.  2495–2504, June 2021.
  25. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In ICML, pp.  9929–9939. PMLR, 2020.
  26. Traffic flow prediction via spatial temporal graph neural network. In WWW, pp.  1082–1092, 2020.
  27. Principal component analysis. Chemometrics and Intelligent Laboratory Systems, pp.  37–52, 1987.
  28. Graph information bottleneck. NIPS, 33:20437–20448, 2020a.
  29. Graph wavenet for deep spatial-temporal graph modeling. In IJCAI, 2019.
  30. Connecting the dots: Multivariate time series forecasting with graph neural networks. In KDD, pp.  753–763, 2020b.
  31. Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908, 2020.
  32. Deep multi-view spatial-temporal network for taxi demand prediction. In AAAI, pp.  2588–2595, 2018.
  33. Coupled layer-wise graph convolution for transportation demand prediction. In AAAI, pp.  4617–4625, 2021.
  34. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In IJCAI, 2018a.
  35. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In IJCAI, 2018b.
  36. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In SIGIR, pp.  1294–1303, 2022.
  37. Deep learning: A generic approach for extreme condition traffic forecasting. In SDM, pp.  777–785, 2017.
  38. Deep spatio-temporal residual networks for citywide crowd flows prediction. In AAAI, 2017.
  39. Spatial-temporal graph learning with adversarial contrastive adaptation. In ICML, pp.  41151–41163. PMLR, 2023a.
  40. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199, 2023b.
  41. Traffic flow forecasting with spatial-temporal graph diffusion network. In AAAI, pp.  15008–15015, 2021.
  42. T-gcn: A temporal graph convolutional network for traffic prediction. Transactions on Transportation Systems (TITS), pp.  3848–3858, 2020.
  43. Gman: A graph multi-attention network for traffic prediction. In AAAI, pp.  1234–1241, 2020.
  44. Conditional prompt learning for vision-language models. In CVPR, pp.  16816–16825, 2022a.
  45. Learning to prompt for vision-language models. International Journal of Computer Vision (IJCV), 130(9):2337–2348, 2022b.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Zhonghang Li (8 papers)
  2. Lianghao Xia (65 papers)
  3. Yong Xu (432 papers)
  4. Chao Huang (244 papers)
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets