Online Test-Time Adaptation of Spatial-Temporal Traffic Flow Forecasting (2401.04148v1)
Abstract: Accurate spatial-temporal traffic flow forecasting is crucial in aiding traffic managers in implementing control measures and assisting drivers in selecting optimal travel routes. Traditional deep-learning based methods for traffic flow forecasting typically rely on historical data to train their models, which are then used to make predictions on future data. However, the performance of the trained model usually degrades due to the temporal drift between the historical and future data. To make the model trained on historical data better adapt to future data in a fully online manner, this paper conducts the first study of the online test-time adaptation techniques for spatial-temporal traffic flow forecasting problems. To this end, we propose an Adaptive Double Correction by Series Decomposition (ADCSD) method, which first decomposes the output of the trained model into seasonal and trend-cyclical parts and then corrects them by two separate modules during the testing phase using the latest observed data entry by entry. In the proposed ADCSD method, instead of fine-tuning the whole trained model during the testing phase, a lite network is attached after the trained model, and only the lite network is fine-tuned in the testing process each time a data entry is observed. Moreover, to satisfy that different time series variables may have different levels of temporal drift, two adaptive vectors are adopted to provide different weights for different time series variables. Extensive experiments on four real-world traffic flow forecasting datasets demonstrate the effectiveness of the proposed ADCSD method. The code is available at https://github.com/Pengxin-Guo/ADCSD.
- X. Liu, X. Qin, M. Zhou, H. Sun, and S. Han, “Community-based dandelion algorithm-enabled feature selection and broad learning system for traffic flow prediction,” IEEE Trans. Intell. Transp. Syst., pp. 1–14, 2023.
- Y. Lv, Y. Duan, W. Kang, Z. Li, and F.-Y. Wang, “Traffic flow prediction with big data: A deep learning approach,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 2, pp. 865–873, 2014.
- L. Zhao, Y. Song, C. Zhang, Y. Liu, P. Wang, T. Lin, M. Deng, and H. Li, “T-gcn: A temporal graph convolutional network for traffic prediction,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 9, pp. 3848–3858, 2019.
- D. A. Tedjopurnomo, Z. Bao, B. Zheng, F. M. Choudhury, and A. K. Qin, “A survey on modern deep neural network for traffic prediction: Trends, methods and challenges,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 4, pp. 1544–1561, 2020.
- W. Jiang and J. Luo, “Graph neural network for traffic forecasting: A survey,” Expert Syst. Appl., vol. 207, p. 117921, 2022.
- A. M. Nagy and V. Simon, “Survey on traffic prediction in smart cities,” Pervasive Mob. Comput., vol. 50, pp. 148–163, 2018.
- S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan, “Attention based spatial-temporal graph convolutional networks for traffic flow forecasting,” in Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, 2019, pp. 922–929.
- L. Bai, L. Yao, C. Li, X. Wang, and C. Wang, “Adaptive graph convolutional recurrent network for traffic forecasting,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 33, pp. 17 804–17 815, 2020.
- G. Woo, C. Liu, D. Sahoo, A. Kumar, and S. Hoi, “Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2022.
- J. Jiang, C. Han, W. X. Zhao, and J. Wang, “Pdformer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction,” in Proc. AAAI Conf. Artif. Intell., 2023.
- V. Kuznetsov and M. Mohri, “Generalization bounds for time series prediction with non-stationary processes,” in Proc. Algorithmic Learning Theory (ALT), 2014, pp. 260–274.
- Y. Du, J. Wang, W. Feng, S. Pan, T. Qin, R. Xu, and C. Wang, “Adarnn: Adaptive learning and forecasting of time series,” in Proc. Conf. Inf. Know. Mana. (CIKM), 2021, pp. 402–411.
- W. Duan, X. He, L. Zhou, L. Thiele, and H. Rao, “Combating distribution shift for accurate time series forecasting via hypernetworks,” in Proc. Int. Conf. Para. Dist. Syst. (ICPADS). IEEE, 2023, pp. 900–907.
- J. Liang, R. He, and T. Tan, “A comprehensive survey on test-time adaptation under distribution shifts,” Preprint arXiv:2303.15361, 2023.
- Y. Sun, X. Wang, Z. Liu, J. Miller, A. Efros, and M. Hardt, “Test-time training with self-supervision for generalization under distribution shifts,” in Proc. Int. Conf. Mach. Learn. (ICML). PMLR, 2020, pp. 9229–9248.
- S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan, “Efficient test-time model adaptation without forgetting,” in Proc. Int. Conf. Mach. Learn. (ICML). PMLR, 2022, pp. 16 888–16 905.
- D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell, “Tent: Fully test-time adaptation by entropy minimization,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2021.
- Y. Gandelsman, Y. Sun, X. Chen, and A. Efros, “Test-time training with masked autoencoders,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 35, pp. 29 374–29 385, 2022.
- T. Gong, J. Jeong, T. Kim, Y. Kim, J. Shin, and S.-J. Lee, “Note: Robust continual test-time adaptation against temporal correlation,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 35, pp. 27 253–27 266, 2022.
- Q. Wang, O. Fink, L. Van Gool, and D. Dai, “Continual test-time domain adaptation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 7201–7211.
- C. Tian and W. K. Chan, “Spatial-temporal attention wavenet: A deep learning framework for traffic prediction considering spatial-temporal dependencies,” IET Intell. Transp. Syst., vol. 15, no. 4, pp. 549–561, 2021.
- K. Clark, K. Guu, M.-W. Chang, P. Pasupat, G. Hinton, and M. Norouzi, “Meta-learning fast weight language models,” Preprint arXiv:2212.02475, 2022.
- R. B. Cleveland, W. S. Cleveland, J. E. McRae, and I. Terpenning, “Stl: A seasonal-trend decomposition,” J. Off. Stat., vol. 6, no. 1, pp. 3–73, 1990.
- Z. Li, H. Yan, C. Zhang, and F. Tsung, “Long-short term spatiotemporal tensor prediction for passenger flow profile,” IEEE Robot. Autom. Lett., vol. 5, no. 4, pp. 5010–5017, 2020.
- F. Amato, F. Guignard, S. Robert, and M. Kanevski, “A novel framework for spatio-temporal prediction of environmental data using deep learning,” Sci Rep, vol. 10, no. 1, p. 22243, 2020.
- D. Liu, J. Wang, S. Shang, and P. Han, “Msdr: Multi-step dependency relation networks for spatial temporal forecasting,” in Proc. SIGKDD Conf. Know. Disc. & Data Mining, 2022, pp. 1042–1050.
- N. Jones, “How machine learning could help to improve climate forecasts,” Nature, vol. 548, no. 7668, 2017.
- A. Longo, M. Zappatore, M. Bochicchio, and S. B. Navathe, “Crowd-sourced data collection for urban monitoring via mobile sensors,” ACM Trans. Internet. Technol., vol. 18, no. 1, pp. 1–21, 2017.
- S. Guo, Y. Lin, H. Wan, X. Li, and G. Cong, “Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 11, pp. 5415–5428, 2021.
- X. You, M. Zhang, D. Ding, F. Feng, and Y. Huang, “Learning to learn the future: Modeling concept drifts in time series prediction,” in Proc. Conf. Inf. Know. Mana. (CIKM), 2021, pp. 2434–2443.
- S. O. Arik, N. C. Yoder, and T. Pfister, “Self-adaptive forecasting for improved deep learning on non-stationary time-series,” Preprint arXiv:2202.02403, 2022.
- T. Kim, J. Kim, Y. Tae, C. Park, J.-H. Choi, and J. Choo, “Reversible instance normalization for accurate time-series forecasting against distribution shift,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2022.
- R. Wang, Y. Dong, S. O. Arik, and R. Yu, “Koopman neural operator forecaster for time-series with temporal distributional shifts,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023.
- G. Bai, C. Ling, and L. Zhao, “Temporal domain generalization with drift-aware dynamic neural networks,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023.
- X. Hu, G. Uzunbas, S. Chen, R. Wang, A. Shah, R. Nevatia, and S.-N. Lim, “Mixnorm: Test-time adaptation through online normalization estimation,” Preprint arXiv:2110.11478, 2021.
- F. Azimi, S. Palacio, F. Raue, J. Hees, L. Bertinetto, and A. Dengel, “Self-supervised test-time adaptation on video data,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2022, pp. 3439–3448.
- J. Hong, L. Lyu, J. Zhou, and M. Spranger, “Mecta: Memory-economic continual test-time model adaptation,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023.
- M. Boudiaf, R. Mueller, I. Ben Ayed, and L. Bertinetto, “Parameter-free online test-time adaptation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 8344–8353.
- J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” Preprint arXiv:1607.06450, 2016.
- D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” Preprint arXiv:1606.08415, 2016.
- C. Song, Y. Lin, S. Guo, and H. Wan, “Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting,” in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 01, 2020, pp. 914–921.
- L. Liu, J. Zhen, G. Li, G. Zhan, Z. He, B. Du, and L. Lin, “Dynamic spatial-temporal representation learning for traffic flow prediction,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 11, pp. 7169–7183, 2020.
- Z. Pan, Y. Liang, W. Wang, Y. Yu, Y. Zheng, and J. Zhang, “Urban traffic prediction from spatio-temporal data using deep meta learning,” in Proc. SIGKDD Conf. Know. Disc. & Data Mining, 2019, pp. 1720–1730.
- C. Chen, K. Petty, A. Skabardonis, P. Varaiya, and Z. Jia, “Freeway performance measurement system: mining loop detector data,” Transp. Res. Record, vol. 1748, no. 1, pp. 96–102, 2001.
- J. Wang, J. Jiang, W. Jiang, C. Li, and W. X. Zhao, “Libcity: An open library for traffic prediction,” in Proc. ACM SIGSPATIAL Conf., 2021, pp. 145–148.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” Preprint arXiv:1412.6980, 2014.