Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook (2310.10196v2)

Published 16 Oct 2023 in cs.LG and cs.AI

Abstract: Temporal data, notably time series and spatio-temporal data, are prevalent in real-world applications. They capture dynamic system measurements and are produced in vast quantities by both physical and virtual sensors. Analyzing these data types is vital to harnessing the rich information they encompass and thus benefits a wide range of downstream tasks. Recent advances in large language and other foundational models have spurred increased use of these models in time series and spatio-temporal data mining. Such methodologies not only enable enhanced pattern recognition and reasoning across diverse domains but also lay the groundwork for artificial general intelligence capable of comprehending and processing common temporal data. In this survey, we offer a comprehensive and up-to-date review of large models tailored (or adapted) for time series and spatio-temporal data, spanning four key facets: data types, model categories, model scopes, and application areas/tasks. Our objective is to equip practitioners with the knowledge to develop applications and further research in this underexplored domain. We primarily categorize the existing literature into two major clusters: large models for time series analysis (LM4TS) and spatio-temporal data mining (LM4STD). On this basis, we further classify research based on model scopes (i.e., general vs. domain-specific) and application areas/tasks. We also provide a comprehensive collection of pertinent resources, including datasets, model assets, and useful tools, categorized by mainstream applications. This survey coalesces the latest strides in large model-centric research on time series and spatio-temporal data, underscoring the solid foundations, current advances, practical applications, abundant resources, and future research opportunities.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (335)
  1. W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al., “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023.
  2. M. Awais, M. Naseer, S. Khan, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, and F. S. Khan, “Foundational models defining a new era in vision: A survey and outlook,” arXiv preprint arXiv:2307.13721, 2023.
  3. S. Latif, M. Shoukat, F. Shamshad, M. Usama, H. Cuayáhuitl, and B. W. Schuller, “Sparks of large audio models: A survey and outlook,” arXiv preprint arXiv:2308.12792, 2023.
  4. K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl et al., “Large language models encode clinical knowledge,” arXiv preprint arXiv:2212.13138, 2022.
  5. S. Mirchandani, F. Xia, P. Florence, B. Ichter, D. Driess, M. G. Arenas, K. Rao, D. Sadigh, and A. Zeng, “Large language models as general pattern machines,” arXiv preprint arXiv:2307.04721, 2023.
  6. N. Kant, R. Puri, N. Yakovenko, and B. Catanzaro, “Practical text classification with large pre-trained language models,” arXiv preprint arXiv:1812.01207, 2018.
  7. D. Su, Y. Xu, G. I. Winata, P. Xu, H. Kim, Z. Liu, and P. Fung, “Generalizing question answering system with pre-trained language model fine-tuning,” in Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019, pp. 203–211.
  8. B. Zhang, B. Haddow, and A. Birch, “Prompting large language model for machine translation: A case study,” arXiv preprint arXiv:2301.07069, 2023.
  9. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler et al., “Emergent abilities of large language models,” arXiv preprint arXiv:2206.07682, 2022.
  10. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  11. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019.
  12. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scaling language modeling with pathways,” arXiv preprint arXiv:2204.02311, 2022.
  13. R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen et al., “Palm 2 technical report,” arXiv preprint arXiv:2305.10403, 2023.
  14. X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, and J. Tang, “Self-supervised learning: Generative or contrastive,” IEEE transactions on knowledge and data engineering, vol. 35, no. 1, pp. 857–876, 2021.
  15. K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” Journal of Big data, vol. 3, no. 1, pp. 1–40, 2016.
  16. M. Z. Hossain, F. Sohel, M. F. Shiratuddin, and H. Laga, “A comprehensive survey of deep learning for image captioning,” ACM Computing Surveys (CSUR), vol. 51, no. 6, pp. 1–36, 2019.
  17. K. Kafle and C. Kanan, “Visual question answering: Datasets, algorithms, and future challenges,” Computer Vision and Image Understanding, vol. 163, pp. 3–20, 2017.
  18. R. Zellers, Y. Bisk, A. Farhadi, and Y. Choi, “From recognition to cognition: Visual commonsense reasoning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 6720–6731.
  19. A. Karpatne, I. Ebert-Uphoff, S. Ravela, H. A. Babaie, and V. Kumar, “Machine learning for the geosciences: Challenges and opportunities,” IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 8, pp. 1544–1554, 2018.
  20. G. Jin, Y. Liang, Y. Fang, J. Huang, J. Zhang, and Y. Zheng, “Spatio-temporal graph neural networks for predictive learning in urban computing: A survey,” arXiv preprint arXiv:2303.14483, 2023.
  21. Z. Zhu, W. Chen, R. Xia, T. Zhou, P. Niu, B. Peng, W. Wang, H. Liu, Z. Ma, Q. Wen et al., “eforecaster: unifying electricity forecasting with robust, flexible, and explainable machine learning algorithms,” in AAAI Conference on Artificial Intelligence, 2023.
  22. H. Harutyunyan, H. Khachatrian, D. C. Kale, G. Ver Steeg, and A. Galstyan, “Multitask learning and benchmarking with clinical time series data,” Scientific data, vol. 6, no. 1, p. 96, 2019.
  23. Y. Xia, Y. Liang, H. Wen, X. Liu, K. Wang, Z. Zhou, and R. Zimmermann, “Deciphering spatio-temporal graph forecasting: A causal lens and treatment,” arXiv preprint arXiv:2309.13378, 2023.
  24. I. Lazaridis and S. Mehrotra, “Capturing sensor-generated time series with quality guarantees,” in Proceedings 19th International Conference on Data Engineering (Cat. No. 03CH37405).   IEEE, 2003, pp. 429–440.
  25. S. Yao, S. Hu, Y. Zhao, A. Zhang, and T. Abdelzaher, “Deepsense: A unified deep learning framework for time-series mobile sensing data processing,” in Proceedings of the 26th international conference on world wide web, 2017, pp. 351–360.
  26. O. B. Sezer, M. U. Gudelek, and A. M. Ozbayoglu, “Financial time series forecasting with deep learning: A systematic literature review: 2005–2019,” Applied soft computing, vol. 90, p. 106181, 2020.
  27. Z. Zhou, C. Zhang, L. Ma, J. Gu, H. Qian, Q. Wen, L. Sun, P. Li, and Z. Tang, “AHPA: Adaptive horizontal pod autoscaling systems on alibaba cloud container service for kubernetes,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  28. G. U. Yule, “Why do we sometimes get nonsense-correlations between time-series?–a study in sampling and the nature of time-series,” Journal of the royal statistical society, vol. 89, no. 1, pp. 1–63, 1926.
  29. H. Hewamalage, C. Bergmeir, and K. Bandara, “Recurrent neural networks for time series forecasting: Current status and future directions,” International Journal of Forecasting, vol. 37, no. 1, pp. 388–427, 2021.
  30. C. Lea, R. Vidal, A. Reiter, and G. D. Hager, “Temporal convolutional networks: A unified approach to action segmentation,” in Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14.   Springer, 2016, pp. 47–54.
  31. M. Jin, H. Y. Koh, Q. Wen, D. Zambon, C. Alippi, G. I. Webb, I. King, and S. Pan, “A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection,” arXiv preprint arXiv:2307.03759, 2023.
  32. J. Selva, A. S. Johansen, S. Escalera, K. Nasrollahi, T. B. Moeslund, and A. Clapés, “Video transformers: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  33. Q. Wen, T. Zhou, C. Zhang, W. Chen, Z. Ma, J. Yan, and L. Sun, “Transformers in time series: A survey,” in International Joint Conference on Artificial Intelligence(IJCAI), 2023, pp. 6778–6786.
  34. K. Zhang, Q. Wen, C. Zhang, R. Cai, M. Jin, Y. Liu, J. Zhang, Y. Liang, G. Pang, D. Song et al., “Self-supervised learning for time series analysis: Taxonomy, progress, and prospects,” arXiv preprint arXiv:2306.10125, 2023.
  35. M. C. Schiappa, Y. S. Rawat, and M. Shah, “Self-supervised learning for videos: A survey,” ACM Computing Surveys, vol. 55, no. 13s, pp. 1–37, 2023.
  36. X. Zhang, Z. Zhao, T. Tsiligkaridis, and M. Zitnik, “Self-supervised contrastive pre-training for time series via time-frequency consistency,” Advances in Neural Information Processing Systems, vol. 35, pp. 3988–4003, 2022.
  37. J. Dong, H. Wu, H. Zhang, L. Zhang, J. Wang, and M. Long, “Simmtm: A simple pre-training framework for masked time-series modeling,” Advances in Neural Information Processing Systems, 2023.
  38. Z. Shao, Z. Zhang, F. Wang, and Y. Xu, “Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 1567–1577.
  39. W. G. C. Bandara, N. Patel, A. Gholami, M. Nikkhah, M. Agrawal, and V. M. Patel, “Adamae: Adaptive masking for efficient spatiotemporal learning with masked autoencoders,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14 507–14 517.
  40. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021.
  41. S. Deldari, H. Xue, A. Saeed, J. He, D. V. Smith, and F. D. Salim, “Beyond just vision: A review on self-supervised representation learning on multimodal and temporal data,” arXiv preprint arXiv:2206.02353, 2022.
  42. C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He et al., “A comprehensive survey on pretrained foundation models: A history from bert to chatgpt,” arXiv preprint arXiv:2302.09419, 2023.
  43. S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, and E. Chen, “A survey on multimodal large language models,” arXiv preprint arXiv:2306.13549, 2023.
  44. C. Wu, X. Zhang, Y. Zhang, Y. Wang, and W. Xie, “Towards generalist foundation model for radiology,” arXiv preprint arXiv:2308.02463, 2023.
  45. Q. Ma, Z. Liu, Z. Zheng, Z. Huang, S. Zhu, Z. Yu, and J. T. Kwok, “A survey on time-series pre-trained models,” arXiv preprint arXiv:2305.10716, 2023.
  46. G. Mai, W. Huang, J. Sun, S. Song, D. Mishra, N. Liu, S. Gao, T. Liu, G. Cong, Y. Hu et al., “On the opportunities and challenges of foundation models for geospatial artificial intelligence,” arXiv preprint arXiv:2304.06798, 2023.
  47. T. Zhou, P. Niu, X. Wang, L. Sun, and R. Jin, “One fits all: Power general time series analysis by pretrained lm,” Advances in Neural Information Processing Systems, 2023.
  48. M. Jin, S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P.-Y. Chen, Y. Liang, Y.-F. Li, S. Pan, and Q. Wen, “Time-LLM: Time series forecasting by reprogramming large language models,” arXiv preprint arXiv:2310.01728, 2023.
  49. K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Accurate medium-range global weather forecasting with 3d neural networks,” Nature, vol. 619, no. 7970, pp. 533–538, 2023.
  50. T. Nguyen, J. Brandstetter, A. Kapoor, J. K. Gupta, and A. Grover, “Climax: A foundation model for weather and climate,” International Conference on Machine Learning, 2023.
  51. X. Wang, D. Wang, L. Chen, and Y. Lin, “Building transportation foundation model via generative graph transformer,” arXiv preprint arXiv:2305.14826, 2023.
  52. R. Luo, Z. Zhao, M. Yang, J. Dong, M. Qiu, P. Lu, T. Wang, and Z. Wei, “Valley: Video assistant with large language model enhanced ability,” arXiv preprint arXiv:2306.07207, 2023.
  53. Y. Zhao, I. Misra, P. Krähenbühl, and R. Girdhar, “Learning video representations from large language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6586–6597.
  54. H. Xu, Q. Ye, M. Yan, Y. Shi, J. Ye, Y. Xu, C. Li, B. Bi, Q. Qian, W. Wang et al., “mplug-2: A modularized multi-modal foundation model across text, image and video,” in International Conference on Machine Learning, 2023.
  55. C.-H. H. Yang, Y.-Y. Tsai, and P.-Y. Chen, “Voice2series: Reprogramming acoustic models for time series classification,” in International conference on machine learning.   PMLR, 2021, pp. 11 808–11 819.
  56. H. Xue and F. D. Salim, “Promptcast: A new prompt-based learning paradigm for time series forecasting,” arXiv preprint arXiv:2210.08964, 2022.
  57. L. Y. Jiang, X. C. Liu, N. P. Nejatian, M. Nasir-Moin, D. Wang, A. Abidin, K. Eaton, H. A. Riina, I. Laufer, P. Punjabi et al., “Health system-scale language models are all-purpose prediction engines,” Nature, pp. 1–6, 2023.
  58. X. Yang, A. Chen, N. PourNejatian, H. C. Shin, K. E. Smith, C. Parisien, C. Compas, C. Martin, A. B. Costa, M. G. Flores et al., “A large language model for electronic health records,” NPJ Digital Medicine, vol. 5, no. 1, p. 194, 2022.
  59. Y. Bengio, R. Ducharme, and P. Vincent, “A neural probabilistic language model,” Advances in neural information processing systems, vol. 13, 2000.
  60. T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur, “Recurrent neural network based language model,” in Interspeech, vol. 2, no. 3.   Makuhari, 2010, pp. 1045–1048.
  61. E. Arisoy, T. N. Sainath, B. Kingsbury, and B. Ramabhadran, “Deep neural network language models,” in Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT, 2012, pp. 20–28.
  62. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  63. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” arXiv preprint arXiv:1910.13461, 2019.
  64. Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui, “A survey for in-context learning,” arXiv preprint arXiv:2301.00234, 2022.
  65. A. Madani, B. Krause, E. R. Greene, S. Subramanian, B. P. Mohr, J. M. Holton, J. L. Olmos Jr, C. Xiong, Z. Z. Sun, R. Socher et al., “Large language models generate functional protein sequences across diverse families,” Nature Biotechnology, pp. 1–8, 2023.
  66. S. Menon and C. Vondrick, “Visual classification via description from large language models,” arXiv preprint arXiv:2210.07183, 2022.
  67. A. Lopez-Lira and Y. Tang, “Can chatgpt forecast stock price movements? return predictability and large language models,” arXiv preprint arXiv:2304.07619, 2023.
  68. Z. Chen, H. Mao, H. Li, W. Jin, H. Wen, X. Wei, S. Wang, D. Yin, W. Fan, H. Liu et al., “Exploring the potential of large language models (llms) in learning on graphs,” arXiv preprint arXiv:2307.03393, 2023.
  69. N. Muennighoff, T. Wang, L. Sutawika, A. Roberts, S. Biderman, T. L. Scao, M. S. Bari, S. Shen, Z.-X. Yong, H. Schoelkopf et al., “Crosslingual generalization through multitask finetuning,” arXiv preprint arXiv:2211.01786, 2022.
  70. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023.
  71. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023.
  72. R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Alpaca: A strong, replicable instruction-following model,” Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html, vol. 3, no. 6, p. 7, 2023.
  73. W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez et al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” See https://vicuna. lmsys. org (accessed 14 April 2023), 2023.
  74. E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, E. Goffinet, D. Heslow, J. Launay, Q. Malartic et al., “Falcon-40b: an open large language model with state-of-the-art performance,” 2023.
  75. OpenAI, “Gpt-4 technical report,” ArXiv, vol. abs/2303.08774, 2023.
  76. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning, 2019, pp. 2790–2799.
  77. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021.
  78. I. Melnyk, V. Chenthamarakshan, P.-Y. Chen, P. Das, A. Dhurandhar, I. Padhi, and D. Das, “Reprogramming large pretrained language models for antibody sequence infilling,” arXiv preprint arXiv:2210.07144, 2022.
  79. P.-Y. Chen, “Model reprogramming: Resource-efficient cross-domain machine learning,” arXiv preprint arXiv:2202.10629, 2022.
  80. T. Sun, Y. Shao, H. Qian, X. Huang, and X. Qiu, “Black-box tuning for language-model-as-a-service,” in International Conference on Machine Learning.   PMLR, 2022, pp. 20 841–20 855.
  81. Y. Li, C. Gao, X. Song, X. Wang, Y. Xu, and S. Han, “Druggpt: A gpt-based strategy for designing potential ligands targeting specific proteins,” bioRxiv, pp. 2023–06, 2023.
  82. M. Tsimpoukelli, J. L. Menick, S. Cabi, S. Eslami, O. Vinyals, and F. Hill, “Multimodal few-shot learning with frozen language models,” Advances in Neural Information Processing Systems, vol. 34, pp. 200–212, 2021.
  83. J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” arXiv preprint arXiv:2301.12597, 2023.
  84. H. Xue and F. D. Salim, “Prompt-based time series forecasting: A new task and dataset,” arXiv preprint arXiv:2210.08964, 2022.
  85. A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo et al., “Segment anything,” arXiv preprint arXiv:2304.02643, 2023.
  86. S. Wu, H. Fei, L. Qu, W. Ji, and T.-S. Chua, “Next-gpt: Any-to-any multimodal llm,” arXiv preprint arXiv:2309.05519, 2023.
  87. K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl et al., “Large language models encode clinical knowledge,” Nature, pp. 1–9, 2023.
  88. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in Neural Information Processing Systems, vol. 35, pp. 24 824–24 837, 2022.
  89. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” arXiv preprint arXiv:2305.10601, 2023.
  90. M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, L. Gianinazzi, J. Gajda, T. Lehmann, M. Podstawski, H. Niewiadomski, P. Nyczyk et al., “Graph of thoughts: Solving elaborate problems with large language models,” arXiv preprint arXiv:2308.09687, 2023.
  91. Z. Wu, Z. Wang, X. Xu, J. Lu, and H. Yan, “Embodied task planning with large language models,” arXiv preprint arXiv:2307.01848, 2023.
  92. Z. Wang, S. Cai, A. Liu, X. Ma, and Y. Liang, “Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents,” arXiv preprint arXiv:2302.01560, 2023.
  93. Q. Wen, L. Yang, T. Zhou, and L. Sun, “Robust time series analysis and applications: An industrial perspective,” in 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 4836–4837.
  94. S. Ji, S. Pan, E. Cambria, P. Marttinen, and S. Y. Philip, “A survey on knowledge graphs: Representation, acquisition, and applications,” IEEE transactions on neural networks and learning systems, vol. 33, no. 2, pp. 494–514, 2021.
  95. S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, “Transformers in vision: A survey,” ACM computing surveys (CSUR), vol. 54, no. 10s, pp. 1–41, 2022.
  96. Z. Liu, Q. Li, X. Chen, C. Wu, S. Ishihara, J. Li, and Y. Ji, “Point cloud video streaming: Challenges and solutions,” IEEE Network, vol. 35, no. 5, pp. 202–209, 2021.
  97. G. Atluri, A. Karpatne, and V. Kumar, “Spatio-temporal data mining: A survey of problems and methods,” ACM Computing Surveys (CSUR), vol. 51, no. 4, pp. 1–41, 2018.
  98. A. Divakaran and A. Mohan, “Temporal link prediction: A survey,” New Generation Computing, vol. 38, pp. 213–258, 2020.
  99. Z. Jia, Y. Lin, J. Wang, R. Zhou, X. Ning, Y. He, and Y. Zhao, “Graphsleepnet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification.” in IJCAI, vol. 2021, 2020, pp. 1324–1330.
  100. Z. A. Sahili and M. Awad, “Spatio-temporal graph neural networks: A survey,” arXiv preprint arXiv:2301.10569, 2023.
  101. B. Cai, Y. Xiang, L. Gao, H. Zhang, Y. Li, and J. Li, “Temporal knowledge graph completion: A survey,” arXiv preprint arXiv:2201.08236, 2022.
  102. Z. Han, Z. Ding, Y. Ma, Y. Gu, and V. Tresp, “Temporal knowledge graph forecasting with neural ode,” arXiv preprint arXiv:2101.05151, 2021.
  103. D. Cao, F. Jia, S. O. Arik, T. Pfister, Y. Zheng, W. Ye, and Y. Liu, “TEMPO: Prompt-based generative pre-trained transformer for time series forecasting,” arXiv preprint arXiv:2310.04948, 2023.
  104. C. Sun, Y. Li, H. Li, and S. Hong, “Test: Text prototype aligned embedding to activate llm’s ability for time series,” arXiv preprint arXiv:2308.08241, 2023.
  105. C. Chang, W.-C. Peng, and T.-F. Chen, “Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms,” arXiv preprint arXiv:2308.08469, 2023.
  106. N. Gruver, M. Finzi, S. Qiu, and A. G. Wilson, “Large language models are zero-shot time series forecasters,” Advances in Neural Information Processing Systems, 2023.
  107. H. Xue, B. P. Voutharoja, and F. D. Salim, “Leveraging language foundation models for human mobility forecasting,” in Proceedings of the 30th International Conference on Advances in Geographic Information Systems, 2022, pp. 1–9.
  108. X. Yu, Z. Chen, Y. Ling, S. Dong, Z. Liu, and Y. Lu, “Temporal data meets llm–explainable financial time series forecasting,” arXiv preprint arXiv:2306.11025, 2023.
  109. Q. Xie, W. Han, Y. Lai, M. Peng, and J. Huang, “The wall street neophyte: A zero-shot analysis of chatgpt over multimodal stock movement prediction challenges,” arXiv preprint arXiv:2304.05351, 2023.
  110. B. Zhang, H. Yang, and X.-Y. Liu, “Instruct-fingpt: Financial sentiment analysis by instruction tuning of general-purpose large language models,” FinLLM Symposium at IJCAI 2023, 2023.
  111. X. Shi, S. Xue, K. Wang, F. Zhou, J. Y. Zhang, J. Zhou, C. Tan, and H. Mei, “Language models can improve event prediction by few-shot abductive reasoning,” in Advances in Neural Information Processing Systems, 2023.
  112. X. Liu, D. McDuff, G. Kovacs, I. Galatzer-Levy, J. Sunshine, J. Zhan, M.-Z. Poh, S. Liao, P. Di Achille, and S. Patel, “Large language models are few-shot health learners,” arXiv preprint arXiv:2305.15525, 2023.
  113. J. Li, C. Liu, S. Cheng, R. Arcucci, and S. Hong, “Frozen language model helps ecg zero-shot learning,” in Medical Imaging with Deep Learning, 2023.
  114. P. Tang and X. Zhang, “Mtsmae: Masked autoencoders for multivariate time-series forecasting,” in 2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI).   IEEE, 2022, pp. 982–989.
  115. Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, “A time series is worth 64 words: Long-term forecasting with transformers,” arXiv preprint arXiv:2211.14730, 2022.
  116. V. Ekambaram, A. Jati, N. Nguyen, P. Sinthong, and J. Kalagnanam, “Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting,” arXiv preprint arXiv:2306.09364, 2023.
  117. Z. Yue, Y. Wang, J. Duan, T. Yang, C. Huang, Y. Tong, and B. Xu, “Ts2vec: Towards universal representation of time series,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 8, 2022, pp. 8980–8987.
  118. Y. Ozyurt, S. Feuerriegel, and C. Zhang, “Contrastive learning for unsupervised domain adaptation of time series,” arXiv preprint arXiv:2206.06243, 2022.
  119. S. Xue, Y. Wang, Z. Chu, X. Shi, C. Jiang, H. Hao, G. Jiang, X. Feng, J. Zhang, and J. Zhou, “Prompt-augmented temporal point process for streaming event sequence,” in Advances in Neural Information Processing Systems, 2023.
  120. H. Xu, Y. Gao, Z. Hui, J. Li, and X. Gao, “Language knowledge-assisted representation learning for skeleton-based action recognition,” arXiv preprint arXiv:2305.12398, 2023.
  121. Z. Chen, L. N. Zheng, C. Lu, J. Yuan, and D. Zhu, “Chatgpt informed graph neural network for stock movement prediction,” arXiv preprint arXiv:2306.03763, 2023.
  122. W. Xu, B. Liu, M. Peng, X. Jia, and M. Peng, “Pre-trained language model with prompts for temporal knowledge graph completion,” in Annual Meeting of the Association for Computational Linguistics, 2023.
  123. J. Chen, L. Ma, X. Li, N. Thakurdesai, J. Xu, J. H. Cho, K. Nag, E. Korpeoglu, S. Kumar, and K. Achan, “Knowledge graph completion models are few-shot learners: An empirical study of relation labeling in e-commerce with llms,” arXiv preprint arXiv:2305.09858, 2023.
  124. D.-H. Lee, K. Ahrabian, W. Jin, F. Morstatter, and J. Pujara, “Temporal knowledge graph forecasting without knowledge using in-context learning,” arXiv preprint arXiv:2305.10613, 2023.
  125. A. Yang, A. Nagrani, P. H. Seo, A. Miech, J. Pont-Tuset, I. Laptev, J. Sivic, and C. Schmid, “Vid2seq: Large-scale pretraining of a visual language model for dense video captioning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10 714–10 726.
  126. A. Yang, A. Miech, J. Sivic, I. Laptev, and C. Schmid, “Zero-shot video question answering via frozen bidirectional language models,” Advances in Neural Information Processing Systems, vol. 35, pp. 124–141, 2022.
  127. J. Pan, Z. Lin, Y. Ge, X. Zhu, R. Zhang, Y. Wang, Y. Qiao, and H. Li, “Retrieving-to-answer: Zero-shot video question answering with frozen large language models,” arXiv preprint arXiv:2306.11732, 2023.
  128. K. Li, Y. He, Y. Wang, Y. Li, W. Wang, P. Luo, Y. Wang, L. Wang, and Y. Qiao, “Videochat: Chat-centric video understanding,” arXiv preprint arXiv:2305.06355, 2023.
  129. E. Song, W. Chai, G. Wang, Y. Zhang, H. Zhou, F. Wu, X. Guo, T. Ye, Y. Lu, J.-N. Hwang et al., “Moviechat: From dense token to sparse memory for long video understanding,” arXiv preprint arXiv:2307.16449, 2023.
  130. H.-T. Su, Y. Niu, X. Lin, W. H. Hsu, and S.-F. Chang, “Language models are causal knowledge extractors for zero-shot video question answering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 4950–4959.
  131. Z. Wang, M. Li, R. Xu, L. Zhou, J. Lei, X. Lin, S. Wang, Z. Yang, C. Zhu, D. Hoiem et al., “Language models with image descriptors are strong few-shot video-language learners,” Advances in Neural Information Processing Systems, vol. 35, pp. 8483–8497, 2022.
  132. G. Chen, Y.-D. Zheng, J. Wang, J. Xu, Y. Huang, J. Pan, Y. Wang, Y. Wang, Y. Qiao, T. Lu et al., “Videollm: Modeling video sequence with large language models,” arXiv preprint arXiv:2305.13292, 2023.
  133. H. Zhang, X. Li, and L. Bing, “Video-llama: An instruction-tuned audio-visual language model for video understanding,” arXiv preprint arXiv:2306.02858, 2023.
  134. L. Li, Z. Gan, K. Lin, C.-C. Lin, Z. Liu, C. Liu, and L. Wang, “Lavender: Unifying video-language understanding as masked language modeling,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23 119–23 129.
  135. J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli et al., “Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators,” arXiv preprint arXiv:2202.11214, 2022.
  136. S. Chen, G. Long, T. Shen, and J. Jiang, “Prompt federated learning for weather forecasting: Toward foundation models on meteorological data,” in International Joint Conference on Artificial Intelligence, 2023.
  137. S. Chen, G. Long, T. Shen, T. Zhou, and J. Jiang, “Spatial-temporal prompt learning for federated weather forecasting,” arXiv preprint arXiv:2305.14244, 2023.
  138. K. Chen, T. Han, J. Gong, L. Bai, F. Ling, J.-J. Luo, X. Chen, L. Ma, T. Zhang, R. Su et al., “Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead,” arXiv preprint arXiv:2304.02948, 2023.
  139. K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Accurate medium-range global weather forecasting with 3d neural networks,” Nature, pp. 1–6, 2023.
  140. X. Man, C. Zhang, C. Li, and J. Shao, “W-mae: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting,” arXiv preprint arXiv:2304.08754, 2023.
  141. W. Duan, L. Jiang, N. Wang, and H. Rao, “Pre-trained bidirectional temporal representation for crowd flows prediction in regular region,” IEEE Access, vol. 7, pp. 143 855–143 865, 2019.
  142. K. Jin, J. Wi, E. Lee, S. Kang, S. Kim, and Y. Kim, “Trafficbert: Pre-trained model with large-scale data for long-range traffic flow forecasting,” Expert Systems with Applications, vol. 186, p. 115738, 2021.
  143. J. Wang, D. Chen, Z. Wu, C. Luo, L. Zhou, Y. Zhao, Y. Xie, C. Liu, Y.-G. Jiang, and L. Yuan, “Omnivl: One foundation model for image-language and video-language tasks,” Advances in neural information processing systems, vol. 35, pp. 5696–5710, 2022.
  144. H. Xu, Q. Ye, X. Wu, M. Yan, Y. Miao, J. Ye, G. Xu, A. Hu, Y. Shi, G. Xu et al., “Youku-mplug: A 10 million large-scale chinese video-language dataset for pre-training and benchmarks,” arXiv preprint arXiv:2306.04362, 2023.
  145. Z. Wang, A. Blume, S. Li, G. Liu, J. Cho, Z. Tang, M. Bansal, and H. Ji, “Paxion: Patching action knowledge in video-language foundation models,” arXiv preprint arXiv:2305.10683, 2023.
  146. A. Gunjal and G. Durrett, “Drafting event schemas using language models,” arXiv preprint arXiv:2305.14847, 2023.
  147. X. Wang, M. Fang, Z. Zeng, and T. Cheng, “Where would i go next? large language models as human mobility predictors,” 2023.
  148. X. Liu, Y. Liang, C. Huang, Y. Zheng, B. Hooi, and R. Zimmermann, “When do contrastive learning signals help spatio-temporal graph forecasting?” in Proceedings of the 30th International Conference on Advances in Geographic Information Systems, 2022, pp. 1–12.
  149. R. Li, T. Zhong, X. Jiang, G. Trajcevski, J. Wu, and F. Zhou, “Mining spatio-temporal relations via self-paced graph contrastive learning,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 936–944.
  150. M. Maaz, H. Rasheed, S. Khan, and F. S. Khan, “Video-chatgpt: Towards detailed video understanding via large vision and language models,” arXiv preprint arXiv:2306.05424, 2023.
  151. E. Qasemi, J. M. Francis, and A. Oltramari, “Traffic-domain video question answering with automatic captioning,” arXiv preprint arXiv:2307.09636, 2023.
  152. X. Ding, Y. Zhang, T. Liu, and J. Duan, “Deep learning for event-driven stock prediction,” in Proceedings of the 24th International Conference on Artificial Intelligence, ser. IJCAI’15.   AAAI Press, 2015, p. 2327–2333.
  153. S. Xue, X. Shi, Y. J. Zhang, and H. Mei, “Hypro: A hybridly normalized probabilistic model for long-horizon prediction of event sequences,” in Advances in Neural Information Processing Systems, 2022.
  154. C. Qu, X. Tan, S. Xue, X. Shi, J. Zhang, and H. Mei, “Bellman meets hawkes: Model-based reinforcement learning via temporal point processes,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  155. H. Mei and J. Eisner, “The neural hawkes process: A neurally self-modulating multivariate point process,” in Proceedings of the International Conference on Machine Learning (ICML), 2017.
  156. H. Mei, G. Qin, M. Xu, and J. Eisner, “Neural datalog through time: Informed temporal modeling via logical specification,” in Advances in Neural Information Processing Systems, 2020.
  157. C. Yang, H. Mei, and J. Eisner, “Transformer embeddings of irregularly spaced events and their participants,” in 10th International Conference on Learning Representations, ICLR, 2022.
  158. Y. Wang, Z. Chu, T. Zhou, C. Jiang, H. Hao, M. Zhu, X. Cai, Q. Cui, L. Li, J. Zhang, S. Xue, and J. Zhou, “Enhancing event sequence modeling with contrastive relational inference,” Advances in Neural Information Processing Systems, vol. 35, pp. 23 751–23 764, 2022.
  159. Z. Pan, W. Zhang, Y. Liang, W. Zhang, Y. Yu, J. Zhang, and Y. Zheng, “Spatio-temporal meta learning for urban traffic prediction,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 3, pp. 1462–1476, 2020.
  160. Z. Pan, S. Ke, X. Yang, Y. Liang, Y. Yu, J. Zhang, and Y. Zheng, “Autostg: Neural architecture search for predictions of spatio-temporal graph✱,” in Proceedings of the Web Conference 2021, 2021, pp. 1846–1855.
  161. H. Wen, Y. Lin, Y. Xia, H. Wan, R. Zimmermann, and Y. Liang, “Diffstg: Probabilistic spatio-temporal graph forecasting with denoising diffusion models,” arXiv preprint arXiv:2301.13629, 2023.
  162. Y. Liang, S. Ke, J. Zhang, X. Yi, and Y. Zheng, “Geoman: Multi-level attention networks for geo-sensory time series prediction.” in IJCAI, vol. 2018, 2018, pp. 3428–3434.
  163. S. Wang, Y. Li, J. Zhang, Q. Meng, L. Meng, and F. Gao, “Pm2. 5-gnn: A domain knowledge enhanced graph neural network for pm2. 5 forecasting,” in Proceedings of the 28th international conference on advances in geographic information systems, 2020, pp. 163–166.
  164. Y. Liang, Y. Xia, S. Ke, Y. Wang, Q. Wen, J. Zhang, Y. Zheng, and R. Zimmermann, “Airformer: Predicting nationwide air quality in china with transformers,” arXiv preprint arXiv:2211.15979, 2022.
  165. Y. Qin, D. Song, H. Chen, W. Cheng, G. Jiang, and G. Cottrell, “A dual-stage attention-based recurrent neural network for time series prediction,” arXiv preprint arXiv:1704.02971, 2017.
  166. D. Matsunaga, T. Suzumura, and T. Takahashi, “Exploring graph neural networks for stock market predictions with rolling window analysis,” arXiv preprint arXiv:1909.10660, 2019.
  167. L. Shi, Y. Zhang, J. Cheng, and H. Lu, “Skeleton-based action recognition with directed graph neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7912–7921.
  168. K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9729–9738.
  169. J. A. Weyn, D. R. Durran, and R. Caruana, “Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 9, p. e2020MS002109, 2020.
  170. R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, A. Pritzel, S. Ravuri, T. Ewalds, F. Alet, Z. Eaton-Rosen et al., “Graphcast: Learning skillful medium-range global weather forecasting,” arXiv preprint arXiv:2212.12794, 2022.
  171. S. Rasp and N. Thuerey, “Data-driven medium-range weather prediction with a resnet pretrained on climate simulations: A new model for WeatherBench,” Journal of Advances in Modeling Earth Systems, vol. 13, no. 2, feb 2021.
  172. K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000–16 009.
  173. F. M. Suchanek, G. Kasneci, and G. Weikum, “Yago: A core of semantic knowledge,” in Proceedings of the 16th International Conference on World Wide Web, ser. WWW ’07.   Association for Computing Machinery, 2007, p. 697–706.
  174. D. Vrandečić and M. Krötzsch, “Wikidata: A free collaborative knowledgebase,” Commun. ACM, vol. 57, no. 10, p. 78–85, sep 2014.
  175. R. Trivedi, H. Dai, Y. Wang, and L. Song, “Know-evolve: Deep temporal reasoning for dynamic knowledge graphs,” in Proceedings of the 34th International Conference on Machine Learning, 2017.
  176. R. Trivedi, M. Farajtabar, P. Biswal, and H. Zha, “Dyrep: Learning representations over dynamic graphs,” in 7th International Conference on Learning Representations, ICLR, 2019.
  177. Leetaru and Schrodt, “Gdelt: Global data on events, location, and tone,1979–2012,” ISA annual convention, vol. 2, pp. 1–49, 04 2013.
  178. S. Vishwakarma and A. Agrawal, “A survey on activity recognition and behavior understanding in video surveillance,” The Visual Computer, vol. 29, pp. 983–1009, 2013.
  179. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, “Large-scale video classification with convolutional neural networks,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, pp. 1725–1732.
  180. L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool, “Temporal segment networks for action recognition in videos,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 11, pp. 2740–2755, 2018.
  181. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497.
  182. H. Xu, A. Das, and K. Saenko, “R-c3d: Region convolutional 3d network for temporal activity detection,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5783–5792.
  183. K. Hara, H. Kataoka, and Y. Satoh, “Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 6546–6555.
  184. ——, “Learning spatio-temporal features with 3d residual networks for action recognition,” in Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 3154–3160.
  185. D. Neimark, O. Bar, M. Zohar, and D. Asselmann, “Video transformer network,” arXiv preprint arXiv:2102.00719, 2021.
  186. H. Fan, B. Xiong, K. Mangalam, Y. Li, Z. Yan, J. Malik, and C. Feichtenhofer, “Multiscale vision transformers,” arXiv preprint arXiv:2104.11227, 2021.
  187. A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lučić, and C. Schmid, “Vivit: A video vision transformer,” arXiv preprint arXiv:2103.15691, 2021.
  188. Z. Liu, J. Ning, Y. Cao, Y. Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” arXiv preprint arXiv:2106.13230, 2021.
  189. H. Xu, G. Ghosh, P.-Y. Huang, D. Okhonko, A. Aghajanyan, F. Metze, L. Zettlemoyer, and C. Feichtenhofer, “Videoclip: Contrastive pre-training for zero-shot video-text understanding,” arXiv preprint arXiv:2109.14084, 2021.
  190. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022.
  191. Y. Wang, K. Li, Y. Li, Y. He, B. Huang, Z. Zhao, H. Zhang, J. Xu, Y. Liu, Z. Wang et al., “Internvideo: General video foundation models via generative and discriminative learning,” arXiv preprint arXiv:2212.03191, 2022.
  192. Y. Li, F. Liang, L. Zhao, Y. Cui, W. Ouyang, J. Shao, F. Yu, and J. Yan, “Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm,” International Conference on Learning Representations, 2022.
  193. Q. Ye, G. Xu, M. Yan, H. Xu, Q. Qian, J. Zhang, and F. Huang, “Hitea: Hierarchical temporal-aware video-language pre-training,” arXiv preprint arXiv:2212.14546, 2022.
  194. J. Wang, Y. Ge, R. Yan, Y. Ge, K. Q. Lin, S. Tsutsui, X. Lin, G. Cai, J. Wu, Y. Shan et al., “All in one: Exploring unified video-language pre-training,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6598–6608.
  195. P. Bagad, M. Tapaswi, and C. G. Snoek, “Test of time: Instilling video-language models with a sense of time,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2503–2516.
  196. F. Ma, X. Jin, H. Wang, J. Huang, L. Zhu, J. Feng, and Y. Yang, “Temporal perceiving video-language pre-training,” arXiv preprint arXiv:2301.07463, 2023.
  197. X. He, S. Chen, F. Ma, Z. Huang, X. Jin, Z. Liu, D. Fu, Y. Yang, J. Liu, and J. Feng, “Vlab: Enhancing video language pre-training by feature adapting and blending,” arXiv preprint arXiv:2305.13167, 2023.
  198. J. Huang, Y. Li, J. Feng, X. Wu, X. Sun, and R. Ji, “Clover: Towards a unified video-language alignment and fusion model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14 856–14 866.
  199. F. Cheng, X. Wang, J. Lei, D. Crandall, M. Bansal, and G. Bertasius, “Vindlu: A recipe for effective video-and-language pretraining,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10 739–10 750.
  200. W. Zhong, M. Zheng, D. Tang, X. Luo, H. Gong, X. Feng, and B. Qin, “Stoa-vlp: Spatial-temporal modeling of object and action for video-language pre-training,” arXiv preprint arXiv:2302.09736, 2023.
  201. T.-J. Fu, L. Li, Z. Gan, K. Lin, W. Y. Wang, L. Wang, and Z. Liu, “An empirical study of end-to-end video-language transformers with masked visual modeling,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 898–22 909.
  202. Y. Lin, C. Wei, H. Wang, A. Yuille, and C. Xie, “Smaug: Sparse masked autoencoder for efficient video-language pre-training,” arXiv preprint arXiv:2211.11446, 2022.
  203. J. Yang, X. Li, M. Zheng, Z. Wang, Y. Zhu, X. Guo, Y. Yuan, Z. Chai, and S. Jiang, “Membridge: Video-language pre-training with memory-augmented inter-modality bridge,” IEEE Transactions on Image Processing, 2023.
  204. Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,” in International Conference on Learning Representations (ICLR ’18), 2018.
  205. M. Jin, Y. Zheng, Y.-F. Li, S. Chen, B. Yang, and S. Pan, “Multivariate time series forecasting with dynamic graph neural odes,” IEEE Transactions on Knowledge and Data Engineering, 2022.
  206. C. Chen, K. Petty, A. Skabardonis, P. Varaiya, and Z. Jia, “Freeway performance measurement system: Mining loop detector data,” Transportation Research Record, vol. 1748, no. 1, pp. 96–102, 2001.
  207. L. Xu, H. Huang, and J. Liu, “SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning Over Traffic Events,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021, pp. 9878–9888.
  208. J. Zhang, Y. Zheng, and D. Qi, “Deep spatio-temporal residual networks for citywide crowd flows prediction,” in Proceedings of the AAAI conference on artificial intelligence, vol. 31, no. 1, 2017.
  209. C. Song, Y. Lin, S. Guo, and H. Wan, “Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 01, 2020, pp. 914–921.
  210. C. Zheng, X. Fan, C. Wang, and J. Qi, “Gman: A graph multi-attention network for traffic prediction,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 01, 2020, pp. 1234–1241.
  211. Z. Ullah, F. Al-Turjman, L. Mostarda, and R. Gagliardi, “Applications of artificial intelligence and machine learning in smart cities,” Computer Communications, vol. 154, pp. 313–323, 2020.
  212. A. Ali, Y. Zhu, and M. Zakarya, “Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction,” Neural networks, vol. 145, pp. 233–247, 2022.
  213. H. Yao, X. Tang, H. Wei, G. Zheng, and Z. Li, “Revisiting spatial-temporal similarity: A deep learning framework for traffic prediction,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5668–5675, 07 2019.
  214. M. Li and Z. Zhu, “Spatial-temporal fusion graph neural networks for traffic flow forecasting,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 5, 2021, pp. 4189–4196.
  215. K. Klise, W. Beyeler, P. Finley, and M. Makvandi, “Analysis of mobility data to build contact networks for covid-19,” PLoS One, vol. 16, no. 4, p. e0249726, 2021.
  216. W. Yao, J. Yu, Y. Yang, N. Chen, S. Jin, Y. Hu, and C. Bai, “Understanding travel behavior adjustment under covid-19,” Communications in Transportation Research, vol. 2, p. 100068, 2022.
  217. P. Rutten, M. H. Lees, S. Klous, H. Heesterbeek, and P. M. Sloot, “Modelling the dynamic relationship between spread of infection and observed crowd movement patterns at large scale events,” Scientific Reports, vol. 12, no. 1, p. 14825, 2022.
  218. X. Liu, Y. Xia, Y. Liang, J. Hu, Y. Wang, L. Bai, C. Huang, Z. Liu, B. Hooi, and R. Zimmermann, “Largest: A benchmark dataset for large-scale traffic forecasting,” arXiv preprint arXiv:2306.08259, 2023.
  219. P. Wagner, N. Strodthoff, R.-D. Bousseljot, D. Kreiseler, F. Lunze, W. Samek, and T. Schaeffter, “Ptb-xl, a large publicly available electrocardiography dataset,” Scientific Data, vol. 7, p. 154, 05 2020.
  220. N. Strodthoff, P. Wagner, T. Schaeffter, and W. Samek, “Deep learning for ecg analysis: Benchmarks and insights from ptb-xl,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 5, pp. 1519–1528, 2020.
  221. T. Mehari and N. Strodthoff, “Self-supervised representation learning from 12-lead ecg data,” Computers in biology and medicine, vol. 141, p. 105114, 2022.
  222. J. Wang, X. Qiao, C. Liu, X. Wang, Y. Liu, L. Yao, and H. Zhang, “Automated ecg classification using a non-local convolutional block attention module,” Computer Methods and Programs in Biomedicine, vol. 203, p. 106006, 2021.
  223. R. Hu, J. Chen, and L. Zhou, “A transformer-based deep neural network for arrhythmia detection using continuous ecg signals,” Computers in Biology and Medicine, vol. 144, p. 105325, 2022.
  224. F. Imrie, R. Davis, and M. van der Schaar, “Multiple stakeholders drive diverse interpretability requirements for machine learning in healthcare,” Nature Machine Intelligence, pp. 1–6, 2023.
  225. A. Valizadeh, M. Moassefi, A. Nakhostin-Ansari, S. Heidari Some’eh, H. Hosseini-Asl, M. Saghab Torbati, R. Aghajani, Z. Maleki Ghorbani, I. Menbari-Oskouie, F. Aghajani et al., “Automated diagnosis of autism with artificial intelligence: State of the art,” Reviews in the Neurosciences, no. 0, 2023.
  226. H. Qi, Q. Zhao, C. Song, W. Zhai, D. Luo, S. Liu, Y. J. Yu, F. Wang, H. Zou, B. X. Yang et al., “Evaluating the efficacy of supervised learning vs large language models for identifying cognitive distortions and suicidal risks in chinese social media,” arXiv preprint arXiv:2309.03564, 2023.
  227. T. Han, S. Nebelung, F. Khader, T. Wang, G. Mueller-Franzes, C. Kuhl, S. Försch, J. Kleesiek, C. Haarburger, K. K. Bressem et al., “Medical foundation models are susceptible to targeted misinformation attacks,” arXiv preprint arXiv:2309.17007, 2023.
  228. A. J. Thirunavukarasu, D. S. J. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. W. Ting, “Large language models in medicine,” Nature medicine, pp. 1–11, 2023.
  229. M. Moor, O. Banerjee, Z. S. H. Abad, H. M. Krumholz, J. Leskovec, E. J. Topol, and P. Rajpurkar, “Foundation models for generalist medical artificial intelligence,” Nature, vol. 616, no. 7956, pp. 259–265, 2023.
  230. J. Qiu, L. Li, J. Sun, J. Peng, P. Shi, R. Zhang, Y. Dong, K. Lam, F. P.-W. Lo, B. Xiao et al., “Large ai models in health informatics: Applications, challenges, and the future,” IEEE Journal of Biomedical and Health Informatics, 2023.
  231. W. Sun, A. Rumshisky, and O. Uzuner, “Evaluating temporal relations in clinical text: 2012 i2b2 challenge,” Journal of the American Medical Informatics Association : JAMIA, vol. 20, 04 2013.
  232. Y. Wang, L. Wang, M. Rastegar-Mojarad, S. Moon, F. Shen, N. Afzal, S. Liu, Y. Zeng, S. Mehrabi, S. Sohn et al., “Clinical information extraction applications: a literature review,” Journal of biomedical informatics, vol. 77, pp. 34–49, 2018.
  233. P. Lewis, M. Ott, J. Du, and V. Stoyanov, “Pretrained language models for biomedical and clinical tasks: understanding and extending the state-of-the-art,” in Proceedings of the 3rd Clinical Natural Language Processing Workshop, 2020, pp. 146–157.
  234. S. Henry, K. Buchan, M. Filannino, A. Stubbs, and O. Uzuner, “2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records,” Journal of the American Medical Informatics Association, vol. 27, no. 1, pp. 3–12, 2020.
  235. A. Johnson, T. Pollard, L. Shen, L.-w. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Celi, and R. Mark, “Mimic-iii, a freely accessible critical care database,” Scientific Data, vol. 3, p. 160035, 05 2016.
  236. Y. Peng, S. Yan, and Z. Lu, “Transfer learning in biomedical natural language processing: an evaluation of bert and elmo on ten benchmarking datasets,” arXiv preprint arXiv:1906.05474, 2019.
  237. A. Zhang, L. Xing, J. Zou, and J. C. Wu, “Shifting machine learning for healthcare from development to deployment and from models to data,” Nature Biomedical Engineering, vol. 6, no. 12, pp. 1330–1345, 2022.
  238. J. Xu, B. S. Glicksberg, C. Su, P. Walker, J. Bian, and F. Wang, “Federated learning for healthcare informatics,” Journal of Healthcare Informatics Research, vol. 5, pp. 1–19, 2021.
  239. J. Oliveira, F. Renna, P. D. Costa, M. Nogueira, C. Oliveira, C. Ferreira, A. Jorge, S. Mattos, T. Hatem, T. Tavares, A. Elola, A. B. Rad, R. Sameni, G. D. Clifford, and M. T. Coimbra, “The circor digiscope dataset: From murmur detection to murmur classification,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 6, pp. 2524–2535, 2022.
  240. M. A. Reyna, Y. Kiarashi, A. Elola, J. Oliveira, F. Renna, A. Gu, E. A. P. Alday, N. Sadr, A. Sharma, S. Mattos et al., “Heart murmur detection from phonocardiogram recordings: The george b. moody physionet challenge 2022,” in 2022 Computing in Cardiology (CinC), vol. 498.   IEEE, 2022, pp. 1–4.
  241. Y. N. Fuadah, M. A. Pramudito, and K. M. Lim, “An optimal approach for heart sound classification using grid search in hyperparameter optimization of machine learning,” Bioengineering, vol. 10, no. 1, p. 45, 2022.
  242. A. Ballas, V. Papapanagiotou, A. Delopoulos, and C. Diou, “Listen2yourheart: A self-supervised approach for detecting murmur in heart-beat sounds,” in 2022 Computing in Cardiology (CinC), vol. 498.   IEEE, 2022, pp. 1–4.
  243. M. Veillette, S. Samsi, and C. Mattioli, “Sevir: A storm event imagery dataset for deep learning applications in radar and satellite meteorology,” Advances in Neural Information Processing Systems, vol. 33, pp. 22 009–22 019, 2020.
  244. M. Seo, H. Lee, D. Kim, and J. Seo, “Implicit stacked autoregressive model for video prediction,” arXiv preprint arXiv:2303.07849, 2023.
  245. Z. Gao, X. Shi, H. Wang, Y. Zhu, Y. Wang, M. Li, and D.-Y. Yeung, “Earthformer: Exploring space-time transformers for earth system forecasting,” in NeurIPS, 2022.
  246. A. Malinin, N. Band, G. Chesnokov, Y. Gal, M. J. Gales, A. Noskov, A. Ploskonosov, L. Prokhorenkova, I. Provilkov, V. Raina et al., “Shifts: A dataset of real distributional shift across multiple large-scale tasks,” arXiv preprint arXiv:2107.07455, 2021.
  247. I. Bondarenko, “More layers! end-to-end regression and uncertainty on tabular data with deep learning,” arXiv preprint arXiv:2112.03566, 2021.
  248. Z. Hammoudeh and D. Lowd, “Reducing certified regression to certified classification for general poisoning attacks,” in Proceedings of the 1st IEEE Conference on Secure and Trustworthy Machine Learning, ser. SaTML’23, 2023.
  249. S. Chen, G. Long, T. Shen, T. Zhou, and J. Jiang, “Spatial-temporal prompt learning for federated weather forecasting,” 2023.
  250. S. Chen, T. Shu, H. Zhao, and Y. Y. Tang, “Mask-cnn-transformer for real-time multi-label weather recognition,” Knowledge-Based Systems, vol. 278, p. 110881, 2023.
  251. S. Rasp, P. D. Dueben, S. Scher, J. A. Weyn, S. Mouatadid, and N. Thuerey, “WeatherBench: A benchmark data set for data-driven weather forecasting,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 11, nov 2020.
  252. J. A. Weyn, D. R. Durran, and R. Caruana, “Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 9, sep 2020.
  253. V. Eyring, S. Bony, G. A. Meehl, C. A. Senior, B. Stevens, R. J. Stouffer, and K. E. Taylor, “Overview of the coupled model intercomparison project phase 6 (cmip6) experimental design and organization,” Geoscientific Model Development, vol. 9, no. 5, pp. 1937–1958, 2016.
  254. J.-Y. Lee, J. Marotzke, G. Bala, L. Cao, S. Corti, J. P. Dunne, F. Engelbrecht, E. Fischer, J. C. Fyfe, C. Jones et al., “Future global climate: scenario-based projections and near-term information,” in Climate change 2021: The physical science basis. Contribution of working group I to the sixth assessment report of the intergovernmental panel on climate change.   Cambridge University Press, 2021, pp. 553–672.
  255. A. Hassani, A. Azapagic, and N. Shokri, “Global predictions of primary soil salinization under changing climate in the 21st century,” Nature communications, vol. 12, no. 1, p. 6663, 2021.
  256. M. W. Jones, J. T. Abatzoglou, S. Veraverbeke, N. Andela, G. Lasslop, M. Forkel, A. J. Smith, C. Burton, R. A. Betts, G. R. van der Werf et al., “Global and regional trends and drivers of fire under climate change,” Reviews of Geophysics, vol. 60, no. 3, p. e2020RG000726, 2022.
  257. A. Farhangi, J. Bian, A. Huang, H. Xiong, J. Wang, and Z. Guo, “Aa-forecast: Anomaly-aware forecast for extreme events,” arXiv preprint arXiv:2208.09933, 2022.
  258. Q. Wu, J. Li, Z. Liu, Y. Li, and M. Cucuringu, “Symphony in the latent space: provably integrating high-dimensional techniques with non-linear machine learning models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 9, 2023, pp. 10 361–10 369.
  259. Y. Xu and S. B. Cohen, “Stock movement prediction from tweets and historical prices,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 1970–1979.
  260. R. Sawhney, S. Agarwal, A. Wadhwa, and R. Shah, “Deep attentive learning for stock movement prediction from social media text and company correlations,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 8415–8426.
  261. Q. Xie, W. Han, X. Zhang, Y. Lai, M. Peng, A. Lopez-Lira, and J. Huang, “Pixiu: A large language model, instruction data and evaluation benchmark for finance,” arXiv preprint arXiv:2306.05443, 2023.
  262. W. Jiang, “Applications of deep learning in stock market prediction: recent progress,” Expert Systems with Applications, vol. 184, p. 115537, 2021.
  263. J. Zou, H. Cao, L. Liu, Y. Lin, E. Abbasnejad, and J. Q. Shi, “Astock: A new dataset and automated stock trading based on stock-specific news analyzing model,” arXiv preprint arXiv:2206.06606, 2022.
  264. Y. Xie, D. Wang, P.-Y. Chen, J. Xiong, S. Liu, and S. Koyejo, “A word is worth a thousand dollars: Adversarial attack on tweets fools stock prediction,” arXiv preprint arXiv:2205.01094, 2022.
  265. F. Feng, H. Chen, X. He, J. Ding, M. Sun, and T.-S. Chua, “Enhancing stock movement prediction with adversarial training,” arXiv preprint arXiv:1810.09936, 2018.
  266. Z. Zhou, L. Ma, and H. Liu, “Trade the event: Corporate events detection for news-based event-driven trading,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.   Online: Association for Computational Linguistics, Aug. 2021, pp. 2114–2124.
  267. M. Zhang, J. Yang, M. Wan, X. Zhang, and J. Zhou, “Predicting long-term stock movements with fused textual features of chinese research reports,” Expert Systems with Applications, vol. 210, p. 118312, 2022.
  268. X.-Y. Liu, G. Wang, and D. Zha, “Fingpt: Democratizing internet-scale data for financial large language models,” 2023.
  269. J. Choi, S. Yoo, X. Zhou, and Y. Kim, “Hybrid information mixing module for stock movement prediction,” IEEE Access, vol. 11, pp. 28 781–28 790, 2023.
  270. A. Reneau, J. Y.-C. Hu, C. Xu, W. Li, A. Gilani, and H. Liu, “Feature programming for multivariate time series prediction,” 2023.
  271. X.-Y. Liu, H. Yang, Q. Chen, R. Zhang, L. Yang, B. Xiao, and C. D. Wang, “Finrl: A deep reinforcement learning library for automated stock trading in quantitative finance,” arXiv preprint arXiv:2011.09607, 2020.
  272. Y. Jang, Y. Song, C. D. Kim, Y. Yu, Y. Kim, and G. Kim, “Video Question Answering with Spatio-Temporal Reasoning,” IJCV, 2019.
  273. S. Chen, H. Li, Q. Wang, Z. Zhao, M. Sun, X. Zhu, and J. Liu, “Vast: A vision-audio-subtitle-text omni-modality foundation model and dataset,” arXiv preprint arXiv:2305.18500, 2023.
  274. J. Xu, B. Liu, Y. Chen, M. Cheng, and X. Shi, “Multi: Efficient video-and-language understanding with multiway-sampler and multiple choice modeling,” arXiv preprint arXiv:2303.05707, 2023.
  275. J. Xu, T. Mei, T. Yao, and Y. Rui, “Msr-vtt: A large video description dataset for bridging video and language,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 5288–5296.
  276. J. Jiang, S. Min, W. Kong, H. Wang, Z. Li, and W. Liu, “Tencent text-video retrieval: Hierarchical cross-modal interactions with multi-level representations,” IEEE Access, 2022.
  277. A. Zeng, M. Attarian, B. Ichter, K. Choromanski, A. Wong, S. Welker, F. Tombari, A. Purohit, M. Ryoo, V. Sindhwani et al., “Socratic models: Composing zero-shot multimodal reasoning with language,” arXiv preprint arXiv:2204.00598, 2022.
  278. M. Bain, A. Nagrani, G. Varol, and A. Zisserman, “Frozen in time: A joint video and image encoder for end-to-end retrieval,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1728–1738.
  279. W. Wang, H. Yang, Z. Tuo, H. He, J. Zhu, J. Fu, and J. Liu, “Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation,” arXiv preprint arXiv:2305.10874, 2023.
  280. Z. Luo, D. Chen, Y. Zhang, Y. Huang, L. Wang, Y. Shen, D. Zhao, J. Zhou, and T. Tan, “Videofusion: Decomposed diffusion models for high-quality video generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10 209–10 218.
  281. U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni et al., “Make-a-video: Text-to-video generation without text-video data,” arXiv preprint arXiv:2209.14792, 2022.
  282. H. Luo, L. Ji, M. Zhong, Y. Chen, W. Lei, N. Duan, and T. Li, “Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning,” Neurocomputing, vol. 508, pp. 293–304, 2022.
  283. D. Chen and W. B. Dolan, “Collecting highly parallel data for paraphrase evaluation,” in Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, 2011, pp. 190–200.
  284. W. Kuo, A. Piergiovanni, D. Kim, X. Luo, B. Caine, W. Li, A. Ogale, L. Zhou, A. Dai, Z. Chen et al., “Mammut: A simple architecture for joint learning for multimodal tasks,” arXiv preprint arXiv:2303.16839, 2023.
  285. K. Li, Y. Wang, Y. Li, Y. Wang, Y. He, L. Wang, and Y. Qiao, “Unmasked teacher: Towards training-efficient video foundation models,” arXiv preprint arXiv:2303.16058, 2023.
  286. L. Anne Hendricks, O. Wang, E. Shechtman, J. Sivic, T. Darrell, and B. Russell, “Localizing moments in video with natural language,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5803–5812.
  287. M. Soldan, M. Xu, S. Qu, J. Tegner, and B. Ghanem, “Vlg-net: Video-language graph matching network for video grounding,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3224–3234.
  288. D. Li, J. Li, H. Le, G. Wang, S. Savarese, and S. C. Hoi, “Lavis: A library for language-vision intelligence,” arXiv preprint arXiv:2209.09019, 2022.
  289. Y. Wang, Y. He, Y. Li, K. Li, J. Yu, X. Ma, X. Chen, Y. Wang, P. Luo, Z. Liu et al., “Internvid: A large-scale video-text dataset for multimodal understanding and generation,” arXiv preprint arXiv:2307.06942, 2023.
  290. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13.   Springer, 2014, pp. 740–755.
  291. Z. Zong, G. Song, and Y. Liu, “Detrs with collaborative hybrid assignments training,” arXiv preprint arXiv:2211.12860, 2022.
  292. W. Wang, J. Dai, Z. Chen, Z. Huang, Z. Li, X. Zhu, X. Hu, T. Lu, L. Lu, H. Li et al., “Internimage: Exploring large-scale vision foundation models with deformable convolutions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14 408–14 419.
  293. J. Yu, Y. Xu, J. Y. Koh, T. Luong, G. Baid, Z. Wang, V. Vasudevan, A. Ku, Y. Yang, B. K. Ayan et al., “Scaling autoregressive models for content-rich text-to-image generation,” arXiv preprint arXiv:2206.10789, vol. 2, no. 3, p. 5, 2022.
  294. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
  295. S. Xue, X. Shi, Z. Chu, Y. Wang, F. Zhou, H. Hao, C. Jiang, C. Pan, Y. Xu, J. Y. Zhang, Q. Wen, J. Zhou, and H. Mei, “Easytpp: Towards open benchmarking the temporal point processes,” arXiv preprint arXiv:2307.08097, 2023.
  296. Q. Zhang, A. Lipani, O. Kirnap, and E. Yilmaz, “Self-attentive Hawkes process,” in Proceedings of the International Conference on Machine Learning (ICML), 2020.
  297. S. Zuo, H. Jiang, Z. Li, T. Zhao, and H. Zha, “Transformer Hawkes process,” in International Conference on Machine Learning, 2020, pp. 11 692–11 702.
  298. S. Xue, X. Shi, H. Hao, L. Ma, J. Zhang, S. Wang, and S. Wang, “A graph regularized point process model for event propagation sequence,” in International Joint Conference on Neural Networks (IJCNN), 2021, pp. 1–7.
  299. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, vol. 35, no. 12.   AAAI Press, 2021, pp. 11 106–11 115.
  300. A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  301. M. Liu, A. Zeng, M. Chen, Z. Xu, Q. Lai, L. Ma, and Q. Xu, “Scinet: Time series modeling and forecasting with sample convolution and interaction,” Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), 2022, 2022.
  302. S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” arXiv:1803.01271, 2018.
  303. S. Makridakis, E. Spiliotis, and V. Assimakopoulos, “The m4 competition: Results, findings, conclusion and way forward,” International Journal of Forecasting, vol. 34, no. 4, pp. 802–808, 2018.
  304. P. Remy, “N-beats: Neural basis expansion analysis for interpretable time series forecasting,” https://github.com/philipperemy/n-beats, 2020.
  305. N. Erickson, J. Mueller, A. Shirkov, H. Zhang, P. Larroy, M. Li, and A. Smola, “Autogluon-tabular: Robust and accurate automl for structured data,” arXiv preprint arXiv:2003.06505, 2020.
  306. A. Bhatnagar, P. Kassianik, C. Liu, T. Lan, W. Yang, R. Cassius, D. Sahoo, D. Arpit, S. Subramanian, G. Woo, A. Saha, A. K. Jagota, G. Gopalakrishnan, M. Singh, K. C. Krithika, S. Maddineni, D. Cho, B. Zong, Y. Zhou, C. Xiong, S. Savarese, S. Hoi, and H. Wang, “Merlion: A machine learning library for time series,” arXiv preprint arXiv:2109.09265, 2021.
  307. I. Godfried, K. Mahajan, M. Wang, K. Li, and P. Tiwari, “Flowdb a large scale precipitation, river, and flash flood dataset,” 2020.
  308. G. Hebrail and A. Berard, “Individual household electric power consumption,” UCI Machine Learning Repository, 2012, DOI: https://doi.org/10.24432/C58K54.
  309. W. Du, D. Cote, and Y. Liu, “SAITS: Self-Attention-based Imputation for Time Series,” Expert Systems with Applications, vol. 219, p. 119619, 2023.
  310. Z. Lai, D. Zhang, H. Li, C. S. Jensen, H. Lu, and Y. Zhao, “Lightcts: A lightweight framework for correlated time series forecasting,” Proceedings of the ACM on Management of Data, vol. 1, no. 2, pp. 1–26, 2023.
  311. Y. Cheng, Z. Chai, and A. Anwar, “Characterizing co-located datacenter workloads: An alibaba case study,” 2018.
  312. Y. Fan, Z. Lan, P. Rich, W. E. Allcock, M. E. Papka, B. Austin, and D. Paul, “Scheduling beyond CPUs for HPC,” in Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing.   ACM, jun 2019.
  313. C. Mommessin, R. Yang, N. V. Shakhlevich, X. Sun, S. Kumar, J. Xiao, and J. Xu, “Affinity-aware resource provisioning for long-running applications in shared clusters,” 2022.
  314. J. Hübotter, “Implementation of algorithms for right-sizing data centers,” 2021.
  315. P. Schäfer, A. Ermshaus, and U. Leser, “Clasp - time series segmentation,” in CIKM, 2021.
  316. S. Deldari, D. V. Smith, A. Sadri, and F. D. Salim, “Entropy and shape aware time-series segmentation for processing heterogeneous sensor data,” in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), vol. 4, no. 3, 2020.
  317. A. Ermshaus, P. Sch”afer, and U. Leser, “Window size selection in unsupervised time series analytics: A review and benchmark,” in Advanced Analytics and Learning on Temporal Data, 2023.
  318. H. A. Dau, A. Bagnall, K. Kamgar, C.-C. M. Yeh, Y. Zhu, S. Gharghabi, C. A. Ratanamahatana, and E. Keogh, “The ucr time series archive,” 2019.
  319. A. Guillaume, C. Vrain, and W. Elloumi, “Random dilated shapelet transform: A new approach for time series shapelets,” in Pattern Recognition and Artificial Intelligence.   Cham: Springer International Publishing, 2022, pp. 653–664.
  320. L. Cheng, R. Khalitov, T. Yu, J. Zhang, and Z. Yang, “Classification of long sequential data using circular dilated convolutional neural networks,” Neurocomputing, 2022.
  321. P. Schäfer and U. Leser, “WEASEL 2.0 - A Random Dilated Dictionary Transform for Fast, Accurate and Memory Constrained Time Series Classification,” arXiv preprint arXiv:2301.10194, 2023.
  322. D. Wang, X. Wang, L. Chen, S. Yao, M. Jing, H. Li, L. Li, S. Bao, F.-Y. Wang, and Y. Lin, “Transworldng: Traffic simulation via foundation model,” arXiv preprint arXiv:2305.15743, 2023.
  323. J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “Biobert: a pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020.
  324. K. Huang, J. Altosaar, and R. Ranganath, “Clinicalbert: Modeling clinical notes and predicting hospital readmission,” CHIL 2020 Workshop, 2020.
  325. D. Jarrett, J. Yoon, I. Bica, Z. Qian, A. Ercole, and M. van der Schaar, “Clairvoyance: A pipeline toolkit for medical time series,” in International Conference on Learning Representations, 2020.
  326. S. Rasp, P. D. Dueben, S. Scher, J. A. Weyn, S. Mouatadid, and N. Thuerey, “Weatherbench: a benchmark data set for data-driven weather forecasting,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 11, p. e2020MS002203, 2020.
  327. H. Yang, X.-Y. Liu, and C. D. Wang, “Fingpt: Open-source financial large language models,” arXiv preprint arXiv:2306.06031, 2023.
  328. S. Xue, F. Zhou, Y. Xu, H. Zhao, S. Xie, C. Jiang, J. Zhang, J. Zhou, D. Xiu, and H. Mei, “Weaverbird: Empowering financial decision-making with large language model, knowledge base, and search engine,” 2023.
  329. J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in International Conference on Machine Learning.   PMLR, 2022, pp. 12 888–12 900.
  330. J. Lu, D. Batra, D. Parikh, and S. Lee, “Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks,” Advances in neural information processing systems, vol. 32, 2019.
  331. L. H. Li, M. Yatskar, D. Yin, C.-J. Hsieh, and K.-W. Chang, “Visualbert: A simple and performant baseline for vision and language,” arXiv preprint arXiv:1908.03557, 2019.
  332. E. Bacry, M. Bompaire, S. Gaïffas, and S. Poulsen, “tick: a Python library for statistical learning, with a particular emphasis on time-dependent modeling,” ArXiv e-prints, 2017.
  333. C. Tan, S. Li, Z. Gao, W. Guan, Z. Wang, Z. Liu, L. Wu, and S. Z. Li, “Openstl: A comprehensive benchmark of spatio-temporal predictive learning,” in Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
  334. J. Herzen, F. Lässig, S. G. Piazzetta, T. Neuer, L. Tafti, G. Raille, T. Van Pottelbergh, M. Pasieka, A. Skrodzki, N. Huguenin et al., “Darts: User-friendly modern machine learning for time series,” Journal of Machine Learning Research, vol. 23, no. 124, pp. 1–6, 2022.
  335. B. Rozemberczki, P. Scherer, Y. He, G. Panagopoulos, A. Riedel, M. Astefanoaei, O. Kiss, F. Beres, G. Lopez, N. Collignon, and R. Sarkar, “PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models,” in Proceedings of the 30th ACM International Conference on Information and Knowledge Management, 2021, p. 4564–4573.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (15)
  1. Ming Jin (130 papers)
  2. Qingsong Wen (139 papers)
  3. Yuxuan Liang (126 papers)
  4. Chaoli Zhang (24 papers)
  5. Siqiao Xue (29 papers)
  6. Xue Wang (69 papers)
  7. James Zhang (36 papers)
  8. Yi Wang (1038 papers)
  9. Haifeng Chen (99 papers)
  10. Xiaoli Li (120 papers)
  11. Shirui Pan (198 papers)
  12. Vincent S. Tseng (9 papers)
  13. Yu Zheng (196 papers)
  14. Lei Chen (485 papers)
  15. Hui Xiong (244 papers)
Citations (88)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now