Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 49 tok/s
Gemini 2.5 Pro 53 tok/s Pro
GPT-5 Medium 19 tok/s Pro
GPT-5 High 16 tok/s Pro
GPT-4o 103 tok/s Pro
Kimi K2 172 tok/s Pro
GPT OSS 120B 472 tok/s Pro
Claude Sonnet 4 39 tok/s Pro
2000 character limit reached

Context Neural Networks: A Scalable Multivariate Model for Time Series Forecasting (2405.07117v1)

Published 12 May 2024 in cs.LG and cs.AI

Abstract: Real-world time series often exhibit complex interdependencies that cannot be captured in isolation. Global models that model past data from multiple related time series globally while producing series-specific forecasts locally are now common. However, their forecasts for each individual series remain isolated, failing to account for the current state of its neighbouring series. Multivariate models like multivariate attention and graph neural networks can explicitly incorporate inter-series information, thus addressing the shortcomings of global models. However, these techniques exhibit quadratic complexity per timestep, limiting scalability. This paper introduces the Context Neural Network, an efficient linear complexity approach for augmenting time series models with relevant contextual insights from neighbouring time series without significant computational overhead. The proposed method enriches predictive models by providing the target series with real-time information from its neighbours, addressing the limitations of global models, yet remaining computationally tractable for large datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Neural machine translation by jointly learning to align and translate. ICLR 2015.
  2. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018).
  3. Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach. Expert systems with applications 140 (2020), 112896.
  4. Learning from multiple time series: A deep disentangled approach to diversified time series forecasting. arXiv preprint arXiv:2111.04942 (2021).
  5. Irregularity Reflection for Multivariate Time Series Forecasting Using Convolutional Neural Networks. Available at SSRN 4492823 (2023).
  6. Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research 7 (2006), 1–30.
  7. R2n2: Residual recurrent neural networks for multivariate time series forecasting. arXiv preprint arXiv:1709.03159 (2017).
  8. Learning and Integrating Adaptive Global and Local Graph Structures for Multivariate Time Series Forecasting. Available at SSRN 4439467 (2023).
  9. Forecast evaluation for data scientists: common pitfalls and best practices. Data Mining and Knowledge Discovery 37, 2 (2023), 788–832.
  10. Global models for time series forecasting: A simulation study. Pattern Recognition 124 (2022), 108441.
  11. Ken Holden. 1995. Vector auto regression modeling and forecasting. Journal of Forecasting 14, 3 (1995), 159–166.
  12. DSANet: Dual Self-Attention Network for Multivariate Time Series Forecasting. In ACM CIKM. 2129–2132.
  13. Rob J Hyndman and George Athanasopoulos. 2018. Forecasting: principles and practice. OTexts.
  14. Tue V Jensen and Pierre Pinson. 2017. RE-Europe, a large-scale dataset for modeling a highly renewable European electricity system. Scientific data 4, 1 (2017), 1–18.
  15. Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. In ACM SIGIR. 95–104.
  16. Modeling Temporal Patterns with Dilated Convolutions for Time-Series Forecasting. ACM Transactions on Knowledge Discovery from Data (TKDD) 16, 1 (2021), 1–22.
  17. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926 (2017).
  18. Dynamic graph structure learning for multivariate time series forecasting. Pattern Recognition 138 (2023), 109423.
  19. Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting. (2019). arXiv:1912.09363
  20. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems 35 (2022), 5816–5828.
  21. Rosario N Mantegna. 1999. Hierarchical structure in financial markets. The European Physical Journal B-Condensed Matter and Complex Systems 11, 1 (1999), 193–197.
  22. TDG4MSF: A temporal decomposition enhanced graph neural network for multivariate time series forecasting. Applied Intelligence (2023), 1–14.
  23. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In The Eleventh International Conference on Learning Representations.
  24. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.
  25. Gaussian processes for time-series modelling. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 371 (2013).
  26. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting 36, 3 (2020), 1181–1191.
  27. A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions. Journal of Mathematical Psychology 85 (2018), 1–16.
  28. Discrete graph structure learning for forecasting multiple time series. arXiv preprint arXiv:2101.06861 (2021).
  29. Temporal pattern attention for multivariate time series forecasting. 108, 8-9 (2019), 1421–1441.
  30. Slawek Smyl. 2020. A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting. International Journal of Forecasting 36, 1 (2020), 75–85.
  31. ES-dRNN with dynamic attention for short-term load forecasting. In 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
  32. Adaptive dependency learning graph neural networks. Information Sciences 625 (2023), 700–714.
  33. Attention is All you Need. In Advances in Neural Information Processing Systems 30. Curran Associates, Inc., 5998–6008.
  34. Multivariate temporal convolutional network: A deep neural networks approach for multivariate time series forecasting. Electronics 8, 8 (2019), 876.
  35. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD.
  36. COVID19-MLSF: A multi-task learning-based stock market forecasting framework during the COVID-19 pandemic. Expert Systems with Applications 217 (2023), 119549.
  37. Guoqiang Peter Zhang. 2003. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50 (2003), 159–175.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
Lightbulb On Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets