Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GEANN: Scalable Graph Augmentations for Multi-Horizon Time Series Forecasting (2307.03595v1)

Published 7 Jul 2023 in cs.LG and cs.AI

Abstract: Encoder-decoder deep neural networks have been increasingly studied for multi-horizon time series forecasting, especially in real-world applications. However, to forecast accurately, these sophisticated models typically rely on a large number of time series examples with substantial history. A rapidly growing topic of interest is forecasting time series which lack sufficient historical data -- often referred to as the cold start'' problem. In this paper, we introduce a novel yet simple method to address this problem by leveraging graph neural networks (GNNs) as a data augmentation for enhancing the encoder used by such forecasters. These GNN-based features can capture complex inter-series relationships, and their generation process can be optimized end-to-end with the forecasting task. We show that our architecture can use either data-driven or domain knowledge-defined graphs, scaling to incorporate information from multiple very large graphs with millions of nodes. In our target application of demand forecasting for a large e-commerce retailer, we demonstrate on both a small dataset of 100K products and a large dataset with over 2 million products that our method improves overall performance over competitive baseline models. More importantly, we show that it brings substantially more gains tocold start'' products such as those newly launched or recently out-of-stock.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Cold-start promotional sales forecasting through gradient boosted-based contrastive explanations. IEEE Access 8 (2020), 137574–137586.
  2. Deep Learning for Time Series Forecasting: Tutorial and Literature Survey. Comput. Surveys (may 2022).
  3. Are word embedding methods stable and should we care about it?. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media. 45–55.
  4. Conditional time series forecasting with convolutional neural networks. arXiv preprint arXiv:1703.04691 (2017).
  5. A cross-learning approach for cold-start forecasting of residential photovoltaic generation. Electric Power Systems Research 212 (2022), 108415.
  6. Probabilistic demand forecasting at scale. In VLDB 2017.
  7. Multi-horizon inflation forecasts using disaggregated data. Economic Modelling 27, 3 (2010), 666–677.
  8. Time Series Forecasting for Cold-Start Items by Learning from Related Items using Memory Networks. In Companion Proceedings of the Web Conference 2020. 120–121.
  9. Probabilistic forecasting with temporal convolutional neural network. Neurocomputing 399 (2020), 491–501.
  10. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’19). 257–266.
  11. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
  12. MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attention. https://arxiv.org/abs/2009.14799
  13. LASAGNE: Locality And Structure Aware Graph Node Embedding. IEEE/WIC/ACM International Conference on Web Intelligence (10 2017).
  14. Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
  15. Spatio-Temporal Multi-Graph Networks for Demand Forecasting in Online Marketplaces. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part IV. 187–203.
  16. Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems, Vol. 30.
  17. Deep Convolutional Networks on Graph-Structured Data. https://arxiv.org/abs/1506.05163
  18. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
  19. https://arxiv.org/abs/2106.05150
  20. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
  21. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems 32 (2019).
  22. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926 (2017).
  23. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In International Conference on Learning Representations. https://openreview.net/forum?id=SJiHXGWAZ
  24. Bryan Lim. 2018. Forecasting Treatment Responses Over Time Using Recurrent Marginal Structural Networks. In Advances in Neural Information Processing Systems, Vol. 31.
  25. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting 37, 4 (2021), 1748–1764.
  26. Local Augmentation for Graph Neural Networks. CoRR abs/2109.03856 (2021). arXiv:2109.03856 https://arxiv.org/abs/2109.03856
  27. Sample path generation for probabilistic demand forecasting. In KDD 2018 Workshop on Mining and Learning from Time Series. https://www.amazon.science/publications/sample-path-generation-for-probabilistic-demand-forecasting
  28. Solving the cold-start problem in short-term load forecasting using tree-based methods. Energies 13, 4 (2020), 886.
  29. Metropolis-Hastings Data Augmentation for Graph Neural Networks. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., 19010–19020.
  30. Multivariate Time Series Forecasting with Latent Graph Inference. https://openreview.net/forum?id=JpNH4CW_zl
  31. Omer Berat Sezer and Ahmet Murat Ozbayoglu. 2018. Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach. Applied Soft Computing 70 (2018), 525–538.
  32. Discrete graph structure learning for forecasting multiple time series. arXiv preprint arXiv:2101.06861 (2021).
  33. Sequence to sequence learning with neural networks. In NIPS.
  34. Graph Attention Networks. https://arxiv.org/abs/1710.10903
  35. Graph Attention Networks. In International Conference on Learning Representations.
  36. Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks. arXiv preprint arXiv:1909.01315 (2019).
  37. Adaptive Data Augmentation on Temporal Graphs. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., 1440–1452.
  38. Ruofeng Wen and Kari Torkkola. 2019. Deep generative quantile-copula models for probabilistic forecasting. In ICML 2019 Workshop on Time Series. https://www.amazon.science/publications/deep-generative-quantile-copula-models-for-probabilistic-forecasting
  39. A Multi-Horizon Quantile Recurrent Forecaster. https://arxiv.org/abs/1711.11053
  40. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317 (2020).
  41. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. arXiv:2005.11650 [cs.LG]
  42. MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval Augmentation. KDD. https://doi.org/10.48550/ARXIV.2207.10517
  43. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875 (2017).
  44. A study of joint graph inference and forecasting. In ICML 2021 Time Series Workshop.
  45. J N Zhang and Kazumitsu Nawata. 2018. Multi-step prediction for influenza outbreak by an adjusted long short-term memory. Epidemiology and Infection 146 (2018), 809 – 816.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Sitan Yang (4 papers)
  2. Malcolm Wolff (4 papers)
  3. Shankar Ramasubramanian (3 papers)
  4. Ronak Metha (1 paper)
  5. Michael W. Mahoney (233 papers)
  6. Vincent Quenneville-Belair (1 paper)
Citations (1)
View on Semantic Scholar