Papers
Topics
Authors
Recent
2000 character limit reached

RATSF: Empowering Customer Service Volume Management through Retrieval-Augmented Time-Series Forecasting (2403.04180v2)

Published 7 Mar 2024 in cs.LG and cs.IR

Abstract: An efficient customer service management system hinges on precise forecasting of service volume. In this scenario, where data non-stationarity is pronounced, successful forecasting heavily relies on identifying and leveraging similar historical data rather than merely summarizing periodic patterns. Existing models based on RNN or Transformer architectures may struggle with this flexible and effective utilization. To tackle this challenge, we initially developed the Time Series Knowledge Base (TSKB) with an advanced indexing system for efficient historical data retrieval. We also developed the Retrieval Augmented Cross-Attention (RACA) module, a variant of the cross-attention mechanism within Transformer's decoder layers, designed to be seamlessly integrated into the vanilla Transformer architecture to assimilate key historical data segments. The synergy between TSKB and RACA forms the backbone of our Retrieval-Augmented Time Series Forecasting (RATSF) framework. Based on the above two components, RATSF not only significantly enhances performance in the context of Fliggy hotel service volume forecasting but also adapts flexibly to various scenarios and integrates with a multitude of Transformer variants for time-series forecasting. Extensive experimentation has validated the effectiveness and generalizability of this system design across multiple diverse contexts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Time-series, volume 2nd edn. J. R. Stat. Soc. (Series D). 1976.
  2. Anonymous. Retrieval meets long context large language models. In The Twelfth International Conference on Learning Representations, 2024.
  3. Retrieval-augmented transformer-xl for close-domain dialog generation. In The 34th International FLAIRS Conference, 5 2021.
  4. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206–2240. PMLR, 2022.
  5. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344–16359, 2022.
  6. Long-term forecasting with tiDE: Time-series dense encoder. Transactions on Machine Learning Research, 2023.
  7. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR, 2020.
  8. Learning sparse latent graph representations for anomaly detection in multivariate time series. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2977–2986, 2022.
  9. Forecasting: principles and practice. OTexts, 2018.
  10. Perceiver: General perception with iterative attention. In International conference on machine learning, pages 4651–4664. PMLR, 2021.
  11. Retrieval based time series forecasting. 9 2022.
  12. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representations, 2020.
  13. Reformer: The efficient transformer. In International Conference on Learning Representations, 2020.
  14. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4):1748–1764, 2021.
  15. Lost in the middle: How language models use long contexts, 2023. arXiv:2307.03172.
  16. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations, 2021.
  17. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881–9893, 2022.
  18. Controllable semantic parsing via retrieval augmentation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7683–7698, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
  19. From known to unknown: Knowledge-guided transformer for time-series sales forecasting in alibaba. CoRR, abs/2109.08381, 2021.
  20. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191, 2020.
  21. Retrieval augmentation reduces hallucination in conversation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784–3803, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
  22. DAMO-NLP at SemEval-2023 task 2: A unified retrieval-augmented system for multilingual named entity recognition. In Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, and Elisa Sartori, editors, Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2014–2028, Toronto, Canada, July 2023. Association for Computational Linguistics.
  23. Forecasting at scale. The American Statistician, 72(1):37–45, 2018.
  24. Unsupervised representation learning for time series with temporal neighborhood coding. In International Conference on Learning Representations, 2021.
  25. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  26. Etsformer: Exponential smoothing transformers for time-series forecasting. 2022.
  27. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.
  28. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34:22419–22430, 2021.
  29. Mqretnn: Multi-horizon time series forecasting with retrieval augmentation. arXiv preprint arXiv:2207.10517, 2022.
  30. Learning the evolutionary and multi-scale graph structure for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2296–2306, 2022.
  31. End-to-end distantly supervised information extraction with retrieval augmentation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2449–2455, 2022.
  32. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021.
  33. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proc. 39th International Conference on Machine Learning (ICML 2022), 2022.
  34. Rucir at trec 2019: Conversational assistance track. National Institute of Standards and Technology (NIST), 2019.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free