Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RobustTSF: Towards Theory and Design of Robust Time Series Forecasting with Anomalies (2402.02032v1)

Published 3 Feb 2024 in cs.LG

Abstract: Time series forecasting is an important and forefront task in many real-world applications. However, most of time series forecasting techniques assume that the training data is clean without anomalies. This assumption is unrealistic since the collected time series data can be contaminated in practice. The forecasting model will be inferior if it is directly trained by time series with anomalies. Thus it is essential to develop methods to automatically learn a robust forecasting model from the contaminated data. In this paper, we first statistically define three types of anomalies, then theoretically and experimentally analyze the loss robustness and sample robustness when these anomalies exist. Based on our analyses, we propose a simple and efficient algorithm to learn a robust forecasting model. Extensive experiments show that our method is highly robust and outperforms all existing approaches. The code is available at https://github.com/haochenglouis/RobustTSF.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Self-adaptive forecasting for improved deep learning on non-stationary time-series. arXiv preprint arXiv:2202.02403, 2022.
  2. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
  3. A review on outlier/anomaly detection in time series data. ACM Computing Surveys (CSUR), 54(3):1–33, 2021.
  4. Resilient neural forecasting systems. In Proceedings of ACM SIGMOD Workshop on Data Management for End-to-End Machine Learning (SIGMOD-DEEM 2020), 2020.
  5. Learning with instance-dependent label noise: A sample sieve approach. In International Conference on Learning Representations, 2021.
  6. Recurrent neural networks and robust time series prediction. IEEE transactions on neural networks, 5(2):240–254, 1994.
  7. Normalizing kalman filters for multivariate time series analysis. Advances in Neural Information Processing Systems, 33:2995–3007, 2020.
  8. Learning in nonstationary environments: A survey. IEEE Computational Intelligence Magazine, 10(4):12–25, 2015.
  9. Robust loss functions under label noise for deep neural networks. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.
  10. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
  11. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems, pp.  8527–8537, 2018.
  12. The use of arima models for reliability forecasting and analysis. Computers & industrial engineering, 35(1-2):213–216, 1998.
  13. MentorNet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In International Conference on Machine Learning (ICML), pp.  2304–2313, 2018.
  14. ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT trend filtering. SIAM review, 51(2):339–360, 2009.
  15. Trimmed maximum likelihood estimation for robust learning in generalized linear models. arXiv preprint arXiv:2206.04777, 2022.
  16. Structured inference networks for nonlinear state space models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
  17. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp.  95–104, 2018.
  18. Revisiting time series outlier detection: Definitions and benchmarks. In Thirty-fifth conference on neural information processing systems (NeurIPS), 2021.
  19. Outlier impact characterization for time series data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  11595–11603, 2021.
  20. DivideMix: Learning with noisy labels as semi-supervised learning. In International Conference on Learning Representations, 2020.
  21. Robust learning of deep time series anomaly detection models with contaminated training data. arXiv preprint arXiv:2208.01841, 2022.
  22. Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.
  23. Yang Liu. Understanding instance-level label noise: Disparate impacts and treatments. In International Conference on Machine Learning, pp.  6725–6735, 2021.
  24. Peer loss functions: Learning from noisy labels without knowing noise rates. In International Conference on Machine Learning (ICML), pp.  6226–6236, 2020.
  25. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12):2346–2363, 2018.
  26. Normalized loss functions for deep learning with noisy labels. In International Conference on Machine Learning (ICML), pp.  6543–6553, 2020.
  27. Learning with noisy labels. In Advances in neural information processing systems, pp.  1196–1204, 2013.
  28. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.  1944–1952, 2017.
  29. Robust regression with covariate filtering: Heavy tails and adversarial contamination. arXiv preprint arXiv:2009.12976, 2020.
  30. High-dimensional multivariate forecasting with low-rank Gaussian copula processes. Advances in neural information processing systems, 32, 2019.
  31. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191, 2020.
  32. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  33. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
  34. Huber additive models for non-stationary time series analysis. In International Conference on Learning Representations, 2021a.
  35. Adaptive data augmentation on temporal graphs. Advances in Neural Information Processing Systems, 34:1440–1452, 2021b.
  36. Robust time series analysis and applications: An industrial perspective. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp.  4836–4837, 2022.
  37. DeepTIMe: Deep time-index meta-learning for non-stationary time-series forecasting. arXiv preprint arXiv:2207.06046, 2022.
  38. Dynamic gaussian mixture based deep generative model for robust forecasting on sparse multivariate time series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  651–659, 2021.
  39. L_dmi: A novel information-theoretic loss function for training deep nets robust to label noise. In Advances in Neural Information Processing Systems, volume 32, 2019.
  40. Robust probabilistic time series forecasting. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, pp.  1336–1358, 2022.
  41. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp.  11121–11128, 2023.
  42. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in Neural Information Processing Systems, pp.  8778–8788, 2018.
  43. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp.  11106–11115, 2021.
  44. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pp.  27268–27286. PMLR, 2022.
  45. A second-order approach to learning with instance-dependent label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  10113–10123, 2021a.
  46. Clusterability as an alternative to anchor points when learning with noisy labels. In International Conference on Machine Learning (ICML), pp.  12912–12923, 2021b.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - haochenglouis/RobustTSF: [ICLR 2024] Official code of RobustTSF (20 stars)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets