Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

From Similarity to Superiority: Channel Clustering for Time Series Forecasting (2404.01340v2)

Published 31 Mar 2024 in cs.LG and cs.AI

Abstract: Time series forecasting has attracted significant attention in recent decades. Previous studies have demonstrated that the Channel-Independent (CI) strategy improves forecasting performance by treating different channels individually, while it leads to poor generalization on unseen instances and ignores potentially necessary interactions between channels. Conversely, the Channel-Dependent (CD) strategy mixes all channels with even irrelevant and indiscriminate information, which, however, results in oversmoothing issues and limits forecasting accuracy. There is a lack of channel strategy that effectively balances individual channel treatment for improved forecasting performance without overlooking essential interactions between channels. Motivated by our observation of a correlation between the time series model's performance boost against channel mixing and the intrinsic similarity on a pair of channels, we developed a novel and adaptable Channel Clustering Module (CCM). CCM dynamically groups channels characterized by intrinsic similarities and leverages cluster information instead of individual channel identities, combining the best of CD and CI worlds. Extensive experiments on real-world datasets demonstrate that CCM can (1) boost the performance of CI and CD models by an average margin of 2.4% and 7.2% on long-term and short-term forecasting, respectively; (2) enable zero-shot forecasting with mainstream time series forecasting models; (3) uncover intrinsic time series patterns among channels and improve interpretability of complex time series models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (81)
  1. An empirical comparison of machine learning models for time series forecasting. Econometric reviews, 29(5-6):594–621, 2010.
  2. Energy time series forecasting based on pattern sequence similarity. IEEE Transactions on Knowledge and Data Engineering, 23(8):1230–1243, 2010.
  3. Multivariate time series dataset for space weather data analytics. Scientific data, 7(1):1–13, 2020.
  4. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
  5. Wavelet-based nonlinear multiscale decomposition model for electricity load forecasting. Neurocomputing, 70(1-3):139–154, 2006.
  6. On the marriage of lp-norms and edit distance. In Proceedings of the Thirtieth international conference on Very large data bases-Volume 30, pp.  792–803, 2004.
  7. Tsmixer: An all-mlp architecture for time series forecasting. arXiv preprint arXiv:2303.06053, 2023.
  8. Stl: A seasonal-trend decomposition. J. Off. Stat, 6(1):3–73, 1990.
  9. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08424, 2023.
  10. Querying and mining of time series data: experimental comparison of representations and distance measures. Proceedings of the VLDB Endowment, 1(2):1542–1552, 2008.
  11. Efficient high-resolution time series classification via attention kronecker decomposition. arXiv preprint arXiv:2403.04882, 2024.
  12. An integrated clustering and bert framework for improved topic modeling. International Journal of Information Technology, pp.  1–9, 2023.
  13. Dynamic time warping and geometric edit distance: Breaking the quadratic barrier. ACM Transactions on Algorithms (TALG), 14(4):1–17, 2018.
  14. Clapp: A self constructing feature clustering approach for anomaly detection. Future Generation Computer Systems, 74:417–429, 2017.
  15. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. arXiv preprint arXiv:2304.05206, 2023.
  16. Recurrent neural networks for time series forecasting: Current status and future directions. International Journal of Forecasting, 37(1):388–427, 2021.
  17. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.
  18. Spatial correlation in weather forecast accuracy: a functional time series approach. Computational Statistics, pp.  1–15, 2023.
  19. Spatio-temporal self-supervised learning for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp.  4356–4364, 2023.
  20. A fuzzy self-constructing feature clustering algorithm for text classification. IEEE transactions on knowledge and data engineering, 23(3):335–349, 2010.
  21. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728, 2023.
  22. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
  23. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  24. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451, 2020.
  25. A review and comparison of time series similarity measures. In 29th International Electrotechnical and Computer Science Conference (ERK 2020). Portorozˇ, pp.  21–22, 2020.
  26. Convolutional neural networks for energy time series forecasting. In 2018 international joint conference on neural networks (IJCNN), pp.  1–8. IEEE, 2018.
  27. Modeling long-and short-term temporal patterns with deep neural networks. corr abs/1703.07015 (2017). arXiv preprint arXiv:1703.07015, 2017.
  28. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp.  95–104, 2018.
  29. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416–1421, 2023.
  30. Unsupervised video anomaly detection using feature clustering. IET signal processing, 6(5):521–533, 2012.
  31. Trend modeling for traffic time series analysis: An integrated study. IEEE Transactions on Intelligent Transportation Systems, 16(6):3430–3439, 2015.
  32. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721, 2023a.
  33. Mts-mixers: Multivariate time series forecasting via factorized temporal and channel mixing. arXiv preprint arXiv:2302.04501, 2023b.
  34. Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.
  35. Self-supervised spatiotemporal masking strategy-based models for traffic flow forecasting. Symmetry, 15(11):2002, 2023a.
  36. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816–5828, 2022a.
  37. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference on Learning Representations, 2021.
  38. Non-stationary transformers: Rethinking the stationarity in time series forecasting. arXiv preprint arXiv:2205.14415, 2022b.
  39. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023b.
  40. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. In Thirty-seventh Conference on Neural Information Processing Systems, 2023c.
  41. The m4 competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 34(4):802–808, 2018.
  42. Token pooling in vision transformers for image classification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp.  12–21, 2023.
  43. Principles and algorithms for forecasting groups of time series: Locality and globality. International Journal of Forecasting, 37(4):1632–1653, 2021.
  44. Forecasting the dynamic correlation of stock indices based on deep learning method. Computational Economics, 61(1):35–55, 2023.
  45. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022.
  46. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437, 2019.
  47. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  48. Is channel independent strategy optimal for time series forecasting? arXiv preprint arXiv:2310.17658, 2023.
  49. Wavelet methods for time series analysis, volume 4. Cambridge university press, 2000.
  50. Deep state space models for time series forecasting. Advances in neural information processing systems, 31, 2018.
  51. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191, 2020.
  52. Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. Advances in neural information processing systems, 32, 2019.
  53. Efficient attention: Attention with linear complexities. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp.  3531–3539, 2021.
  54. Smyl, S. A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting. International Journal of Forecasting, 36(1):75–85, 2020.
  55. Unsupervised clustering approach for network anomaly detection. In Networked Digital Technologies: 4th International Conference, NDT 2012, Dubai, UAE, April 24-26, 2012. Proceedings, Part I 4, pp.  135–145. Springer, 2012.
  56. Probabilistic transformer for time series analysis. Advances in Neural Information Processing Systems, 34:23592–23608, 2021.
  57. Forecasting at scale. The American Statistician, 72(1):37–45, 2018.
  58. Team, M. et al. M4 competitor’s guide: prizes and rules. See https://www. m4. unic. ac. cy/wpcontent/uploads/2018/03/M4-CompetitorsGuide. pdf, 2018.
  59. Deep learning for time series forecasting: a survey. Big Data, 9(1):3–21, 2021.
  60. Neuralprophet: Explainable forecasting at scale, 2021.
  61. Trindade, A. ElectricityLoadDiagrams20112014. UCI Machine Learning Repository, 2015. DOI: https://doi.org/10.24432/C58C86.
  62. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  63. Multivariate temporal convolutional network: A deep neural networks approach for multivariate time series forecasting. Electronics, 8(8):876, 2019.
  64. Micn: Multi-scale local and global context modeling for long-term series forecasting. In The Eleventh International Conference on Learning Representations, 2022.
  65. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768, 2020.
  66. Fast robuststl: Efficient and robust seasonal-trend decomposition for time series with complex patterns. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  2203–2213, 2020.
  67. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125, 2022.
  68. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34:22419–22430, 2021.
  69. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022.
  70. Frequency-domain mlps are more effective learners in time series forecasting. arXiv preprint arXiv:2311.06184, 2023.
  71. Forecasting stock prices using stock correlation graph: A graph convolutional network approach. In 2021 International Joint Conference on Neural Networks (IJCNN), pp.  1–8. IEEE, 2021.
  72. Forecasting traffic time series with multivariate predicting method. Applied Mathematics and Computation, 291:266–278, 2016.
  73. Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504, 2022.
  74. Zhang, G. P. Time series forecasting using a hybrid arima and neural network model. Neurocomputing, 50:159–175, 2003.
  75. Compactly supported radial basis function kernels. Technical report, North Carolina State University. Dept. of Statistics, 2004.
  76. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. arXiv preprint arXiv:2207.01186, 2022.
  77. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations, 2022.
  78. Multi-scale transformer pyramid networks for multivariate time series forecasting. IEEE Access, 2024.
  79. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  11106–11115, 2021.
  80. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in Neural Information Processing Systems, 35:12677–12690, 2022a.
  81. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. arXiv preprint arXiv:2201.12740, 2022b.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube