Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting (2312.06220v2)

Published 11 Dec 2023 in cs.LG and cs.AI

Abstract: In the domain of multivariate time series analysis, the concept of channel independence has been increasingly adopted, demonstrating excellent performance due to its ability to eliminate noise and the influence of irrelevant variables. However, such a concept often simplifies the complex interactions among channels, potentially leading to information loss. To address this challenge, we propose a strategy of channel independence followed by mixing. Based on this strategy, we introduce CSformer, a novel framework featuring a two-stage multiheaded self-attention mechanism. This mechanism is designed to extract and integrate both channel-specific and sequence-specific information. Distinctively, CSformer employs parameter sharing to enhance the cooperative effects between these two types of information. Moreover, our framework effectively incorporates sequence and channel adapters, significantly improving the model's ability to identify important information across various dimensions. Extensive experiments on several real-world datasets demonstrate that CSformer achieves state-of-the-art results in terms of overall performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Multivariate time-series sensor vital sign forecasting of cardiovascular and chronic respiratory diseases. Sustainable Computing: Informatics and Systems, 38:100868, 2023.
  2. On the implementation of the artificial neural network approach for forecasting different healthcare events. Diagnostics, 13(7):1310, 2023.
  3. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
  4. David J Bartholomew. Time series analysis forecasting and control., 1971.
  5. Language models are few-shot learners. NeurIPS, 2020.
  6. Towards spatio-temporal aware traffic time series forecasting. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp.  2900–2913. IEEE, 2022.
  7. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08424, 2023.
  8. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  9. Gate-variants of gated recurrent unit (gru) neural networks. In 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS), pp.  1597–1600. IEEE, 2017.
  10. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
  11. Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting. KDD, 2023.
  12. A Graves. Long short-term memory. supervised sequence labelling with recurrent neural networks, 2012, 37–45.
  13. Forecasting with exponential smoothing: the state space approach. Springer Science & Business Media, 2008.
  14. Reversible instance normalization for accurate time-series forecasting against distribution shift. ICLR, 2021.
  15. Informer: Beyond efficient transformer for long sequence time-series forecasting. arXiv: 2012.07436, 2021.
  16. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems, 32, 2019.
  17. Scinet: time series modeling and forecasting with sample convolution and interaction. NeurIPS, 2022a.
  18. Non-stationary transformers: Rethinking the stationarity in time series forecasting. NeurIPS, 2022b.
  19. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023.
  20. Larry R Medsker and LC Jain. Recurrent neural networks. Design and Applications, 5(64-67):2, 2001.
  21. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
  22. Timesql: Improving multivariate time series forecasting with multi-scale patching and smooth quadratic loss. arXiv preprint arXiv:2311.11285, 2023.
  23. A time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023.
  24. Spatio-temporal hierarchical mlp network for traffic forecasting. Information Sciences, 632:543–554, 2023.
  25. Jaydip Sen. A forecasting framework for the indian healthcare sector index. International Journal of Business Forecasting and Marketing Intelligence, 7(4):311–350, 2022.
  26. Wavelet-seq2seq-lstm with attention for time series forecasting of level of dams in hydroelectric power plants. Energy, 274:127350, 2023.
  27. Forecasting power demand in china with a cnn-lstm model including multimodal information. Energy, 263:126012, 2023.
  28. Autoformer: Decomposition transformers with Auto-Correlation for long-term series forecasting. NeurIPS, 2021.
  29. Timesnet: Temporal 2d-variation modeling for general time series analysis. ICLR, 2023.
  30. Frequency-domain mlps are more effective learners in time series forecasting. arXiv preprint arXiv:2311.06184, 2023.
  31. Yi Yin and Pengjian Shang. Forecasting traffic time series with multivariate predicting method. Applied Mathematics and Computation, 291:266–278, 2016.
  32. Are transformers effective for time series forecasting? AAAI, 2023.
  33. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. ICLR, 2023.
  34. A hybrid framework for forecasting power generation of multiple renewable energy sources. Renewable and Sustainable Energy Reviews, 172:113046, 2023.
  35. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. ICML, 2022.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now