Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting (2403.05406v1)

Published 8 Mar 2024 in cs.LG and cs.AI

Abstract: The forecasting of Multivariate Time Series (MTS) has long been an important but challenging task. Due to the non-stationary problem across long-distance time steps, previous studies primarily adopt stationarization method to attenuate the non-stationary problem of the original series for better predictability. However, existing methods always adopt the stationarized series, which ignores the inherent non-stationarity, and has difficulty in modeling MTS with complex distributions due to the lack of stochasticity. To tackle these problems, we first develop a powerful hierarchical probabilistic generative module to consider the non-stationarity and stochastic characteristics within MTS, and then combine it with transformer for a well-defined variational generative dynamic model named Hierarchical Time series Variational Transformer (HTV-Trans), which recovers the intrinsic non-stationary information into temporal dependencies. Being a powerful probabilistic model, HTV-Trans is utilized to learn expressive representations of MTS and applied to forecasting tasks. Extensive experiments on diverse datasets show the efficiency of HTV-Trans on MTS forecasting tasks

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. Passenger demand forecasting with multi-task convolutional recurrent neural networks. in PAKDD, 29–42.
  2. Box, G. E. P. 1976. Time series analysis, forecasting and control rev. ed. Time series analysis, forecasting and control rev. ed.
  3. Spectral temporal graph neural network for multivariate time-series forecasting. in NeurIPS, 33: 17766–17778.
  4. Autoformer: Searching transformers for visual recognition. in CVPR, 12270–12280.
  5. Infinite Switching Dynamic Probabilistic Network with Bayesian Nonparametric Learning. in TSP.
  6. Switching Poisson gamma dynamical systems. in IJCAI.
  7. Switching Gaussian Mixture Variational RNN for Anomaly Detection of Diverse CDN Websites. in INFOCOM.
  8. SDFVAE: Static and Dynamic Factorized VAE for Anomaly Detection of Multivariate CDN KPIs. in WWW, 3076–3086.
  9. An image is worth 16x16 words: Transformers for image recognition at scale. in ICLR.
  10. Forecasting: principles and practice. OTexts.
  11. Reversible instance normalization for accurate time-series forecasting against distribution shift. in CVPR.
  12. Reformer: The efficient transformer. in ICLR.
  13. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In ICLR.
  14. Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting. in NeurIPS.
  15. LSTM-based encoder-decoder for multi-sensor anomaly detection. in ICML.
  16. Adaptive normalization: A novel data normalization approach for non-stationary time series. In The 2010 International Joint Conference on Neural Networks (IJCNN), 1–8. IEEE.
  17. Adaptive Normalization: A novel data normalization approach for non-stationary time series. In International Joint Conference on Neural Networks, IJCNN 2010, Barcelona, Spain, 18-23 July, 2010.
  18. Deep adaptive input normalization for time series forecasting. IEEE transactions on neural networks and learning systems, 31(9): 3760–3765.
  19. Deep Adaptive Input Normalization for Time Series Forecasting. Papers.
  20. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. in International Journal of Forecasting, 36(3): 1181–1191.
  21. Joint modeling of local and global temporal dynamics for multivariate time series forecasting with missing values. in AAAI, 34(04): 5956–5963.
  22. Attention is all you need. in NeurIPS, 30.
  23. Transformers in Time Series: A Survey. in arXiv preprint arXiv:2202.07125.
  24. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34: 22419–22430.
  25. Deep multi-view spatial-temporal network for taxi demand prediction. in AAAI, 32(1).
  26. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. in AAAI, 33(01): 1409–1416.
  27. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting. In The Eleventh International Conference on Learning Representations.
  28. Informer: Beyond efficient transformer for long sequence time-series forecasting. in AAAI.
  29. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Muyao Wang (6 papers)
  2. Wenchao Chen (17 papers)
  3. Bo Chen (309 papers)
Citations (2)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets