Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Pyramidal Hidden Markov Model For Multivariate Time Series Forecasting (2310.14341v2)

Published 22 Oct 2023 in cs.LG

Abstract: The Hidden Markov Model (HMM) can predict the future value of a time series based on its current and previous values, making it a powerful algorithm for handling various types of time series. Numerous studies have explored the improvement of HMM using advanced techniques, leading to the development of several variations of HMM. Despite these studies indicating the increased competitiveness of HMM compared to other advanced algorithms, few have recognized the significance and impact of incorporating multistep stochastic states into its performance. In this work, we propose a Pyramidal Hidden Markov Model (PHMM) that can capture multiple multistep stochastic states. Initially, a multistep HMM is designed for extracting short multistep stochastic states. Next, a novel time series forecasting structure is proposed based on PHMM, which utilizes pyramid-like stacking to adaptively identify long multistep stochastic states. By employing these two schemes, our model can effectively handle non-stationary and noisy data, while also establishing long-term dependencies for more accurate and comprehensive forecasting. The experimental results on diverse multivariate time series datasets convincingly demonstrate the superior performance of our proposed PHMM compared to its competitive peers in time series forecasting.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. “Non-stationary transformers: Exploring the stationarity in time series forecasting,” in Advances in Neural Inf. Process. Syst., S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds. 2022, vol. 35, pp. 9881–9893, Curran Associates, Inc.
  2. “Generalized out-of-distribution detection: A survey,” arXiv preprint arXiv:2110.11334, 2021.
  3. “A reservoir-driven non-stationary hidden markov model,” Pattern recognition, vol. 45, no. 11, pp. 3985–3996, 2012.
  4. Don X Sun and Li Deng, “Non-stationary hidden markov models for speech recognition,” in Image Models (and Their Speech Model Cousins). Springer, 1996, pp. 161–182.
  5. “Principles of non-stationary hidden markov model and its applications to sequence labeling task,” in Natural Language Processing–IJCNLP 2005: Second International Joint Conference, Jeju Island, Korea, October 11-13, 2005. Proceedings 2. Springer, 2005, pp. 827–837.
  6. “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  7. “Auto-encoding variational bayes,” 2022.
  8. “Neural hmms are all you need (for high-quality attention-free tts),” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). May 2022, IEEE.
  9. “Ts2vec: Towards universal representation of time series,” in Proc. Conf. AAAI, 2022, vol. 36, pp. 8980–8987.
  10. “Learning latent seasonal-trend representations for time series forecasting,” Advances in Neural Inf. Process. Syst., vol. 35, pp. 38775–38787, 2022.
  11. “Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting,” CoRR, vol. abs/2202.01575, 2022.
  12. “Domain adaptation for time series forecasting via attention sharing,” in Proc. Int. Conf. Mach. Learn. PMLR, 2022, pp. 10280–10297.
  13. “Anomaly transformer: Time series anomaly detection with association discrepancy,” CoRR, vol. abs/2110.02642, 2021.
  14. Shun-Zheng Yu, “Hidden semi-markov models,” Artificial intelligence, vol. 174, no. 2, pp. 215–243, 2010.
  15. “Hidden markov model for analyzing time-series health checkup data,” in MEDINFO 2013, pp. 491–495. IOS Press, 2013.
  16. “Causal hidden markov model for time series disease forecasting,” in Proc. IEEE Conf. Comp. Vis. Patt. Recogn., 2021, pp. 12105–12114.
  17. “Brain disease diagnosis using deep learning features from longitudinal mr images,” in Web and Big Data: Second International Joint Conference, APWeb-WAIM 2018, Macau, China, July 23-25, 2018, Proceedings, Part I 2. Springer, 2018, pp. 327–339.
  18. “Longitudinal analysis for alzheimer’s disease diagnosis using rnn,” in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018, pp. 1398–1401.
  19. “Recurrent neural networks for time series forecasting: Current status and future directions,” International Journal of Forecasting, vol. 37, no. 1, pp. 388–427, 2021.
  20. Judea Pearl, Causality, Cambridge university press, 2009.
  21. “Empirical evaluation of gated recurrent neural networks on sequence modeling,” CoRR, vol. abs/1412.3555, 2014.
  22. “Early performance prediction using interpretable patterns in programming process data,” CoRR, vol. abs/2102.05765, 2021.
  23. “The UCR time series archive,” IEEE/CAA Journal of Automatica Sinica, vol. 6, no. 6, pp. 1293–1305, Nov. 2019.
  24. “The UEA multivariate time series classification archive, 2018,” CoRR, vol. abs/1811.00075, 2018.
  25. Dulakshi Santhusitha Kumari Karunasingha, “Root mean square error or mean absolute error? use their ratio as well,” Information Sciences, vol. 585, pp. 609–629, 2022.
  26. “A reinforcement learning-informed pattern mining framework for multivariate time series classification,” in In the Proceeding of 31th International Joint Conference on Artificial Intelligence (IJCAI-22), 2022.
  27. “Relative power of the wilcoxon test, the friedman test, and repeated-measures anova on ranks,” The Journal of Experimental Education, vol. 62, no. 1, pp. 75–86, 1993.
  28. “The use and interpretation of the friedman test in the analysis of ordinal-scale data in repeated measures designs,” Physiotherapy Research International, vol. 1, no. 4, pp. 221–228, 1996.
  29. Robert F Woolson, “Wilcoxon signed-rank test,” Wiley encyclopedia of clinical trials, pp. 1–3, 2007.
  30. Sean R Eddy, “Hidden markov models,” Current opinion in structural biology, vol. 6, no. 3, pp. 361–365, 1996.
  31. “Multivariate time series classification with WEASEL+MUSE,” CoRR, vol. abs/1711.11343, 2017.
  32. “Multivariate lstm-fcns for time series classification,” Neural networks, vol. 116, pp. 237–245, 2019.
  33. “Mrsqm: Fast time series classification with symbolic representations,” CoRR, vol. abs/2109.01036, 2021.
  34. “Tapnet: Multivariate time series classification with attentional prototypical network,” in Proc. Conf. AAAI, 2020, vol. 34, pp. 6845–6852.
  35. “Shapenet: A shapelet-neural network approach for multivariate time series classification,” in Proc. Conf. AAAI, 2021, vol. 35, pp. 8375–8383.
  36. “ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels,” CoRR, vol. abs/1910.13051, 2019.
  37. “MINIROCKET: A very fast (almost) deterministic transform for time series classification,” CoRR, vol. abs/2012.08791, 2020.
  38. “Deep reinforcement learning in large discrete action spaces. arxiv 2015,” arXiv preprint arXiv:1512.07679.
  39. “The uea multivariate time series classification archive, 2018,” arXiv preprint arXiv:1811.00075, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now