Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PePNet: A Periodicity-Perceived Workload Prediction Network Supporting Rare Occurrence of Heavy Workload (2308.01917v2)

Published 11 Jul 2023 in cs.DC and cs.LG

Abstract: Cloud providers can greatly benefit from accurate workload prediction. However, the workload of cloud servers is highly variable, with occasional heavy workload bursts. This makes workload prediction challenging. There are mainly two categories of workload prediction methods: statistical methods and neural-network-based ones. The former ones rely on strong mathematical assumptions and have reported low accuracy when predicting highly variable workload. The latter ones offer higher overall accuracy, yet they are vulnerable to data imbalance between heavy workload and common one. This impairs the prediction accuracy of neural network-based models on heavy workload. Either the overall inaccuracy of statistic methods or the heavy-workload inaccuracy of neural-network-based models can cause service level agreement violations. Thus, we propose PePNet to improve overall especially heavy workload prediction accuracy. It has two distinctive characteristics: (i) A Periodicity-Perceived Mechanism to detect the existence of periodicity and the length of one period automatically, without any priori knowledge. Furthermore, it fuses periodic information adaptively, which is suitable for periodic, lax periodic and aperiodic time series. (ii) An Achilles' Heel Loss Function iteratively optimizing the most under-fitting part in predicting sequence for each step, which significantly improves the prediction accuracy of heavy load. Extensive experiments conducted on Alibaba2018, SMD dataset and Dinda's dataset demonstrate that PePNet improves MAPE for overall workload by 20.0% on average, compared with state-of-the-art methods. Especially, PePNet improves MAPE for heavy workload by 23.9% on average.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. M. Niknafs, I. Ukhov, P. Eles, and Z. Peng, “Runtime resource management with workload prediction,” in Annual Design Automation Conference 2019, DAC 2019, 2019, pp. 1–6.
  2. M. Babaioff, Y. Mansour, N. Nisan, G. Noti, C. Curino, N. Ganapathy, I. Menache, O. Reingold, M. Tennenholtz, and E. Timnat, “Era: A framework for economic resource allocation for the cloud,” in International Conference on World Wide Web Companion, WWW 2017, 2017, p. 635–642.
  3. J. Gao, H. Wang, and H. Shen, “Machine learning based workload prediction in cloud computing,” in International Conference on Computer Communications and Networks, ICCCN 2020, 2020, pp. 1–9.
  4. H. Moussa, I. Yen, F. B. Bastani, Y. Dong, and W. He, “Toward better service performance management via workload prediction,” in Services Computing - SCC 2019, ser. Lecture Notes in Computer Science, vol. 11515, 2019, pp. 92–106.
  5. Z. Chen, J. Hu, G. Min, A. Y. Zomaya, and T. A. El-Ghazawi, “Towards accurate prediction for high-dimensional and highly-variable cloud workloads with deep learning,” IEEE Trans. Parallel Distributed Syst., vol. 31, no. 4, pp. 923–934, 2020.
  6. H. Aragon, S. Braganza, E. Boza, J. Parrales, and C. Abad, “Workload characterization of a software-as-a-service web application implemented with a microservices architecture,” in Companion Proceedings of The 2019 World Wide Web Conference, WWW 2019, 2019, p. 746–750.
  7. R. N. Calheiros, E. Masoumi, R. Ranjan, and R. Buyya, “Workload prediction using ARIMA model and its impact on cloud applications’ qos,” IEEE Trans. Cloud Comput., vol. 3, no. 4, pp. 449–458, 2015.
  8. Y. Jiang, M. Shahrad, D. Wentzlaff, D. H. K. Tsang, and C. Joe-Wong, “Burstable instances for clouds: Performance modeling, equilibrium analysis, and revenue maximization,” IEEE/ACM Trans. Netw., vol. 28, no. 6, pp. 2489–2502, 2020.
  9. D. Ding, M. Zhang, X. Pan, M. Yang, and X. He, “Modeling extreme events in time series prediction,” in ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, 2019, pp. 1114–1122.
  10. Y. Zhu, W. Zhang, Y. Chen, and H. Gao, “A novel approach to workload prediction using attention-based LSTM encoder-decoder network in cloud environment,” EURASIP J. Wirel. Commun. Netw., vol. 2019, pp. 1–18, 2019.
  11. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Conference on Artificial Intelligence, AAAI 2021, 2021, pp. 11 106–11 115.
  12. P. A. Dinda, “The statistical properties of host load,” Sci. Program., vol. 7, no. 3-4, pp. 211–229, 1999.
  13. S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan, “Attention based spatial-temporal graph convolutional networks for traffic flow forecasting,” in Conference on Artificial Intelligence, AAAI 2019, 2019, pp. 922–929.
  14. C. Chen, K. Li, S. G. Teo, X. Zou, K. Wang, J. Wang, and Z. Zeng, “Gated residual recurrent graph neural networks for traffic prediction,” in Conference on Artificial Intelligence, AAAI 2019, 2019, pp. 485–492.
  15. Z. Lv, J. Xu, K. Zheng, H. Yin, P. Zhao, and X. Zhou, “LC-RNN: A deep learning model for traffic speed prediction,” in Proceedings of International Joint Conference on Artificial Intelligence, IJCAI 2018, 2018, pp. 3470–3476.
  16. H. Yao, X. Tang, H. Wei, G. Zheng, and Z. Li, “Revisiting spatial-temporal similarity: A deep learning framework for traffic prediction,” in Conference on Artificial Intelligence, AAAI 2019, 2019, pp. 5668–5675.
  17. D. Ding, M. Zhang, X. Pan, M. Yang, and X. He, “Modeling extreme events in time series prediction,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 1114–1122.
  18. K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using rnn encoder-decoder for statistical machine translation,” arXiv preprint arXiv:1406.1078, 2014.
  19. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  20. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  21. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, NeurIPS 2017, vol. 30, 2017.
  22. J. Masci, U. Meier, D. Cireşan, and J. Schmidhuber, “Stacked convolutional auto-encoders for hierarchical feature extraction,” in Artificial Neural Networks and Machine Learning, ICANN 2011.   Springer Berlin Heidelberg, 2011, pp. 52–59.
  23. H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,” in Advances in Neural Information Processing Systems, NeurIPS 2021, vol. 34, 2021, pp. 22 419–22 430.
  24. N. Kitaev, L. Kaiser, and A. Levskaya, “Reformer: The efficient transformer,” in International Conference on Learning Representations, ICLR 2020, 2020.
  25. T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting,” in International Conference on Machine Learning, ICML 2022, vol. 162, 2022, pp. 27 268–27 286.
  26. Y. Su, Y. Zhao, C. Niu, R. Liu, W. Sun, and D. Pei, “Robust anomaly detection for multivariate time series through stochastic recurrent neural network,” in ACM SIGKDD international conference on knowledge discovery & data mining, KDD 2019, 2019, pp. 2828–2837.
  27. Q. Hu, P. Sun, S. Yan, Y. Wen, and T. Zhang, “Characterization and prediction of deep learning workloads in large-scale GPU datacenters,” in The International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’21, 2021, pp. 104:1–104:15.
  28. S. Xue, C. Qu, X. Shi, C. Liao, S. Zhu, X. Tan, L. Ma, S. Wang, S. Wang, Y. Hu et al., “A meta reinforcement learning approach for predictive autoscaling in the cloud,” in ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2022, 2022, pp. 4290–4299.
  29. Y. Zhang, W. Fan, X. Wu, H. Chen, B. Li, and M. Zhang, “CAFE: adaptive VDI workload prediction with multi-grained features,” in Artificial Intelligence, AAAI 2019, 2019, pp. 5821–5828.
  30. A. De Santo, A. Galli, M. Gravina, V. Moscato, and G. Sperlì, “Deep learning for hdd health assessment: An application based on lstm,” IEEE Transactions on Computers, vol. 71, no. 1, pp. 69–80, 2020.
  31. L. Ma, D. Van Aken, A. Hefny, G. Mezerhane, A. Pavlo, and G. J. Gordon, “Query-based workload forecasting for self-driving database management systems,” in Proceedings of the 2018 International Conference on Management of Data, 2018, pp. 631–645.
  32. H. Chen, R. A. Rossi, K. Mahadik, S. Kim, and H. Eldardiry, “Graph deep factors for forecasting with applications to cloud resource allocation,” in ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD 2021, 2021, pp. 106–116.
  33. C. Luo, B. Qiao, X. Chen, P. Zhao, R. Yao, H. Zhang, W. Wu, A. Zhou, and Q. Lin, “Intelligent virtual machine provisioning in cloud computing,” in International Joint Conference on Artificial Intelligence, IJCAI 2020, 2020, pp. 1495–1502.
  34. Y. Yao, D. Li, H. Jie, H. Jie, T. Li, J. Chen, J. Wang, F. Li, and Y. Gao, “Simplets: An efficient and universal model selection framework for time series forecasting,” Proceedings of the VLDB Endowment, vol. 16, no. 12, pp. 3741–3753, 2023.
  35. M. Stockman, M. Awad, H. Akkary, and R. Khanna, “Thermal status and workload prediction using support vector regression,” in International Conference on Energy Aware Computing, 2012, pp. 1–5.
  36. P. Singh, P. Gupta, and K. Jyoti, “TASM: technocrat ARIMA and SVR model for workload prediction of web applications in cloud,” Clust. Comput., vol. 22, no. 2, pp. 619–633, 2019.
  37. W. Zhang, B. Li, D. Zhao, F. Gong, and Q. Lu, “Workload prediction for cloud cluster using a recurrent neural network,” in International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI), 2016, pp. 104–109.
  38. J. Kumar, R. Goomer, and A. K. Singh, “Long short term memory recurrent neural network (lstm-rnn) based workload forecasting model for cloud datacenters,” Procedia Computer Science, vol. 125, pp. 676–682, 2018.
  39. A. Bauer, M. Züfle, N. Herbst, S. Kounev, and V. Curtef, “Telescope: An automatic feature extraction and transformation approach for time series forecasting on a level-playing field,” in 2020 IEEE 36th International Conference on Data Engineering (ICDE), 2020, pp. 1902–1905.
  40. Y. Liang, S. Ke, J. Zhang, X. Yi, and Y. Zheng, “Geoman: Multi-level attention networks for geo-sensory time series prediction.” in International conference on international joint conferences on artificial intelligence, IJCAI 2018, 2018, pp. 3428–3434.
  41. W. Li, R. Bao, K. Harimoto, D. Chen, J. Xu, and Q. Su, “Modeling the stock relation with graph network for overnight stock movement prediction,” in International conference on international joint conferences on artificial intelligence, IJCAI 2021, 2021, pp. 4541–4547.
  42. R.-G. Cirstea, T. Kieu, C. Guo, B. Yang, and S. J. Pan, “Enhancenet: Plugin neural networks for enhancing correlated time series forecasting,” in 2021 IEEE 37th International Conference on Data Engineering (ICDE), 2021, pp. 1739–1750.
  43. Y. Cui, K. Zheng, D. Cui, J. Xie, L. Deng, F. Huang, and X. Zhou, “Metro: a generic graph neural network framework for multivariate time series forecasting,” Proceedings of the VLDB Endowment, vol. 15, no. 2, pp. 224–236, 2021.
  44. X. Wu, D. Zhang, C. Guo, C. He, B. Yang, and C. S. Jensen, “Autocts: Automated correlated time series forecasting,” Proceedings of the VLDB Endowment, vol. 15, no. 4, pp. 971–983, 2021.
  45. T. Belkhouja, Y. Yan, and J. R. Doppa, “Training robust deep models for time-series domain: Novel algorithms and theoretical analysis,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 6, 2022, pp. 6055–6063.
  46. Q. Tan, M. Ye, B. Yang, S. Liu, A. J. Ma, T. C.-F. Yip, G. L.-H. Wong, and P. Yuen, “Data-gru: Dual-attention time-aware gated recurrent unit for irregular multivariate time series,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, 2020, pp. 930–937.
  47. X. Zhang, Y. Gao, J. Lin, and C.-T. Lu, “Tapnet: Multivariate time series classification with attentional prototypical network,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, 2020, pp. 6845–6852.
  48. X. Tang, H. Yao, Y. Sun, C. Aggarwal, P. Mitra, and S. Wang, “Joint modeling of local and global temporal dynamics for multivariate time series forecasting with missing values,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, 2020, pp. 5956–5963.
  49. Y. Luo, Y. Zhang, X. Cai, and X. Yuan, “E2gan: End-to-end generative adversarial network for multivariate time series imputation,” in Proceedings of the 28th international joint conference on artificial intelligence.   AAAI Press, 2019, pp. 3094–3100.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Feiyi Chen (10 papers)
  2. Zhen Qin (105 papers)
  3. Hailiang Zhao (16 papers)
  4. Shuiguang Deng (45 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets