Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LARA: A Light and Anti-overfitting Retraining Approach for Unsupervised Time Series Anomaly Detection (2310.05668v4)

Published 9 Oct 2023 in cs.LG

Abstract: Most of current anomaly detection models assume that the normal pattern remains same all the time. However, the normal patterns of Web services change dramatically and frequently. The model trained on old-distribution data is outdated after such changes. Retraining the whole model every time is expensive. Besides, at the beginning of normal pattern changes, there is not enough observation data from the new distribution. Retraining a large neural network model with limited data is vulnerable to overfitting. Thus, we propose a Light and Anti-overfitting Retraining Approach (LARA) for deep variational auto-encoder based time series anomaly detection methods (VAEs). This work aims to make three novel contributions: 1) the retraining process is formulated as a convex problem and can converge at a fast rate as well as prevent overfitting; 2) designing a ruminate block, which leverages the historical data without the need to store them; 3) mathematically proving that when fine-tuning the latent vector and reconstructed data, the linear formations can achieve the least adjusting errors between the ground truths and the fine-tuned ones. Moreover, we have performed many experiments to verify that retraining LARA with even 43 time slots of data from new distribution can result in its competitive F1 Score in comparison with the state-of-the-art anomaly detection models trained with sufficient data. Besides, we verify its light overhead.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Vicente Alarcon-Aquino and Javier A Barria. 2001. Anomaly detection in communication networks using wavelets. IEE Proceedings-Communications 148, 6 (2001), 355–362.
  2. Convex optimization. Cambridge university press.
  3. Anomaly Detection Under Distribution Shift. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 6511–6523.
  4. Hierarchical particle swarm optimization-incorporated latent factor analysis for large-scale incomplete matrices. IEEE Transactions on Big Data 8, 6 (2021), 1524–1536.
  5. Deep variational graph convolutional recurrent network for multivariate time series anomaly detection. In International Conference on Machine Learning. PMLR, 3621–3633.
  6. Crowdsourcing service-level network event monitoring. In Proceedings of the ACM SIGCOMM 2010 Conference. 387–398.
  7. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning. PMLR, 1126–1135.
  8. RobustTAD: Robust time series anomaly detection via decomposition and convolutional neural networks. arXiv preprint arXiv:2002.09545 (2020).
  9. Dendritic neuron model with effective learning algorithms for classification, approximation, and prediction. IEEE transactions on neural networks and learning systems 30, 2 (2018), 601–614.
  10. Your classifier is secretly an energy based model and you should treat it like one. In 8th International Conference on Learning Representations, ICLR 2020.
  11. Robust random cut forest based anomaly detection on streams. In International conference on machine learning. PMLR, 2712–2721.
  12. Dan Hendrycks and Kevin Gimpel. 2017. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In 5th International Conference on Learning Representations, ICLR 2017.
  13. CyCADA: Cycle-Consistent Adversarial Domain Adaptation. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018 (Proceedings of Machine Learning Research, Vol. 80). PMLR, 1994–2003.
  14. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 387–395.
  15. Anomaly detection in time series with robust variational quasi-recurrent autoencoders. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 1342–1354.
  16. Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Yoshua Bengio and Yann LeCun (Eds.).
  17. Transfer anomaly detection by inferring latent domain representations. Advances in neural information processing systems 32 (2019).
  18. A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018. 7167–7177.
  19. Learning robust deep state space for unsupervised anomaly detection in contaminated time-series. IEEE Transactions on Knowledge and Data Engineering (2022).
  20. Prototype-oriented unsupervised anomaly detection for multivariate time series. In International Conference on Machine Learning. PMLR, 19407–19424.
  21. Conditional adversarial domain adaptation. Advances in neural information processing systems 31 (2018).
  22. Unsupervised domain adaptation with residual transfer networks. Advances in neural information processing systems 29 (2016).
  23. An effective scheme for QoS estimation via alternating direction method-based matrix factorization. IEEE Transactions on Services Computing 12, 4 (2016), 503–518.
  24. Generalized Nesterov’s Acceleration-Incorporated, Non-Negative and Adaptive Latent Factor Analysis. IEEE Transactions on Services Computing 15, 5 (2021), 2809–2823.
  25. Minghua Ma and Shenglin Zhang. 2021. Jump-starting multivariate time series anomaly detection for online service systems. In Proceedings of the 2021 USENIX Annual Technical Conference.
  26. Nicholas Metropolis and Stanislaw Ulam. 1949. The monte carlo method. Journal of the American statistical association 44, 247 (1949), 335–341.
  27. Thomas P Minka. 2013. Expectation propagation for approximate Bayesian inference. arXiv preprint arXiv:1301.2294 (2013).
  28. Unified Deep Supervised Domain Adaptation and Generalization. In IEEE International Conference on Computer Vision, ICCV 2017. IEEE Computer Society, 5716–5726.
  29. Joseph Ndong and Kavé Salamatian. 2011. Signal processing-based anomaly detection techniques: a comparative analysis. In Proc. 2011 3rd International Conference on Evolving Internet. 32–39.
  30. Deep learning for anomaly detection: A review. ACM computing surveys (CSUR) 54, 2 (2021), 1–38.
  31. LeSiNN: Detecting anomalies by identifying least similar nearest neighbours. In 2015 IEEE international conference on data mining workshop (ICDMW). IEEE, 623–630.
  32. Deep One-Class Classification. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018 (Proceedings of Machine Learning Research, Vol. 80). 4390–4399.
  33. Rethinking Assumptions in Deep Anomaly Detection. CoRR abs/2006.00339 (2020).
  34. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3723–3732.
  35. Timeseries anomaly detection using temporal hierarchical one-class network. Advances in Neural Information Processing Systems 33 (2020), 13016–13026.
  36. Time series anomaly detection with multiresolution ensemble decoding. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 9567–9575.
  37. Anomaly detection in streams with extreme value theory. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 1067–1075.
  38. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2828–2837.
  39. Learning competitive and discriminative reconstructions for anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5167–5174.
  40. Ryan Tibshirani. Fall 2013. Lecture 6 of Convex Optimization (Carnegie Mellon University 10-725). (Fall 2013).
  41. Tranad: Deep transformer networks for anomaly detection in multivariate time series data. arXiv preprint arXiv:2201.07284 (2022).
  42. Dorothy Walsh. 1979. Occam’s razor: A principle of intellectual elegance. American Philosophical Quarterly 16, 3 (1979), 241–244.
  43. Felix Wiewel and Bin Yang. 2019. Continual learning for anomaly detection with variational autoencoder. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3837–3841.
  44. Learning unsupervised metaformer for anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4369–4378.
  45. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. In Proceedings of the 2018 world wide web conference. 187–196.
  46. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022.
  47. DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection. arXiv preprint arXiv:2306.10347 (2023).
  48. Energy-efficient and QoS-optimized adaptive task scheduling and management in clouds. IEEE Transactions on Automation Science and Engineering 19, 2 (2020), 1233–1244.
  49. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 1409–1416.
  50. Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects. arXiv preprint arXiv:2306.10125 (2023).
  51. Predicting quality of services based on a two-stream deep learning model with user and service graphs. IEEE Transactions on Services Computing (2023).
  52. CloudRCA: A root cause analysis framework for cloud computing platforms. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 4373–4382.
  53. Automatic and generic periodicity adaptation for kpi anomaly detection. IEEE Transactions on Network and Service Management 16, 3 (2019), 1170–1183.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now