Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications

Abstract: To ensure undisrupted business, large Internet companies need to closely monitor various KPIs (e.g., Page Views, number of online users, and number of orders) of its Web applications, to accurately detect anomalies and trigger timely troubleshooting/mitigation. However, anomaly detection for these seasonal KPIs with various patterns and data quality has been a great challenge, especially without labels. In this paper, we proposed Donut, an unsupervised anomaly detection algorithm based on VAE. Thanks to a few of our key techniques, Donut greatly outperforms a state-of-arts supervised ensemble approach and a baseline VAE approach, and its best F-scores range from 0.75 to 0.9 for the studied KPIs from a top global Internet company. We come up with a novel KDE interpretation of reconstruction for Donut, making it the first VAE-based anomaly detection algorithm with solid theoretical explanation.

• The paper introduces Donut, a novel unsupervised anomaly detection algorithm that leverages VAEs to achieve F-scores up to 0.9.
• It presents a breakthrough by applying a modified ELBO with KDE interpretation and MCMC imputation to handle anomalies and missing data.
• Empirical results demonstrate Donut's superior performance in capturing seasonal patterns and temporal gradients in web application KPIs.

Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications

The research paper entitled "Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications" proposes a novel unsupervised anomaly detection algorithm named Donut, designed specifically for monitoring key performance indicators (KPIs) in web applications. The algorithm leverages Variational Auto-Encoders (VAEs) to detect anomalies in time series data characterized by seasonal patterns and local variations.

Core Contributions

The paper primarily focuses on three significant contributions:

1. Introduction of Donut: The authors present Donut, an anomaly detection algorithm that relies on VAE, thus encapsulating the benefits of deep generative models while addressing the challenges of unsupervised anomaly detection in highly dynamic and noisy KPI datasets.
2. Solid Theoretical Foundation: A novel kernel density estimation (KDE) interpretation of reconstruction within the VAE framework is introduced, yielding the first VAE-based anomaly detection algorithm with a concrete theoretical underpinning.
3. Outperforming State-of-the-Art Approaches: The paper shows, through empirical evaluation, that Donut not only surpasses the baseline VAE approach but also outperforms a contemporary supervised ensemble method, EGADS, with superior best F-scores ranging from 0.75 to 0.9 across multiple datasets.

Key Techniques and Approaches

Modified ELBO and Missing Data Injection

One of the notable innovations in Donut is the modification of the evidence lower bound (ELBO), essential to the VAE framework. This modified version selectively excludes contributions from anomalies and missing data during the training process. Alongside this, missing data injection is employed to synthetically introduce missing points during training. This enhances the network’s ability to handle incomplete data and subsequently improves detection robustness.

MCMC Imputation

For the detection phase, Donut uses Markov Chain Monte Carlo (MCMC) sampling to impute missing points, ensuring the model's output is not biased by missing data. This iterative approach is crucial for maintaining the integrity and accuracy of the anomaly scores calculated by the VAE.

Competitive Performance

The algorithm was rigorously tested against alternative methods, such as the VAE baseline and the supervised ensemble approach EGADS. The results were convincing across multiple datasets characterized by different levels of noise and variability. Donut's best F-scores and AUC metrics consistently show its superiority, especially in scenarios with incomplete labels. Even with no labels, Donut's performance remains distinctly high, demonstrating its strength in purely unsupervised environments.

KDE Interpretation and Time Gradient Effect

A fundamental aspect of the paper is its KDE interpretation, which offers a new lens to understand VAE reconstruction in anomaly detection tasks. The KDE interpretation postulates that the VAE reconstructs normal data patterns effectively while treating anomalies as deviations, providing high sensitivity to abnormal behaviors. Moreover, the discovery of the time gradient effect, where similar KPI patterns map closely in the latent space, underlies the continuity and generalization prowess of the Donut model. This time gradient ensures that Donut can maintain high detection accuracy by leveraging the temporal coherence in KPI data.

Implications and Future Work

The implications of this research extend beyond anomaly detection in web applications. The KDE interpretation and the necessity of including both normal and abnormal patterns during the training of generative models could influence broader anomaly detection frameworks and applications. Future research could focus on automatically determining the optimal latent space dimensions (K) and exploring more complex architectures like sequence-to-sequence models to capture longer temporal dependencies in time series data.

Conclusion

The Donut algorithm provides a significant step forward in the field of unsupervised anomaly detection for web KPIs. Its integration of VAE with innovative training and detection techniques allows it to address the inherent challenges of noisy, seasonal data without the need for exhaustive label sets. By offering both solid empirical performance and a robust theoretical foundation, this paper contributes valuable insights and practical methodologies enhancing the reliability and efficiency of web application monitoring systems.