A Unifying Review of Deep and Shallow Anomaly Detection (2009.11732v3)

Abstract: Deep learning approaches to anomaly detection have recently improved the state of the art in detection performance on complex datasets such as large collections of images or text. These results have sparked a renewed interest in the anomaly detection problem and led to the introduction of a great variety of new methods. With the emergence of numerous such methods, including approaches based on generative models, one-class classification, and reconstruction, there is a growing need to bring methods of this field into a systematic and unified perspective. In this review we aim to identify the common underlying principles as well as the assumptions that are often made implicitly by various methods. In particular, we draw connections between classic 'shallow' and novel deep approaches and show how this relation might cross-fertilize or extend both directions. We further provide an empirical assessment of major existing methods that is enriched by the use of recent explainability techniques, and present specific worked-through examples together with practical advice. Finally, we outline critical open challenges and identify specific paths for future research in anomaly detection.

• The paper introduces a unified framework that bridges deep and shallow anomaly detection, showing that deep models excel in high-dimensional data while shallow methods perform well in simpler settings.
• It systematically categorizes techniques into probabilistic models, one-class classification, and reconstruction models, clarifying their theoretical foundations and practical trade-offs.
• The review emphasizes adaptive method selection based on dataset characteristics and highlights promising directions in robust, semi-supervised, and explainable anomaly detection.

A Review of Deep and Shallow Anomaly Detection: Bridging Two Worlds

Introduction

The paper "A Unifying Review of Deep and Shallow Anomaly Detection" offers a comprehensive review of the field of anomaly detection, addressing both deep learning and traditional, or shallow, methods. The authors aim to provide a systematic perspective that harmonizes these approaches, thereby enhancing the understanding and development of anomaly detection techniques.

Core Contributions

The authors identify a pressing need for a unified perspective due to the diverse array of methods emerging in anomaly detection. Significant strides in deep learning have led to advanced methods capable of handling complex datasets, such as images and text. However, classical methods remain robust and effective in many contexts. The paper delineates the theoretical connections between these two spheres, promoting cross-fertilization of ideas.

Methodology Overview

The review systematically categorizes anomaly detection techniques into three primary paradigms:

1. Probabilistic Models: These include classic density estimation methods like Gaussian Mixture Models and modern techniques such as Energy-Based Models, Variational Autoencoders (VAEs), and Normalizing Flows. The emphasis is on estimating the probability distribution of normal data and identifying anomalies as low probability events.
2. One-Class Classification: This involves learning a decision boundary that separates normal data from potential anomalies, typically utilizing Support Vector Data Description (SVDD) and its deep learning extensions. These methods focus on optimizing classification risk with limited anomaly samples.
3. Reconstruction Models: These models, including Autoencoders and their deep variants, focus on reconstructing input data. Anomalies are detected based on reconstruction errors, assuming normal data can be better reconstructed than anomalous data.

Strong Results and Claims

The authors provide empirical evaluations of various methods, demonstrating heterogeneous performance across different datasets. They emphasize the importance of adaptively choosing methods based on specific dataset characteristics and anomaly types. The paper presents strong results showing that deep models can outperform shallow ones when dealing with high-dimensional, complex data, but shallow models remain effective in low-dimensional scenarios.

Theoretical and Practical Implications

• Unified Framework: By proposing a unifying framework, the paper enables systematic exploration of new algorithmic combinations. This framework provides insights for transferring successful strategies from one domain (deep learning) to another (shallow learning), and vice versa.
• Performance Evaluation: Thorough evaluation strategies and the use of explainability techniques highlight the necessity of transparency in anomaly detection models. This approach aids in identifying potential model failures, such as overfitting to noise or exhibiting biased decision boundaries.
• Future Research Directions: The paper shines light on potential areas of exploration, such as robust learning under high-dimensional noise, semi-supervised anomaly detection, and the integration of anomaly detection with related fields like open set recognition and out-of-distribution detection.

Conclusion

This review extensively covers the landscape of anomaly detection, bridging the gap between deep and shallow techniques. It provides a holistic view that not only uncovers theoretical insights but also proposes practical directives for advancing anomaly detection research. By fostering a unified perspective, the authors open avenues for innovative methodologies that leverage the strengths of both deep and shallow approaches. This work undoubtedly serves as a vital resource for researchers aiming to develop more robust and effective anomaly detection systems.