DROCC: Deep Robust One-Class Classification (2002.12718v2)

Abstract: Classical approaches for one-class problems such as one-class SVM and isolation forest require careful feature engineering when applied to structured domains like images. State-of-the-art methods aim to leverage deep learning to learn appropriate features via two main approaches. The first approach based on predicting transformations (Golan & El-Yaniv, 2018; Hendrycks et al., 2019a) while successful in some domains, crucially depends on an appropriate domain-specific set of transformations that are hard to obtain in general. The second approach of minimizing a classical one-class loss on the learned final layer representations, e.g., DeepSVDD (Ruff et al., 2018) suffers from the fundamental drawback of representation collapse. In this work, we propose Deep Robust One-Class Classification (DROCC) that is both applicable to most standard domains without requiring any side-information and robust to representation collapse. DROCC is based on the assumption that the points from the class of interest lie on a well-sampled, locally linear low dimensional manifold. Empirical evaluation demonstrates that DROCC is highly effective in two different one-class problem settings and on a range of real-world datasets across different domains: tabular data, images (CIFAR and ImageNet), audio, and time-series, offering up to 20% increase in accuracy over the state-of-the-art in anomaly detection. Code is available at https://github.com/microsoft/EdgeML.

• The paper introduces a novel deep learning approach that leverages a low-dimensional manifold assumption to generate reliable synthetic anomalies.
• The paper mitigates representation collapse with a discriminative classifier, achieving up to a 20% accuracy boost over state-of-the-art methods.
• The paper demonstrates DROCC's versatility across various domains, including images, audio, and time series, with significant improvements in false positive rates.

Deep Robust One-Class Classification: A Summary

The paper presents a novel approach to one-class classification, specifically addressing the challenges of anomaly detection using deep learning methods. Traditional techniques such as one-class SVMs and isolation forests require extensive feature engineering, particularly in complex domains like images or audio. This research introduces Deep Robust One-Class Classification (DROCC), which leverages the assumption that data points from the class of interest reside on a well-sampled, locally linear low-dimensional manifold. DROCC effectively addresses the representation collapse issue often encountered in existing deep learning methods for one-class classification.

Key Contributions

1. Manifold Assumption: DROCC is predicated on the hypothesis that normal data points lie on a low-dimensional manifold, which can be well-represented using straightforward Euclidean distances locally. This assumption allows for the generation of synthetic anomalies more reliably than existing methodologies.
2. Robust to Representation Collapse: Unlike DeepSVDD and similar methods, DROCC mitigates the risk of representation collapse by employing a discriminative classifier that prevents degenerate solutions.
3. Applicability Across Domains: The method is tested on a variety of real-world datasets including tabular data, images (CIFAR-10, ImageNet), audio, and time series data, demonstrating its versatility.
4. Empirical Results: Experiments show that DROCC achieves up to a 20% increase in accuracy over state-of-the-art methods in anomaly detection across diverse datasets.
5. One-Class Classification with Limited Negatives (OCLN): The paper extends DROCC to scenarios where negative instances are limited but crucial, focusing on low false positive rates (FPR). Significant improvements in recall were observed when evaluated under controlled FPR conditions.

Implications

DROCC offers a robust, generalizable approach to one-class classification tasks without relying on domain-specific transformations or extensive feature engineering. Its ability to maintain accuracy across various datasets and domains marks a significant step forward in the practical application of anomaly detection. Furthermore, the exploration of the OCLN problem sets a new precedent for handling limited negative datasets in realistic settings, such as audio wake-word detection.

Future Directions

Future work could involve exploring stricter optimization methods to accommodate more complex data manifolds, enhancing computational efficiency, and further refining the approach to handle high-dimensional, less-curated datasets. Additionally, the integration of DROCC into real-world systems will likely spur advancements in applications requiring precise anomaly detection and classification with constrained negatives.

In summary, DROCC represents a potent addition to the toolbox for deep learning-based anomaly detection, bridging gaps left by previous methods and paving the way for more sophisticated and adaptable strategies in handling one-class classification tasks.