Learning and Evaluating Representations for Deep One-class Classification (2011.02578v2)

Abstract: We present a two-stage framework for deep one-class classification. We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations. The framework not only allows to learn better representations, but also permits building one-class classifiers that are faithful to the target task. We argue that classifiers inspired by the statistical perspective in generative or discriminative models are more effective than existing approaches, such as a normality score from a surrogate classifier. We thoroughly evaluate different self-supervised representation learning algorithms under the proposed framework for one-class classification. Moreover, we present a novel distribution-augmented contrastive learning that extends training distributions via data augmentation to obstruct the uniformity of contrastive representations. In experiments, we demonstrate state-of-the-art performance on visual domain one-class classification benchmarks, including novelty and anomaly detection. Finally, we present visual explanations, confirming that the decision-making process of deep one-class classifiers is intuitive to humans. The code is available at https://github.com/google-research/deep_representation_one_class.

• The paper proposes a two-stage framework where self-supervised representation learning is used to construct efficient one-class classifiers.
• It evaluates various self-supervised techniques, including novel distribution-augmented contrastive learning, to significantly boost performance.
• Experiments on benchmarks like CIFAR-10/100 and Fashion MNIST demonstrate superior accuracy and improved interpretability, affirming the framework's practical impact.

Overview of "Learning and Evaluating Representations for Deep One-class Classification"

The paper presents a robust two-stage framework aimed at enhancing deep one-class classification effectiveness. Initially, the focus is on learning self-supervised representations from one-class data. Following this, one-class classifiers are constructed based on the learned data representations. This paradigm is advantageous for improving representation quality and developing classifiers consonant with the intended classification task.

Key Contributions

• Two-Stage Framework: The first stage encompasses learning representations, facilitating unsupervised and self-supervised methodologies. The second stage involves deploying shallow one-class classifiers using these representations. This bifurcation allows for a straightforward, efficient approach to integrating state-of-the-art representation learning algorithms.
• Evaluation of Self-Supervised Techniques: The paper offers a comprehensive review of various self-supervised techniques, such as contrastive learning and rotation prediction, within the context of one-class classification. It proposes a novel distribution-augmented contrastive learning that effectively uses data augmentation methods to enhance representation learning.
• Performance Analysis: The framework demonstrated superior performance across visual domain benchmarks like CIFAR-10/100 and Fashion MNIST, establishing its practicality by outperforming contemporary methods based on surrogate classifiers.
• Visual Explanation Integration: The paper underscores the importance of understanding decision-making processes within one-class classifiers. It introduces an approachable method for visual explanations based on gradient analysis, facilitating interpretability for end-users.

Numerical Results and Claims

The paper records impressive numerical performance gains across tested benchmarks, achieving state-of-the-art results. Notably, the adoption of distribution-augmented contrastive learning notably improved performance metrics. This suggests the proposed methodology not only surpasses existing frameworks but does so with substantial measurable gains.

Implications and Speculation on Future Developments

Practically, the proposed two-stage framework simplifies the integration of advanced representation learning into the deployment of effective classifiers. Theoretically, it provides a bedrock for evolving more intricate model architectures that retain simplicity in fundamental application yet boast efficiency improvements.

Looking ahead, this research opens new avenues in anomaly detection across varied domains — from manufacturing defect identification to fraud detection. As AI technologies advance, integrating continually enhanced self-supervised learning techniques could address more complex patterns of anomaly detection, thereby increasing applicability in surveillance and health monitoring systems.

Furthermore, exploring the intersection between one-class classification and transfer learning could lead to insightful developments on leveraging pre-trained models in resource-constrained settings. Lastly, expanding the framework to accommodate real-time data augmentation processing may yield profound impacts in real-world applications.

In summary, the paper offers a structured approach toward advancing deep one-class classification through a bifurcated methodology that capitalizes on the strengths of self-supervised learning, paired with innovative augmentative techniques and thorough performance validation.