OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations (1903.08550v1)

Abstract: We present a novel model called OCGAN for the classical problem of one-class novelty detection, where, given a set of examples from a particular class, the goal is to determine if a query example is from the same class. Our solution is based on learning latent representations of in-class examples using a denoising auto-encoder network. The key contribution of our work is our proposal to explicitly constrain the latent space to exclusively represent the given class. In order to accomplish this goal, firstly, we force the latent space to have bounded support by introducing a tanh activation in the encoder's output layer. Secondly, using a discriminator in the latent space that is trained adversarially, we ensure that encoded representations of in-class examples resemble uniform random samples drawn from the same bounded space. Thirdly, using a second adversarial discriminator in the input space, we ensure all randomly drawn latent samples generate examples that look real. Finally, we introduce a gradient-descent based sampling technique that explores points in the latent space that generate potential out-of-class examples, which are fed back to the network to further train it to generate in-class examples from those points. The effectiveness of the proposed method is measured across four publicly available datasets using two one-class novelty detection protocols where we achieve state-of-the-art results.

• The paper introduces a constrained latent representation approach within GANs to enhance one-class novelty detection.
• It employs latent and visual discriminators along with informative-negative mining to align in-class distributions effectively.
• Empirical results on MNIST, fMNIST, COIL100, and CIFAR10 demonstrate its state-of-the-art performance in diverse, challenging settings.

OCGAN: One-Class Novelty Detection Using GANs with Constrained Latent Representations

The paper presents a model named OCGAN, specifically developed for the task of one-class novelty detection. This task involves identifying whether a query sample belongs to the same class as a given training dataset, which only contains examples from one class.

Methodology Overview

OCGAN leverages Generative Adversarial Networks (GANs) and denoising auto-encoders to create a robust latent space tailored to represent solely the in-class examples. The novelty of OCGAN lies in its approach to explicitly constrain this latent space, thus ensuring that representations correspond strictly to the in-class distribution:

1. Latent Space Constriction: The encoder uses a tanh activation function to bound the latent space, which helps in maintaining a compact representation.
2. Latent Discriminator: This component enforces a uniform distribution of the in-class latent representations over the bounded support, using adversarial training to align them with uniformly random samples.
3. Visual Discriminator: By employing a secondary discriminator in the input space, the model ensures that any random sample from the latent space generates outputs that resemble real samples of the in-class distribution.
4. Informative-Negative Mining: A gradient-descent-based technique is introduced to explore latent regions that potentially generate out-of-class samples. These are then fed back to the system to further fine-tune the model, ensuring greater alignment with in-class examples.

Experimental Results

The efficacy of OCGAN is demonstrated through experiments across four datasets: MNIST, fMNIST, COIL100, and CIFAR10. Two evaluation protocols are employed to benchmark its performance, and OCGAN consistently achieves state-of-the-art results.

• MNIST and fMNIST: OCGAN improves upon existing methods by up to 3\% using protocol 1 and shows consistent superiority, especially when the in-class representation is complex.
• COIL100: With minimal intra-class variance, OCGAN obtains an AUC of 0.995, significantly outperforming baselines.
• CIFAR10: Despite its complexity, OCGAN's performance is comparable to the previous best result, showcasing robustness even in challenging environments.

The ablation paper provides insights into the impact of each component, with individual contributions from the latent and visual discriminators significantly improving performance.

Implications and Future Work

OCGAN addresses a fundamental limitation in one-class novelty detection by ensuring that even diverse in-class examples restrict the latent space representations effectively. This ensures that out-of-class samples are poorly represented, reducing false negatives significantly.

The implications of this work extend to many domains requiring anomaly detection, such as cybersecurity and biomedical data analysis. By effectively managing the latent space, OCGAN offers an enhanced ability to discriminate against novel inputs accurately.

Future research might focus on adapting OCGAN for natural image datasets that include complex structures or leveraging its architecture for temporal data, such as video streams, to detect novel events over time. The model’s adaptability and precision promise advancements in various real-world applications where novelty detection is crucial.

In summary, OCGAN presents an innovative approach to a classic machine learning problem, offering nuanced solutions that are theoretically sound and empirically validated.