Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection (2209.12148v2)

Abstract: Anomaly detection has recently gained increasing attention in the field of computer vision, likely due to its broad set of applications ranging from product fault detection on industrial production lines and impending event detection in video surveillance to finding lesions in medical scans. Regardless of the domain, anomaly detection is typically framed as a one-class classification task, where the learning is conducted on normal examples only. An entire family of successful anomaly detection methods is based on learning to reconstruct masked normal inputs (e.g. patches, future frames, etc.) and exerting the magnitude of the reconstruction error as an indicator for the abnormality level. Unlike other reconstruction-based methods, we present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. The proposed self-supervised block is extremely flexible, enabling information masking at any layer of a neural network and being compatible with a wide range of neural architectures. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss. Furthermore, we show that our block is applicable to a wider variety of tasks, adding anomaly detection in medical images and thermal videos to the previously considered tasks based on RGB images and surveillance videos. We exhibit the generality and flexibility of SSMCTB by integrating it into multiple state-of-the-art neural models for anomaly detection, bringing forth empirical results that confirm considerable performance improvements on five benchmarks. We release our code and data as open source at: https://github.com/ristea/ssmctb.

Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection

The paper "Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection" presents a novel approach to anomaly detection through the integration of a Self-Supervised Masked Convolutional Transformer Block (SSMCTB) into existing neural network architectures. Anomaly detection is framed as a one-class classification task, with the primary objective being to learn the typical characteristics of normal data and identify deviations as anomalies. This is particularly challenging across various domains, such as industrial quality assessment, video surveillance, and healthcare, where the definition of "normal" significantly varies based on context.

Key Contributions

1. Novel Integrated Block: The authors propose the SSMCTB, which combines a masked convolutional layer and a transformer module to predict masked information within the receptive field, promoting the network's capability to model long-range dependencies. This block is self-supervised through a custom reconstruction loss based on Huber loss, which helps in managing outliers effectively.
2. Flexibility and Compatibility: The block exhibits high flexibility, allowing for integration at any layer within a network, and compatibility with a diverse range of neural architectures, including both CNNs and transformers. This adaptability enables broad application to various tasks and signifies potential utility beyond anomaly detection.
3. Improved Anomaly Detection: By introducing SSMCTB to state-of-the-art frameworks, the authors report substantial improvements across multiple benchmarks such as MVTec AD, BRATS, and video anomaly detection datasets like Avenue and ShanghaiTech. The block's ability to focus on masked regions and its channel-wise attention mechanism markedly enhance anomaly detection performance.

Experimental and Numerical Results

The paper details empirical evaluations showcasing improvements in anomaly detection tasks. For instance, substantial performance increments are observed when SSMCTB is integrated with established methods such as DRAEM and NSA for image anomaly detection, and with frameworks by Liu et al. and Georgescu et al. for video anomaly detection. These improvements are particularly notable on standard datasets, resulting in leading-edge performance. For instance, state-of-the-art scores are attained on the Avenue dataset for multiple metrics, including micro and macro AUC.

Implications and Future Directions

From a theoretical lens, SSMCTB's architecture indicates advancements in understanding and leveraging self-supervised learning within the context of anomaly detection. The fusion of masked convolution with transformer-based attention modules exemplifies an innovative solution for capturing contextual information and reconstructing missing data in anomaly detection.

Practically, this work suggests significant ramifications for enhancing anomaly detection systems, notably in fields requiring fine-grain detection like manufacturing defect analysis, and medical imaging where patient safety is critical.

Looking forward, SSMCTB's structural integration paves the way for future explorations in applying similar self-supervised approaches across different domains, potentially extending to self-supervised pre-training tasks beyond anomaly detection. Moreover, the flexibility in block integration within the architectures posits SSMCTB as a reusable component across various application domains, offering a broader contribution to the landscape of computer vision tasks.

In summation, the introduction of SSMCTB addresses key challenges in anomaly detection, providing both a robust improvement in detection capabilities and a versatile tool for further exploration in related tasks. The efficiency and adaptability of the proposed block underscore its potential as a valuable asset within advanced neural network architectures.