Continuous Memory Representation for Anomaly Detection (2402.18293v3)

Abstract: There have been significant advancements in anomaly detection in an unsupervised manner, where only normal images are available for training. Several recent methods aim to detect anomalies based on a memory, comparing or reconstructing the input with directly stored normal features (or trained features with normal images). However, such memory-based approaches operate on a discrete feature space implemented by the nearest neighbor or attention mechanism, suffering from poor generalization or an identity shortcut issue outputting the same as input, respectively. Furthermore, the majority of existing methods are designed to detect single-class anomalies, resulting in unsatisfactory performance when presented with multiple classes of objects. To tackle all of the above challenges, we propose CRAD, a novel anomaly detection method for representing normal features within a "continuous" memory, enabled by transforming spatial features into coordinates and mapping them to continuous grids. Furthermore, we carefully design the grids tailored for anomaly detection, representing both local and global normal features and fusing them effectively. Our extensive experiments demonstrate that CRAD successfully generalizes the normal features and mitigates the identity shortcut, furthermore, CRAD effectively handles diverse classes in a single model thanks to the high-granularity continuous representation. In an evaluation using the MVTec AD dataset, CRAD significantly outperforms the previous state-of-the-art method by reducing 65.0% of the error for multi-class unified anomaly detection. The project page is available at https://tae-mo.github.io/crad/.

• The paper introduces CRAD, which leverages continuous memory representation to overcome limitations of discrete memory systems in anomaly detection.
• It employs dual continuous memories—local and global—using grid interpolations to transform spatial features for improved detection precision.
• CRAD achieved a 65% error reduction on the MVTec AD dataset, demonstrating robust generalization and computational efficiency in multi-class anomaly settings.

Continuous Memory Representation for Anomaly Detection

The paper "Continuous Memory Representation for Anomaly Detection" introduces a novel approach, CRAD, designed to enhance the capabilities of anomaly detection using continuous memory representation rather than traditional discrete memory systems. The main contribution of the paper lies in addressing several known challenges of memory-based anomaly detection, namely, weak generalization, identity shortcut problems, and the inefficiency of handling multiple classes in a single model.

Technical Approach

CRAD proposes a significant shift from conventional anomaly detection paradigms that leverage discrete memory spaces for storing normal features — often realized through nearest neighbor searches or attention mechanisms. Instead, CRAD utilizes continuous memory representation, situated in grid-based structures. This continuous structure is adept at transforming the spatial features of input data into low-dimensional coordinates, which are then mapped onto continuous feature grids. Such a transformation allows for more efficient retrieval and representation of normal features when encountering potentially anomalous inputs.

The CRAD system is meticulously designed to capture both local and global perspectives of normalcy. This is achieved through two distinct continuous memories: one, focusing on local features (patch-wise) and another on global features (image-wise). These memories utilize unique grid interpolations to ensure effective representation and synthesis of normal features, which significantly enhance anomaly detection performance. Notably, the paper details the use of pixel-wise transformed coordinates to tap into the local grid and feature-wise coordinates for the global grid, thereby capturing various scales of normal attributes.

To counteract false positives that result from potentially misaligned reconstructions of normal features, CRAD integrates a feature refinement step. This refinement uses a combination of mean squared error (MSE) and cosine similarity to ensure consistency between reconstructed and original inputs, especially in normal regions.

Results and Implications

The results of CRAD showcased on the MVTec AD dataset highlight its superiority, particularly in the challenging unified multi-class anomaly detection setting. Here, CRAD reduced the error by 65.0% compared to the former state-of-the-art methods. These results underline not only improved accuracy in detecting anomalies across different classes with a single model but also a marked computational efficiency due to its compact memory design.

Furthermore, the continuous nature of CRAD's memory maps allows for intricate decision boundaries within complex, multi-class feature spaces, a previously arduous task for discrete memory-based models. The CRAD model maintains strong performance even when only limited normal data are available, demonstrating robust generalization capabilities. These facets hold potential for broad implications, suggesting that continuous memory representation can become a foundational component in building more effective and efficient anomaly detection systems.

Future Directions

The paper acknowledges the limitation CRAD faces with extremely limited data (e.g., 1-shot or zero-shot scenarios). Subsequently, it opens avenues for further exploration into enhancing the adaptability and efficiency of continuous memory systems in such sparse data conditions. Addressing these challenges could pave the way for more agile, real-time anomaly detection frameworks applicable across varied domains including, but not limited to, manufacturing, surveillance, and medical imaging.

In summation, CRAD introduces a robust and efficient alternative to traditional anomaly detection systems, showcasing the value of continuous memory representation in improving anomaly detection accuracy and efficiency. As the research community extends this concept, it will likely contribute to significant advancements in real-time anomaly detection applications and beyond.