AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model (2312.05767v2)

Abstract: Anomaly inspection plays an important role in industrial manufacture. Existing anomaly inspection methods are limited in their performance due to insufficient anomaly data. Although anomaly generation methods have been proposed to augment the anomaly data, they either suffer from poor generation authenticity or inaccurate alignment between the generated anomalies and masks. To address the above problems, we propose AnomalyDiffusion, a novel diffusion-based few-shot anomaly generation model, which utilizes the strong prior information of latent diffusion model learned from large-scale dataset to enhance the generation authenticity under few-shot training data. Firstly, we propose Spatial Anomaly Embedding, which consists of a learnable anomaly embedding and a spatial embedding encoded from an anomaly mask, disentangling the anomaly information into anomaly appearance and location information. Moreover, to improve the alignment between the generated anomalies and the anomaly masks, we introduce a novel Adaptive Attention Re-weighting Mechanism. Based on the disparities between the generated anomaly image and normal sample, it dynamically guides the model to focus more on the areas with less noticeable generated anomalies, enabling generation of accurately-matched anomalous image-mask pairs. Extensive experiments demonstrate that our model significantly outperforms the state-of-the-art methods in generation authenticity and diversity, and effectively improves the performance of downstream anomaly inspection tasks. The code and data are available in https://github.com/sjtuplayer/anomalydiffusion.

AnomalyDiffusion: A Diffusion-Based Approach to Anomaly Image Generation

The paper entitled "AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model" proposes a novel method in the landscape of industrial anomaly inspection, particularly targeting the limitations faced by existing methods due to the scarcity of anomaly data. AnomalyDiffusion integrates a few-shot learning paradigm with a diffusion model to enhance anomaly data generation, aiming to improve downstream tasks such as anomaly detection, localization, and classification.

Methodological Innovations

At the core of AnomalyDiffusion is the use of a Latent Diffusion Model (LDM), which is enriched with prior information from a large-scale dataset, specifically LAION, to improve anomaly generation authenticity when only a few anomalous samples are available. This approach leverages two novel mechanisms: Spatial Anomaly Embedding and an Adaptive Attention Re-weighting Mechanism.

1. Spatial Anomaly Embedding: This technique disentangles anomaly information into two facets—appearance and location—via a learnable anomaly embedding and a spatial embedding, respectively. The anomaly embedding captures appearance specifics, while the spatial embedding encodes the location derived from an anomaly mask. This separation facilitates more precise control over the generated anomalies' type and location.
2. Adaptive Attention Re-weighting Mechanism: To align the generated anomalies more accurately with their masks, this mechanism adjusts the model's focus dynamically during the generation process. It emphasizes areas with less noticeable anomalies, thus improving spatial alignment in generated images.

These components collectively enable the production of highly authentic and diverse anomalous image-mask pairs using minimal data, a significant step forward over existing anomaly generation techniques like GAN-based models, which typically require extensive anomaly datasets.

Experimental Results

The paper reports substantial experimental results across various metrics. AnomalyDiffusion outperforms state-of-the-art methods in generation authenticity and diversity, as measured by Inception Score (IS) and Intra-cluster pairwise LPIPS distance (IC-LPIPS), showcasing its capability in generating high-quality anomaly data. The generated data's positive impact on anomaly detection, localization, and classification tasks is quantitatively demonstrated with pixel-level AUROC achieving 99.1% and an AP score of 81.4% on the MVTec dataset. This clearly indicates the strong potential of AnomalyDiffusion to enhance practical industrial anomaly inspection systems.

Implications and Future Directions

The implications of AnomalyDiffusion extend beyond mere improvements in generation quality. The disentanglement of spatial and appearance information in anomaly embeddings offers a new avenue for exploring controlled generation tasks, adding versatility to the application of diffusion models in industrial scenarios. Furthermore, the robust alignment between generated anomalies and their masks provided by the adaptive attention re-weighting mechanism can inspire novel approaches in generating context-accurate synthetic data for other domains.

Future studies could explore enhancing anomaly resolution and investigating other types of diffusion models to further elevate the quality of generated anomalies. Additionally, integrating this approach with real-time anomaly detection systems could be explored to assess its utility in dynamic environments.

In conclusion, AnomalyDiffusion introduces a significant advancement in the generation of few-shot anomaly data using diffusion models. By improving the accuracy and authenticity of generated anomalies, this method holds promising potential for advancing the domain of industrial anomaly inspection, with broad implications for related fields requiring anomaly detection and classification capabilities.