MixedTeacher : Knowledge Distillation for fast inference textural anomaly detection (2306.09859v1)
Abstract: For a very long time, unsupervised learning for anomaly detection has been at the heart of image processing research and a stepping stone for high performance industrial automation process. With the emergence of CNN, several methods have been proposed such as Autoencoders, GAN, deep feature extraction, etc. In this paper, we propose a new method based on the promising concept of knowledge distillation which consists of training a network (the student) on normal samples while considering the output of a larger pretrained network (the teacher). The main contributions of this paper are twofold: First, a reduced student architecture with optimal layer selection is proposed, then a new Student-Teacher architecture with network bias reduction combining two teachers is proposed in order to jointly enhance the performance of anomaly detection and its localization accuracy. The proposed texture anomaly detector has an outstanding capability to detect defects in any texture and a fast inference time compared to the SOTA methods.
- Falko Kähler, Ole Schmedemann and Thorsten Schüppstuhl “Anomaly detection for industrial surface inspection: application in maintenance of aircraft components” In Procedia CIRP 107, 2022, pp. 246–251 DOI: 10.1016/j.procir.2022.05.197
- Manpreet Singh Minhas and John Zelek “AnoNet: Weakly Supervised Anomaly Detection in Textured Surfaces” arXiv, 2019 arXiv: http://arxiv.org/abs/1911.10608
- “Unsupervised Industrial Anomaly Detection via Pattern Generative and Contrastive Networks” In arXiv:2207.09792 [cs], 2022
- “Analysis of network traffic features for anomaly detection” In Machine Learning 101.1, 2015, pp. 59–84 DOI: 10.1007/s10994-014-5473-9
- “Student-Teacher Feature Pyramid Matching for Anomaly Detection” In arXiv:2103.04257 [cs], 2021 arXiv: http://arxiv.org/abs/2103.04257
- “Deep Residual Learning for Image Recognition” arXiv, 2015 arXiv: http://arxiv.org/abs/1512.03385
- Mingxing Tan and Quoc V. Le “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks” arXiv, 2020 arXiv: http://arxiv.org/abs/1905.11946
- “VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization” In 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), 2021, pp. 01–06 DOI: 10.1109/ISIE45552.2021.9576231
- Shuang Mei, Yudan Wang and Guojun Wen “Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model” In Sensors 18.4, 2018, pp. 1064 DOI: 10.3390/s18041064
- “GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection” In arXiv:1903.06661 [cs, stat], 2019 arXiv: http://arxiv.org/abs/1903.06661
- “Generative Adversarial Networks” In arXiv:1406.2661 [cs, stat], 2014 arXiv: http://arxiv.org/abs/1406.2661
- “f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks” In Medical Image Analysis 54, 2019, pp. 30–44 DOI: 10.1016/j.media.2019.01.010
- “G2D: Generate to Detect Anomaly” event-place: Waikoloa, HI, USA In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) IEEE, 2021, pp. 2002–2011 DOI: 10.1109/WACV48630.2021.00205
- “Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection” In arXiv:2203.00259 [cs], 2022 arXiv: http://arxiv.org/abs/2203.00259
- Marco Rudolph, Bastian Wandt and Bodo Rosenhahn “Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows” event-place: Waikoloa, HI, USA In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) IEEE, 2021, pp. 1906–1915 DOI: 10.1109/WACV48630.2021.00195
- “Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection” In arXiv:2110.02855 [cs], 2021 arXiv: http://arxiv.org/abs/2110.02855
- “FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows” In arXiv:2111.07677 [cs], 2021 arXiv: http://arxiv.org/abs/2111.07677
- “Reconstruction Student with Attention for Student-Teacher Pyramid Matching” In arXiv:2111.15376 [cs], 2022
- “Anomaly Detection via Reverse Distillation from One-Class Embedding” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 9737-9746, 2022
- Sungwook Lee, Seunghyun Lee and Byung Cheol Song “CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localization” arXiv, 2022 arXiv: http://arxiv.org/abs/2206.04325
- “MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection” In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Long Beach, CA, USA: IEEE, 2019, pp. 9584–9592 DOI: 10.1109/CVPR.2019.00982
- “Towards Total Recall in Industrial Anomaly Detection” In arXiv:2106.08265 [cs], 2021 arXiv: http://arxiv.org/abs/2106.08265
- “CutPaste: Self-Supervised Learning for Anomaly Detection and Localization” In arXiv:2104.04015 [cs], 2021 arXiv: http://arxiv.org/abs/2104.04015
- Simon Thomine (4 papers)
- Hichem Snoussi (11 papers)
- Mahmoud Soua (2 papers)