Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A SAM-guided Two-stream Lightweight Model for Anomaly Detection (2402.19145v2)

Published 29 Feb 2024 in cs.CV

Abstract: In industrial anomaly detection, model efficiency and mobile-friendliness become the primary concerns in real-world applications. Simultaneously, the impressive generalization capabilities of Segment Anything (SAM) have garnered broad academic attention, making it an ideal choice for localizing unseen anomalies and diverse real-world patterns. In this paper, considering these two critical factors, we propose a SAM-guided Two-stream Lightweight Model for unsupervised anomaly detection (STLM) that not only aligns with the two practical application requirements but also harnesses the robust generalization capabilities of SAM. We employ two lightweight image encoders, i.e., our two-stream lightweight module, guided by SAM's knowledge. To be specific, one stream is trained to generate discriminative and general feature representations in both normal and anomalous regions, while the other stream reconstructs the same images without anomalies, which effectively enhances the differentiation of two-stream representations when facing anomalous regions. Furthermore, we employ a shared mask decoder and a feature aggregation module to generate anomaly maps. Our experiments conducted on MVTec AD benchmark show that STLM, with about 16M parameters and achieving an inference time in 20ms, competes effectively with state-of-the-art methods in terms of performance, 98.26% on pixel-level AUC and 94.92% on PRO. We further experiment on more difficult datasets, e.g., VisA and DAGM, to demonstrate the effectiveness and generalizability of STLM.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. A robust variational autoencoder using beta divergence. Knowledge-Based Systems (2022), 107886. https://doi.org/10.1016/j.knosys.2021.107886
  2. Douglas G Altman and J Martin Bland. 1994. Statistics Notes: Diagnostic tests 2: predictive values. British Medical Journal (1994).
  3. MVTec AD–A comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 9592–9600. https://doi.org/10.1109/cvpr.2019.00982
  4. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 4183–4192. https://doi.org/10.1109/cvpr42600.2020.00424
  5. Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders. In Proceedings of the 14th. International Conference on Computer Vision Theory and Applications. IEEE, 372–380. https://doi.org/10.5220/0007364503720380
  6. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence (2017), 834–848. https://doi.org/10.1109/tpami.2017.2699184
  7. Robust Anomaly Detection and Localization via Simulated Anomalies. In Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. ACM Press, 1–8. https://doi.org/10.1145/3574131.3574463
  8. Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches. Proceedings of the 28th ACM International Conference on Multimedia. https://doi.org/10.1145/3394171.3413813
  9. Describing textures in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 3606–3613. https://doi.org/10.1109/cvpr.2014.461
  10. Anne-Sophie Collin and Christophe De Vleeschouwer. 2021. Improved anomaly detection by training an autoencoder with skip connections on images corrupted with stain-shaped noise. In Proceedings of the 25th International Conference on Pattern Recognition. IEEE, 7915–7922.
  11. Hanqiu Deng and Xingyu Li. 2022. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 9737–9746. https://doi.org/10.1109/cvpr52688.2022.00951
  12. Terrance DeVries and Graham W Taylor. 2017. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017).
  13. Deep Learning for Medical Anomaly Detection – A Survey. Comput. Surveys (2021), 37. https://doi.org/10.1145/3464423
  14. Unsupervised Representation Learning by Predicting Image Rotations. In Proceedings of the International Conference on Learning Representations. OpenReview.net, 8330–8339.
  15. Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 1440–1448. https://doi.org/10.1109/iccv.2015.169
  16. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 1705–1714.
  17. Generative adversarial nets. Advances in Neural Information Processing Systems (2014). https://doi.org/10.3156/JSOFT.29.5_177_2
  18. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE, 98–107. https://doi.org/10.1109/wacv51458.2022.00188
  19. An Efficient Motion Detection and Tracking Scheme for Encrypted Surveillance Videos. ACM Transactions on MultimediaComputing Communications and Applications (2017), 1–23. https://doi.org/10.1145/3131342
  20. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).
  21. Segment anything is not always perfect: An investigation of sam on different real-world applications. arXiv preprint arXiv:2304.05750 (2023).
  22. SANFlow: Semantic-Aware Normalizing Flow for Anomaly Detection. Advances in Neural Information Processing Systems 36 (2024).
  23. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  24. Segment anything. arXiv preprint arXiv:2304.02643 (2023).
  25. Cfa: Coupled-hypersphere-based feature adaptation for target-oriented anomaly localization. IEEE Access (2022). https://doi.org/10.1109/access.2022.3193699
  26. Cutpaste: Self-supervised learning for anomaly detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 9664–9674. https://doi.org/10.1109/cvpr46437.2021.00954
  27. Rethinking Vision Transformers for MobileNet Size and Speed. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 16889–16900. https://doi.org/10.1109/iccv51070.2023.01549
  28. 3D Tensor Auto-encoder with Application to Video Compression. ACM Transactions on MultimediaComputing Communications and Applications (2021), 1–18. https://doi.org/10.1145/3431768
  29. Focal loss for dense object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 2980–2988. https://doi.org/10.1109/iccv.2017.324
  30. Kun Liu and Huadong Ma. 2019. Exploring Background-bias for Anomaly Detection in Surveillance Videos. In Proceedings of the 27th ACM International Conference on Multimedia. ACM Press, 1490–1499. https://doi.org/10.1145/3343031.3350998
  31. Towards Visually Explaining Variational Autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 8642–8651.
  32. Simplenet: A simple network for image anomaly detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 20402–20411. https://doi.org/10.1109/cvpr52729.2023.01954
  33. Learning memory-guided normality for anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. IEEE, 14372–14381.
  34. Yuxin Peng and Jinwei Qi. 2019. CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning. ACM Transactions on MultimediaComputing Communications and Applications (2019), 1–24. https://doi.org/10.1145/3284750
  35. Efficiently scaling transformer inference. Proceedings of Machine Learning and Systems (2023).
  36. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. PMLR, 8748–8763.
  37. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 14318–14328. https://doi.org/10.1109/cvpr52688.2022.01392
  38. Takaya Saito and Marc Rehmsmeier. 2015. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE (2015), 1–21. https://doi.org/10.1371/journal.pone.0118432
  39. Multiresolution knowledge distillation for anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 14902–14912. https://doi.org/10.1109/cvpr46437.2021.01466
  40. Natasa Sarafijanovic-Djukic and Jesse Davis. 2019. Fast distance-based anomaly detection in images using an inception-like autoencoder. In International Conference on Discovery Science. Springer-Verlag, 493–508. https://doi.org/10.1007/978-3-030-33778-0_37
  41. Natural synthetic anomalies for self-supervised anomaly detection and localization. In Proceedings of the European Conference on Computer Vision. Springer-Verlag, 474–489.
  42. Unsupervised anomaly segmentation via deep feature reconstruction. Neurocomputing (2021), 9–22. https://doi.org/10.1016/j.neucom.2020.11.018
  43. Deep learning for unsupervised anomaly localization in industrial images: A survey. IEEE Transactions on Instrumentation and Measurement (2022). https://doi.org/10.1109/tim.2022.3196436
  44. Revisiting reverse distillation for anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 24511–24520. https://doi.org/10.1109/cvpr52729.2023.02348
  45. q-Space novelty detection with variational autoencoders. In Computational Diffusion MRI. Springer-Verlag, 113–124. https://doi.org/10.1007/978-3-030-52893-5_10
  46. Student-teacher feature pyramid matching for anomaly detection. arXiv preprint arXiv:2103.04257 (2021).
  47. Matthias Wieler and Tobias Hahn. 2007. Weakly supervised learning for industrial optical inspection. In DAGM symposium in.
  48. Tinyvit: Fast pretraining distillation for small vision transformers. In Proceedings of the European Conference on Computer Vision. Springer-Verlag, 68–85. https://doi.org/10.1007/978-3-031-19803-8_5
  49. Diverse Image Captioning via Conditional Variational Autoencoder and Dual Contrastive Learning. ACM Transactions on MultimediaComputing Communications and Applications (2023), 1–16. https://doi.org/10.1145/3614435
  50. MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities. Engineering Applications Of Artificial Intelligence (2023).
  51. Focus the Discrepancy: Intra-and Inter-Correlation Learning for Image Anomaly Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 6803–6813. https://doi.org/10.1109/iccv51070.2023.00626
  52. Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 24490–24499.
  53. Fastflow: Unsupervised anomaly detection and localization via 2d normalizing flows. arXiv preprint arXiv:2111.07677 (2021).
  54. Draem-a discriminatively trained reconstruction embedding for surface anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 8330–8339. https://doi.org/10.1109/iccv48922.2021.00822
  55. Reconstruction by inpainting for visual anomaly detection. Pattern Recognition (2021), 107706. https://doi.org/10.1016/j.patcog.2020.107706
  56. Faster Segment Anything: Towards Lightweight SAM for Mobile Applications. arXiv preprint arXiv:2306.14289 (2023).
  57. Prototypical residual networks for anomaly detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 16281–16291. https://doi.org/10.1109/cvpr52729.2023.01562
  58. Unsupervised Surface Anomaly Detection with Diffusion Probabilistic Model. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 6782–6791. https://doi.org/10.1109/iccv51070.2023.00624
  59. DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 3914–3923. https://doi.org/10.1109/cvpr52729.2023.00381
  60. Cross-CBAM: A Lightweight network for Scene Segmentation. arXiv preprint arXiv:2306.02306 (2023).
  61. Decoupled knowledge distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 11953–11962. https://doi.org/10.1109/cvpr52688.2022.01165
  62. Chong Zhou and Randy C. Paffenroth. 2017. Anomaly Detection with Robust Deep Autoencoders. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM Press, 665–674. https://doi.org/10.1145/3097983.3098052
  63. Spot-the-difference self-supervised pre-training for anomaly detection and segmentation. In Proceedings of the European Conference on Computer Vision. Springer-Verlag, 392–408. https://doi.org/10.1007/978-3-031-20056-4_23
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets