Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Diffusion Modeling for Anomaly Detection (2305.18593v2)

Published 29 May 2023 in cs.LG and cs.AI

Abstract: Known for their impressive performance in generative modeling, diffusion models are attractive candidates for density-based anomaly detection. This paper investigates different variations of diffusion modeling for unsupervised and semi-supervised anomaly detection. In particular, we find that Denoising Diffusion Probability Models (DDPM) are performant on anomaly detection benchmarks yet computationally expensive. By simplifying DDPM in application to anomaly detection, we are naturally led to an alternative approach called Diffusion Time Estimation (DTE). DTE estimates the distribution over diffusion time for a given input and uses the mode or mean of this distribution as the anomaly score. We derive an analytical form for this density and leverage a deep neural network to improve inference efficiency. Through empirical evaluations on the ADBench benchmark, we demonstrate that all diffusion-based anomaly detection methods perform competitively for both semi-supervised and unsupervised settings. Notably, DTE achieves orders of magnitude faster inference time than DDPM, while outperforming it on this benchmark. These results establish diffusion-based anomaly detection as a scalable alternative to traditional methods and recent deep-learning techniques for standard unsupervised and semi-supervised anomaly detection settings.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. A survey of network anomaly detection techniques. Journal of Network and Computer Applications, 60:19–31, 2016a.
  2. A survey of anomaly detection techniques in financial domain. Future Generation Computer Systems, 55:278–288, 2016b.
  3. Variational autoencoder based anomaly detection using reconstruction probability. 2015.
  4. Classification-based anomaly detection for general data. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=H1lK_lBtvS.
  5. Lof: Identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, SIGMOD ’00, pp.  93–104, New York, NY, USA, 2000. Association for Computing Machinery. ISBN 1581132174. doi: 10.1145/342009.335388. URL https://doi.org/10.1145/342009.335388.
  6. Anomaly detection: A survey. ACM Comput. Surv., 41(3), jul 2009. ISSN 0360-0300. doi: 10.1145/1541880.1541882. URL https://doi.org/10.1145/1541880.1541882.
  7. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019. URL https://api.semanticscholar.org/CorpusID:52967399.
  8. High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Pattern Recognition, 58:121–134, 2016. ISSN 0031-3203. doi: https://doi.org/10.1016/j.patcog.2016.03.028. URL https://www.sciencedirect.com/science/article/pii/S0031320316300267.
  9. C. Fauconnier and Gentiane Haesbroeck. Outliers detection with the minimum covariance determinant estimator in practice. Statistical Methodology, 6:363–379, 07 2009. doi: 10.1016/j.stamet.2008.12.005.
  10. Challenges for unsupervised anomaly detection in particle physics. Journal of High Energy Physics, 2022(3), mar 2022. doi: 10.1007/jhep03(2022)066. URL https://doi.org/10.1007%2Fjhep03%282022%29066.
  11. Histogram-based outlier score (hbos): A fast unsupervised anomaly detection algorithm. 09 2012.
  12. Lunar: Unifying local outlier detection methods via graph neural networks. 2022.
  13. Revisiting deep learning models for tabular data. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=i_Q1yrOegLY.
  14. Drocc: Deep robust one-class classification. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org, 2020.
  15. Denoising diffusion models for out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.  2947–2956, June 2023.
  16. ADBench: Anomaly detection benchmark. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL https://openreview.net/forum?id=foA_SFQ9zo0.
  17. Douglas M Hawkins. Identification of outliers, volume 11. Springer, 1980.
  18. Deep residual learning for image recognition, 2015. URL https://arxiv.org/abs/1512.03385.
  19. Discovering cluster-based local outliers. Pattern Recogn. Lett., 24(9–10):1641–1650, jun 2003. ISSN 0167-8655. doi: 10.1016/S0167-8655(03)00003-5. URL https://doi.org/10.1016/S0167-8655(03)00003-5.
  20. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  21. A survey of outlier detection methodologies. Artificial Intelligence Review, 22:85–126, 10 2004. doi: 10.1023/B:AIRE.0000045502.10941.a9.
  22. Tabddpm: Modelling tabular data with diffusion models. arXiv preprint arXiv:2209.15421, 2022.
  23. Feature bagging for outlier detection. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, KDD ’05, pp.  157–166, New York, NY, USA, 2005. Association for Computing Machinery. ISBN 159593135X. doi: 10.1145/1081870.1081891. URL https://doi.org/10.1145/1081870.1081891.
  24. COPOD: Copula-based outlier detection. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, nov 2020. doi: 10.1109/icdm50108.2020.00135. URL https://doi.org/10.1109%2Ficdm50108.2020.00135.
  25. ECOD: Unsupervised outlier detection using empirical cumulative distribution functions. IEEE Transactions on Knowledge and Data Engineering, pp. 1–1, 2022. doi: 10.1109/tkde.2022.3159580. URL https://doi.org/10.1109%2Ftkde.2022.3159580.
  26. Isolation forest. In 2008 Eighth IEEE International Conference on Data Mining, pp.  413–422, 2008. doi: 10.1109/ICDM.2008.17.
  27. Generative adversarial active learning for unsupervised outlier detection, 2019.
  28. Unsupervised out-of-distribution detection with diffusion inpainting. In International Conference on Machine Learning, 2023. URL https://api.semanticscholar.org/CorpusID:257050245.
  29. J. B. MacQueen. Some methods for classification and analysis of multivariate observations. In L. M. Le Cam and J. Neyman (eds.), Proc. of the fifth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pp.  281–297. University of California Press, 1967.
  30. Anomaly detection in medical wireless sensor networks using machine learning algorithms. Procedia Computer Science, 70:325–333, 2015. ISSN 1877-0509. doi: https://doi.org/10.1016/j.procs.2015.10.026. URL https://www.sciencedirect.com/science/article/pii/S1877050915031907. Proceedings of the 4th International Conference on Eco-friendly Computing and Communication Systems.
  31. Deep learning for anomaly detection. ACM Computing Surveys, 54(2):1–38, mar 2021. doi: 10.1145/3439950. URL https://doi.org/10.1145%2F3439950.
  32. Tomáš Pevný. Loda: Lightweight on-line detector of anomalies. Mach. Learn., 102(2):275–304, feb 2016. ISSN 0885-6125. doi: 10.1007/s10994-015-5521-0. URL https://doi.org/10.1007/s10994-015-5521-0.
  33. Efficient algorithms for mining outliers from large data sets. SIGMOD Rec., 29(2):427–438, may 2000. ISSN 0163-5808. doi: 10.1145/335191.335437. URL https://doi.org/10.1145/335191.335437.
  34. Variational inference with normalizing flows. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp.  1530–1538, Lille, France, 07–09 Jul 2015. PMLR. URL https://proceedings.mlr.press/v37/rezende15.html.
  35. Deep one-class classification. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.  4393–4402. PMLR, 10–15 Jul 2018. URL https://proceedings.mlr.press/v80/ruff18a.html.
  36. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE, 109(5):756–795, may 2021. doi: 10.1109/jproc.2021.3052449. URL https://doi.org/10.1109%2Fjproc.2021.3052449.
  37. Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis, MLSDA’14, pp.  4–11, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 9781450331593. doi: 10.1145/2689746.2689747. URL https://doi.org/10.1145/2689746.2689747.
  38. Sensor fault and patient anomaly detection and classification in medical wireless sensor networks. In 2013 IEEE International Conference on Communications (ICC), pp.  4373–4378, 2013. doi: 10.1109/ICC.2013.6655254.
  39. Support vector method for novelty detection. volume 12, pp.  582–588, 01 1999.
  40. Anomaly detection for tabular data with internal contrastive learning. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=_hszZbt46bT.
  41. A novel anomaly detection scheme based on principal component classifier. 01 2003.
  42. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp. 2256–2265. PMLR, 2015.
  43. Anomaly detection approaches for semiconductor manufacturing. Procedia Manufacturing, 11:2018–2024, 2017.
  44. Diffusion models for medical anomaly detection. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VIII, pp.  35–45. Springer, 2022.
  45. Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.  649–655, 2022. doi: 10.1109/CVPRW56347.2022.00080.
  46. Learning discriminative reconstructions for unsupervised outlier removal. 2015 IEEE International Conference on Computer Vision (ICCV), pp.  1511–1519, 2015.
  47. Telemetry-mining: a machine learning approach to anomaly detection and fault diagnosis for space systems. In 2nd IEEE International Conference on Space Mission Challenges for Information Technology (SMC-IT’06), pp.  8 pp.–476, 2006. doi: 10.1109/SMC-IT.2006.79.
  48. Diffusionad: Denoising diffusion for anomaly detection, 2023.
  49. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, pp.  665–674, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450348874. doi: 10.1145/3097983.3098052. URL https://doi.org/10.1145/3097983.3098052.
  50. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations, 2018.
  51. Spot-the-difference self-supervised pre-training for anomaly detection and segmentation. In European Conference on Computer Vision, pp.  392–408. Springer, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Victor Livernoche (1 paper)
  2. Vineet Jain (6 papers)
  3. Yashar Hezaveh (45 papers)
  4. Siamak Ravanbakhsh (52 papers)
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets