Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Targeted collapse regularized autoencoder for anomaly detection: black hole at the center (2306.12627v2)

Published 22 Jun 2023 in cs.LG, cs.AI, cs.CV, q-bio.NC, and stat.ML

Abstract: Autoencoders have been extensively used in the development of recent anomaly detection techniques. The premise of their application is based on the notion that after training the autoencoder on normal training data, anomalous inputs will exhibit a significant reconstruction error. Consequently, this enables a clear differentiation between normal and anomalous samples. In practice, however, it is observed that autoencoders can generalize beyond the normal class and achieve a small reconstruction error on some of the anomalous samples. To improve the performance, various techniques propose additional components and more sophisticated training procedures. In this work, we propose a remarkably straightforward alternative: instead of adding neural network components, involved computations, and cumbersome training, we complement the reconstruction loss with a computationally light term that regulates the norm of representations in the latent space. The simplicity of our approach minimizes the requirement for hyperparameter tuning and customization for new applications which, paired with its permissive data modality constraint, enhances the potential for successful adoption across a broad range of applications. We test the method on various visual and tabular benchmarks and demonstrate that the technique matches and frequently outperforms more complex alternatives. We further demonstrate that implementing this idea in the context of state-of-the-art methods can further improve their performance. We also provide a theoretical analysis and numerical simulations that help demonstrate the underlying process that unfolds during training and how it helps with anomaly detection. This mitigates the black-box nature of autoencoder-based anomaly detection algorithms and offers an avenue for further investigation of advantages, fail cases, and potential new directions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. A survey of network anomaly detection techniques. Journal of Network and Computer Applications, 60:19–31, 2016.
  2. Ganomaly: Semi-supervised anomaly detection via adversarial training. In Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III 14, pages 622–637. Springer, 2019.
  3. Deep learning approach combining sparse autoencoder with svm for network intrusion detection. Ieee Access, 6:52843–52856, 2018.
  4. What regularized auto-encoders learn from the data-generating distribution. The Journal of Machine Learning Research, 15(1):3563–3593, 2014.
  5. Network anomaly detection with stochastically improved autoencoder based models. In 2017 IEEE 4th international conference on cyber security and cloud computing (CSCloud), pages 193–198. IEEE, 2017.
  6. Greedy layer-wise training of deep networks. Advances in neural information processing systems, 19, 2006.
  7. Classification-based anomaly detection for general data. arXiv preprint arXiv:2005.02359, 2020.
  8. Hierarchical rnn-based framework for throughput prediction in automotive production systems. International Journal of Production Research, pages 1–16, 2023.
  9. Entity embedding-based anomaly detection for heterogeneous categorical events. arXiv preprint arXiv:1608.07502, 2016.
  10. Adaptive transfer learning for multimode process monitoring and unsupervised anomaly detection in steam turbines. Reliability Engineering & System Safety, 234:109162, 2023.
  11. Automated anomaly detection and performance modeling of enterprise applications. ACM Transactions on Computer Systems (TOCS), 27(3):1–32, 2009.
  12. Anomaly detection with generative adversarial networks. arXiv preprint arXiv:1809.04758, 2018.
  13. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  14. Adversarial feature learning. arXiv preprint arXiv:1605.09782, 2016.
  15. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  16. Uci machine learning repository. 2017.
  17. Adversarially learned inference. arXiv preprint arXiv:1606.00704, 2016.
  18. A deep learning ensemble for network anomaly and cyber-attack detection. Sensors, 20(16):4583, 2020.
  19. Defective wafer detection using a denoising autoencoder for semiconductor manufacturing processes. Advanced Engineering Informatics, 46:101166, 2020.
  20. Deep learning for medical anomaly detection–a survey. ACM Computing Surveys (CSUR), 54(7):1–37, 2021.
  21. Autoencoders for unsupervised anomaly detection in high energy physics. Journal of High Energy Physics, 2021(6):1–32, 2021.
  22. Tsmae: a novel anomaly detection approach for internet of things time series data using memory-augmented autoencoder. IEEE Transactions on network science and engineering, 2022.
  23. Deep anomaly detection using geometric transformations. Advances in neural information processing systems, 31, 2018.
  24. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1705–1714, 2019.
  25. Deep learning for visual understanding: A review. Neurocomputing, 187:27–48, 2016.
  26. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 297–304. JMLR Workshop and Conference Proceedings, 2010.
  27. Madgan: Unsupervised medical anomaly detection gan using multiple adjacent brain mri slice reconstruction. BMC bioinformatics, 22(2):1–20, 2021.
  28. Financial fraud: A review of anomaly detection techniques and recent advances. 2022.
  29. Reducing the dimensionality of data with neural networks. science, 313(5786):504–507, 2006.
  30. Attribute restoration framework for anomaly detection. arXiv preprint arXiv:1911.10676, 2019.
  31. Performance anomaly detection and bottleneck identification. ACM Computing Surveys (CSUR), 48(1):1–35, 2015.
  32. Denoising autoencoders for unsupervised anomaly detection in brain mri. In International Conference on Medical Imaging with Deep Learning, pages 653–664. PMLR, 2022.
  33. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  34. Learning multiple layers of features from tiny images. 2009.
  35. Mnist handwritten digit database, 2010.
  36. A sparse autoencoder-based unsupervised scheme for pump fault detection and isolation. Applied Sciences, 10(19):6789, 2020.
  37. Isolation forest. In 2008 eighth ieee international conference on data mining, pages 413–422. IEEE, 2008.
  38. Deep contractive autoencoder-based anomaly detection for in-vehicle controller area network (can). Progress in Engineering Technology: Automotive, Energy Generation, Quality Control and Efficiency, pages 195–205, 2019.
  39. K-sparse autoencoders. arXiv preprint arXiv:1312.5663, 2013.
  40. Winner-take-all autoencoders. Advances in neural information processing systems, 28, 2015.
  41. Fake it until you make it: Towards accurate near-distribution novelty detection. In The Eleventh International Conference on Learning Representations, 2022.
  42. Medhini G Narasimhan. Dynamic video anomaly detection and localization using sparse denoising autoencoders. Multimedia Tools and Applications, 77:13173–13195, 2018.
  43. Enhanced network anomaly detection based on deep neural networks. IEEE access, 6:48231–48246, 2018.
  44. Gee: A gradient-based explainable variational autoencoder for network anomaly detection. In 2019 IEEE Conference on Communications and Network Security (CNS), pages 91–99. IEEE, 2019.
  45. Deep weakly-supervised anomaly detection. arXiv preprint arXiv:1910.13601, 2019.
  46. Deep anomaly detection with deviation networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 353–362, 2019.
  47. Emanuel Parzen. On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3):1065–1076, 1962.
  48. Ocgan: One-class novelty detection using gans with constrained latent representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2898–2906, 2019.
  49. Elad Plaut. From principal subspaces to principal components with linear autoencoders. arXiv preprint arXiv:1804.10253, 2018.
  50. Fraud detection: A systematic literature review of graph-based anomaly detection approaches. Decision Support Systems, 133:113303, 2020.
  51. A data mining approach for fault diagnosis: An application of anomaly detection algorithm. Measurement, 55:343–352, 2014.
  52. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedings of the 28th international conference on international conference on machine learning, pages 833–840, 2011.
  53. Deep one-class classification. In International conference on machine learning, pages 4393–4402. PMLR, 2018.
  54. Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the MLSDA 2014 2nd workshop on machine learning for sensory data analysis, pages 4–11, 2014.
  55. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In Information Processing in Medical Imaging: 25th International Conference, IPMI 2017, Boone, NC, USA, June 25-30, 2017, Proceedings, pages 146–157. Springer, 2017.
  56. Support vector method for novelty detection. Advances in neural information processing systems, 12, 1999.
  57. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
  58. Anomaly detection in medical imaging-a mini review. In Data Science–Analytics and Applications: Proceedings of the 4th International Data Science Conference–iDSC2021, pages 33–38. Springer, 2022.
  59. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(12), 2010.
  60. Cloud intrusion detection method based on stacked contractive auto-encoder and support vector machine. IEEE transactions on cloud computing, 10(3):1634–1646, 2020.
  61. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
  62. Robust feature extraction for geochemical anomaly recognition using a stacked convolutional denoising autoencoder. Mathematical Geosciences, pages 1–22, 2021.
  63. Hybrid robust convolutional autoencoder for unsupervised anomaly detection of machine tools under noises. Robotics and Computer-Integrated Manufacturing, 79:102441, 2023.
  64. Real-time intrusion detection in wireless network: A deep learning-based intelligent mechanism. IEEE Access, 8:170128–170139, 2020.
  65. Regularized cycle consistent generative adversarial network for anomaly detection. arXiv preprint arXiv:2001.06591, 2020.
  66. Memory-augmented generative adversarial networks for anomaly detection. IEEE Transactions on Neural Networks and Learning Systems, 33(6):2324–2334, 2021.
  67. Adversarially learned anomaly detection. In 2018 IEEE International conference on data mining (ICDM), pages 727–736. IEEE, 2018.
  68. Deep structured energy based models for anomaly detection. In International conference on machine learning, pages 1100–1109. PMLR, 2016.
  69. Anomaly detection and diagnosis for wind turbines using long short-term memory-based stacked denoising autoencoders and xgboost. Reliability Engineering & System Safety, 222:108445, 2022.
  70. Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126, 2016.
  71. One-class adversarial nets for fraud detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1286–1293, 2019.
  72. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 665–674, 2017.
  73. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International conference on learning representations, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets