Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fusing Dictionary Learning and Support Vector Machines for Unsupervised Anomaly Detection (2404.04064v1)

Published 5 Apr 2024 in cs.LG, cs.CR, cs.NA, and math.NA

Abstract: We study in this paper the improvement of one-class support vector machines (OC-SVM) through sparse representation techniques for unsupervised anomaly detection. As Dictionary Learning (DL) became recently a common analysis technique that reveals hidden sparse patterns of data, our approach uses this insight to endow unsupervised detection with more control on pattern finding and dimensions. We introduce a new anomaly detection model that unifies the OC-SVM and DL residual functions into a single composite objective, subsequently solved through K-SVD-type iterative algorithms. A closed-form of the alternating K-SVD iteration is explicitly derived for the new composite model and practical implementable schemes are discussed. The standard DL model is adapted for the Dictionary Pair Learning (DPL) context, where the usual sparsity constraints are naturally eliminated. Finally, we extend both objectives to the more general setting that allows the use of kernel functions. The empirical convergence properties of the resulting algorithms are provided and an in-depth analysis of their parametrization is performed while also demonstrating their numerical performance in comparison with existing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Dictionary learning enhancement framework: Learning a non-linear mapping model to enhance discriminative dictionary learning methods. Neurocomputing 357, 135–150.
  2. K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Trans. Signal Proc. 54, 4311–4322.
  3. Outlier-aware dictionary learning for sparse representation, in: 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), IEEE. pp. 1–6.
  4. The’k’in k-fold cross validation., in: ESANN, pp. 441–446.
  5. Hidden convexity in some nonconvex quadratically constrained quadratic programming. Mathematical Programming 72, 51–63.
  6. LOF: identifying density-based local outliers, in: Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 93–104.
  7. Support vector guided dictionary learning, in: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13, Springer. pp. 624–639.
  8. Discriminative dictionary pair learning with scale-constrained structured representation for image classification. IEEE Transactions on Neural Networks and Learning Systems 34, 10225–10239.
  9. Multiple structured latent double dictionary pair learning for cross-domain industrial process monitoring. Information Sciences 648, 119514.
  10. Discriminative analysis dictionary learning with adaptively ordinal locality preserving. Neural Networks 165, 298–309.
  11. Support vector machine embedding discriminative dictionary pair learning for pattern classification. Neural Networks 155, 498–511.
  12. Structured discriminant analysis dictionary learning for pattern classification. Knowledge-Based Systems 216, 106794.
  13. Dictionary learning algorithms and applications. Springer.
  14. Sparse and redundant representations: from theory to applications in signal and image processing. Springer Science & Business Media.
  15. Isolation forest, in: 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422.
  16. Sparsity-driven laplacian-regularized outlier identification for dictionary learning. IEEE Transactions on Signal Processing 65, 3803–3817.
  17. Projective dictionary pair learning for pattern classification. Advances in neural information processing systems 27.
  18. Dictionary learning with uniform sparse representations for anomaly detection, in: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 3378–3382.
  19. Principal component analysis: a review and recent developments. Philosophical transactions of the royal society A: Mathematical, Physical and Engineering Sciences 374, 20150202.
  20. Autoassociative neural networks. Computers & Chemical Engineering 16, 313–328. Neutral network applications in chemical engineering.
  21. Hyperspectral anomaly detection by the use of background joint sparse representation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 8, 2523–2533.
  22. Scalable and interpretable one-class svms with deep learning and random fourier features, in: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part I 18, Springer. pp. 157–172.
  23. Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet Decomposition, in: 27th Asilomar Conf. Signals Systems Computers, pp. 40–44.
  24. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science 2, 559–572.
  25. Deep one-class classification, in: International conference on machine learning, PMLR. pp. 4393–4402.
  26. Support vector method for novelty detection., in: NIPS, Citeseer. pp. 582–588.
  27. Relaxed support vector based dictionary learning for image classification. Multimedia Tools and Applications , 1–25.
  28. Exploiting class-wise coding coefficients: Learning a discriminative dictionary for pattern classification. Neurocomputing 321, 114–125.
  29. Multi-layer discriminative dictionary learning with locality constraint for image classification. Pattern Recognition 91, 135–146.
  30. Design of non-linear kernel dictionaries for object recognition. IEEE Transactions on Image Processing 22, 5123–5135.
  31. Efficient and robust discriminant dictionary pair learning for pattern classification. Digital Signal Processing 118, 103227.
  32. Learning anomalous features via sparse coding using matrix norms, in: 2015 IEEE Signal Processing and Signal Processing Education Workshop (SP/SPE), IEEE. pp. 196–201.
  33. The statistical properties analysis of reconstruction error in greedy pursuit algorithms: Taking omp as an example. Electronics Letters 59, e13023.
  34. Svms multi-class loss feedback based discriminative dictionary learning for image classification. Pattern Recognition 112, 107690.
  35. Fisher Discrimination Dictionary Learning for Sparse Representation, in: IEEE International Conference on Computer Vision, pp. 543–550.
  36. Deep discriminative dictionary pair learning for image classification. Applied Intelligence , 1–14.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now