Semisupervised Anomaly Detection using Support Vector Regression with Quantum Kernel (2308.00583v2)
Abstract: Anomaly detection (AD) involves identifying observations or events that deviate in some way from the rest of the data. Machine learning techniques have shown success in automating this process by detecting hidden patterns and deviations in large-scale data. The potential of quantum computing for machine learning has been widely recognized, leading to extensive research efforts to develop suitable quantum machine learning (QML) algorithms. In particular, the search for QML algorithms for near-term NISQ devices is in full swing. However, NISQ devices pose additional challenges due to their limited qubit coherence times, low number of qubits, and high error rates. Kernel methods based on quantum kernel estimation have emerged as a promising approach to QML on NISQ devices, offering theoretical guarantees, versatility, and compatibility with NISQ constraints. Especially support vector machines (SVM) utilizing quantum kernel estimation have shown success in various supervised learning tasks. However, in the context of AD, semisupervised learning is of great relevance, and yet there is limited research published in this area. This paper introduces an approach to semisupervised AD based on the reconstruction loss of a support vector regression (SVR) with quantum kernel. This novel model is an alternative to the variational quantum and quantum kernel one-class classifiers, and is compared to a quantum autoencoder as quantum baseline and a SVR with radial-basis-function (RBF) kernel as well as a classical autoencoder as classical baselines. The models are benchmarked extensively on 10 real-world AD data sets and one toy data set, and it is shown that our SVR model with quantum kernel performs better than the SVR with RBF kernel as well as all other models, achieving highest mean AUC over all data sets. In addition, our QSVR outperforms the quantum autoencoder on 9 out of 11 data sets.
- L. Ruff, J. R. Kauffmann, R. A. Vandermeulen, G. Montavon, W. Samek, M. Kloft, T. G. Dietterich, and K.-R. Müller, “A unifying review of deep and shallow anomaly detection,” Proceedings of the IEEE, vol. 109, no. 5, pp. 756–795, 2021.
- S. Thudumu, P. Branch, J. Jin, and J. J. Singh, “A comprehensive survey of anomaly detection techniques for high dimensional big data,” Journal of Big Data, vol. 7, 12 2020.
- K. Kottmann, F. Metz, J. Fraxanet, and N. Baldelli, “Variational quantum anomaly detection: Unsupervised mapping of phase diagrams on a physical quantum computer,” Phys. Rev. Res., vol. 3, p. 043184, 12 2021. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevResearch.3.043184
- V. Havlíček, A. D. Córcoles, K. Temme, A. W. Harrow, A. Kandala, J. M. Chow, and J. M. Gambetta, “Supervised learning with quantum-enhanced feature spaces,” Nature, vol. 567, no. 7747, pp. 209–212, Mar. 2019.
- A. Perdomo-Ortiz, M. Benedetti, J. Realpe-Gómez, and R. Biswas, “Opportunities and challenges for quantum-assisted machine learning in near-term quantum computers,” Quantum Science and Technology, vol. 3, no. 3, p. 030502, 06 2018. [Online]. Available: https://dx.doi.org/10.1088/2058-9565/aab859
- P. Rebentrost, M. Mohseni, and S. Lloyd, “Quantum support vector machine for big data classification,” Phys. Rev. Lett., vol. 113, p. 130503, 09 2014. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevLett.113.130503
- A. Delilbasic, G. Cavallaro, M. Willsch, F. Melgani, M. Riedel, and K. Michielsen, “Quantum support vector machine algorithms for remote sensing data classification,” in 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2021, pp. 2608–2611.
- S. Farhan Ahmad, R. Rawat, and M. Moharir, “Quantum machine learning with hqc architectures using non-classically simulable feature maps,” in 2021 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE), 2021, pp. 345–349.
- M. Mafu and M. Senekane, “Design and implementation of efficient quantum support vector machine,” in 2021 International Conference on Electrical, Computer and Energy Technologies (ICECET), 2021, pp. 1–4.
- K. A. Woźniak, V. Belis, E. Puljak, P. Barkoutsos, G. Dissertori, M. Grossi, M. Pierini, F. Reiter, I. Tavernelli, and S. Vallecorsa, “Quantum anomaly detection in the latent space of proton collision events at the lhc,” 2023.
- O. Kyriienko and E. B. Magnusson, “Unsupervised quantum machine learning for fraud detection,” 2022.
- M. Schuld, “Supervised quantum machine learning models are kernel methods,” arXiv preprint arXiv:2101.11020, 2021.
- J. Romero, J. P. Olson, and A. Aspuru-Guzik, “Quantum autoencoders for efficient compression of quantum data,” Quantum Science and Technology, vol. 2, no. 4, p. 045001, aug 2017. [Online]. Available: https://dx.doi.org/10.1088/2058-9565/aa8072
- C. Bravo-Prieto, “Quantum autoencoders with enhanced data encoding,” Machine Learning: Science and Technology, vol. 2, 9 2021.
- G. Park, J. Huh, and D. K. Park, “Variational quantum one-class classifier,” Machine Learning: Science and Technology, vol. 4, no. 1, p. 015006, 01 2023. [Online]. Available: https://dx.doi.org/10.1088/2632-2153/acafd5
- P. Sperl, J.-P. Schulze, and K. Böttinger, “Activation anomaly analysis,” in Machine Learning and Knowledge Discovery in Databases, F. Hutter, K. Kersting, J. Lijffijt, and I. Valera, Eds. Cham: Springer International Publishing, 2021, pp. 69–84.
- V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Comput. Surv., vol. 41, no. 3, 07 2009. [Online]. Available: https://doi.org/10.1145/1541880.1541882
- Z. Huang, B. Zhang, G. Hu, L. Li, Y. Xu, and Y. Jin, “Enhancing unsupervised anomaly detection with score-guided network,” 2022.
- C. Ciliberto, M. Herbster, A. D. Ialongo, M. Pontil, A. Rocchetto, S. Severini, and L. Wossnig, “Quantum machine learning: a classical perspective,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 474, no. 2209, p. 20170551, 2018. [Online]. Available: https://royalsocietypublishing.org/doi/abs/10.1098/rspa.2017.0551
- W. Guan, G. Perdue, A. Pesah, M. Schuld, K. Terashi, S. Vallecorsa, and J.-R. Vlimant, “Quantum machine learning in high energy physics,” Machine Learning: Science and Technology, vol. 2, no. 1, p. 011003, 03 2021. [Online]. Available: https://dx.doi.org/10.1088/2632-2153/abc17d
- M. Schuld, R. Sweke, and J. J. Meyer, “Effect of data encoding on the expressive power of variational quantum-machine-learning models,” Phys. Rev. A, vol. 103, p. 032430, Mar 2021. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevA.103.032430
- S. Sim, P. D. Johnson, and A. Aspuru-Guzik, “Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms,” Advanced Quantum Technologies, vol. 2, no. 12, p. 1900070, 2019. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/qute.201900070
- Z. Holmes, K. Sharma, M. Cerezo, and P. J. Coles, “Connecting ansatz expressibility to gradient magnitudes and barren plateaus,” PRX Quantum, vol. 3, p. 010313, Jan 2022. [Online]. Available: https://link.aps.org/doi/10.1103/PRXQuantum.3.010313
- H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola, and V. Vapnik, “Support vector regression machines,” in Advances in Neural Information Processing Systems, M. Mozer, M. Jordan, and T. Petsche, Eds., vol. 9. MIT Press, 1996. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/1996/file/d38901788c533e8286cb6400b40b386d-Paper.pdf
- J. R. Glick, T. P. Gujarati, A. D. Corcoles, Y. Kim, A. Kandala, J. M. Gambetta, and K. Temme, “Covariant quantum kernels for data with group structure,” 2022.
- J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush, and H. Neven, “Barren plateaus in quantum neural network training landscapes,” Nature Communications, vol. 9, no. 1, 11 2018. [Online]. Available: https://doi.org/10.1038%2Fs41467-018-07090-4
- M. Cerezo, A. Arrasmith, R. Babbush, S. C. Benjamin, S. Endo, K. Fujii, J. R. McClean, K. Mitarai, X. Yuan, L. Cincio, and P. J. Coles, “Variational quantum algorithms,” Nature Reviews Physics, vol. 3, no. 9, pp. 625–644, 08 2021. [Online]. Available: https://doi.org/10.1038%2Fs42254-021-00348-9
- Qiskit contributors, “Qiskit: An open-source framework for quantum computing,” 2023.
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
- Wes McKinney, “Data Structures for Statistical Computing in Python,” in Proceedings of the 9th Python in Science Conference, Stéfan van der Walt and Jarrod Millman, Eds., 2010, pp. 56 – 61.
- C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sep. 2020. [Online]. Available: https://doi.org/10.1038/s41586-020-2649-2
- J.-P. Schulze, P. Sperl, and K. Böttinger, “Double-adversarial activation anomaly detection: Adversarial autoencoders are anomaly generators,” in 2022 International Joint Conference on Neural Networks. IEEE, 07 2022. [Online]. Available: https://doi.org/10.1109/ijcnn55064.2022.9892896
- J.-P. Schulze, P. Sperl, A. Răduțoiu, C. Sagebiel, and K. Böttinger, “R2-ad2: Detecting anomalies by analysing the raw gradient,” in Machine Learning and Knowledge Discovery in Databases, M.-R. Amini, S. Canu, A. Fischer, T. Guns, P. Kralj Novak, and G. Tsoumakas, Eds. Cham: Springer International Publishing, 2023, pp. 209–224.
- A. D. Pozzolo, O. Caelen, R. A. Johnson, and G. Bontempi, “Calibrating probability with undersampling for unbalanced classification,” in 2015 IEEE Symposium Series on Computational Intelligence, 2015, pp. 159–166.
- D. Dua and C. Graff, “UCI machine learning repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml
- J. A. Blackard and D. J. Dean, “Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables,” Computers and Electronics in Agriculture, vol. 24, no. 3, pp. 131–151, 1999. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0168169999000460
- M. MontazeriShatoori, L. Davidson, G. Kaur, and A. Habibi Lashkari, “Detection of doh tunnels using time-series classification of encrypted traffic,” in 2020 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), 2020, pp. 63–70.
- G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “Emnist: Extending mnist to handwritten letters,” in 2017 International Joint Conference on Neural Networks (IJCNN), 2017, pp. 2921–2926.
- H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” 2017.
- M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the kdd cup 99 data set,” in 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009, pp. 1–6.
- Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
- K. S. WOODS, C. C. DOSS, K. W. BOWYER, J. L. SOLKA, C. E. PRIEBE, and W. P. KEGELMEYER, “Comparative evaluation of pattern recognition techniques for detection of microcalcifications in mammography,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 07, no. 06, pp. 1417–1436, 1993. [Online]. Available: https://doi.org/10.1142/S0218001493000698
- M. S. I. Mamun, M. A. Rathore, A. H. Lashkari, N. Stakhanova, and A. A. Ghorbani, “Detecting malicious urls using lexical analysis,” in Network and System Security, J. Chen, V. Piuri, C. Su, and M. Yung, Eds. Cham: Springer International Publishing, 2016, pp. 467–482.