Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
140 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Federated Learning for Misbehaviour Detection with Variational Autoencoders and Gaussian Mixture Models (2405.09903v1)

Published 16 May 2024 in cs.LG and cs.DC

Abstract: Federated Learning (FL) has become an attractive approach to collaboratively train Machine Learning (ML) models while data sources' privacy is still preserved. However, most of existing FL approaches are based on supervised techniques, which could require resource-intensive activities and human intervention to obtain labelled datasets. Furthermore, in the scope of cyberattack detection, such techniques are not able to identify previously unknown threats. In this direction, this work proposes a novel unsupervised FL approach for the identification of potential misbehavior in vehicular environments. We leverage the computing capabilities of public cloud services for model aggregation purposes, and also as a central repository of misbehavior events, enabling cross-vehicle learning and collective defense strategies. Our solution integrates the use of Gaussian Mixture Models (GMM) and Variational Autoencoders (VAE) on the VeReMi dataset in a federated environment, where each vehicle is intended to train only with its own data. Furthermore, we use Restricted Boltzmann Machines (RBM) for pre-training purposes, and Fedplus as aggregation function to enhance model's convergence. Our approach provides better performance (more than 80 percent) compared to recent proposals, which are usually based on supervised techniques and artificial divisions of the VeReMi dataset.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Z. Ahmad, A. Shahid Khan, C. Wai Shiang, J. Abdullah, and F. Ahmad, “Network intrusion detection system: A systematic study of machine learning and deep learning approaches,” Transactions on Emerging Telecommunications Technologies, vol. 32, no. 1, p. e4150, 2021.
  2. C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao, “A survey on federated learning,” Knowledge-Based Systems, vol. 216, p. 106775, 2021.
  3. L. Lyu, H. Yu, and Q. Yang, “Threats to federated learning: A survey,” arXiv preprint arXiv:2003.02133, 2020.
  4. M. Aledhari, R. Razzak, R. M. Parizi, and F. Saeed, “Federated learning: A survey on enabling technologies, protocols, and applications,” IEEE Access, vol. 8, pp. 140 699–140 725, 2020.
  5. M. Billah, S. Mehedi, A. Anwar, Z. Rahman, R. Islam et al., “A systematic literature review on blockchain enabled federated learning framework for internet of vehicles,” arXiv preprint arXiv:2203.05192, 2022.
  6. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  7. R. W. van der Heijden, S. Dietzel, T. Leinmüller, and F. Kargl, “Survey on misbehavior detection in cooperative intelligent transportation systems,” IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 779–811, 2018.
  8. A. Uprety, D. B. Rawat, and J. Li, “Privacy preserving misbehavior detection in iov using federated machine learning,” in 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC).   IEEE, 2021, pp. 1–6.
  9. A. Boualouache and T. Engel, “Federated learning-based scheme for detecting passive mobile attackers in 5g vehicular edge computing,” Annals of Telecommunications, vol. 77, no. 3, pp. 201–220, 2022.
  10. ——, “A survey on machine learning-based misbehavior detection systems for 5g and beyond vehicular networks,” arXiv preprint arXiv:2201.10500, 2022.
  11. A. I. Károly, R. Fullér, and P. Galambos, “Unsupervised clustering for deep learning: A tutorial survey,” Acta Polytechnica Hungarica, vol. 15, no. 8, pp. 29–53, 2018.
  12. R. Saravanan and P. Sujatha, “A state of art techniques on machine learning algorithms: a perspective of supervised learning approaches in data classification,” in 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS).   IEEE, 2018, pp. 945–949.
  13. M. Usama, J. Qadir, A. Raza, H. Arif, K.-L. A. Yau, Y. Elkhatib, A. Hussain, and A. Al-Fuqaha, “Unsupervised machine learning for networking: Techniques, applications and research challenges,” IEEE access, vol. 7, pp. 65 579–65 615, 2019.
  14. F. Zhang, K. Kuang, L. Chen, Z. You, T. Shen, J. Xiao, Y. Zhang, C. Wu, F. Wu, Y. Zhuang et al., “Federated unsupervised representation learning,” Frontiers of Information Technology & Electronic Engineering, vol. 24, no. 8, pp. 1181–1193, 2023.
  15. Y. Jin, Y. Liu, K. Chen, and Q. Yang, “Federated learning without full labels: A survey,” arXiv preprint arXiv:2303.14453, 2023.
  16. J. L. Hernandez-Ramos, G. Karopoulos, E. Chatzoglou, V. Kouliaridis, E. Marmol, A. Gonzalez-Vidal, and G. Kambourakis, “Intrusion detection based on federated learning: a systematic review,” arXiv preprint arXiv:2308.09522, 2023.
  17. R. W. van der Heijden, T. Lukaseder, and F. Kargl, “Veremi: A dataset for comparable evaluation of misbehavior detection in vanets,” in Security and Privacy in Communication Networks: 14th International Conference, SecureComm 2018, Singapore, Singapore, August 8-10, 2018, Proceedings, Part I.   Springer, 2018, pp. 318–337.
  18. V. Vashishth, A. Chhabra, and D. K. Sharma, “Gmmr: A gaussian mixture model based unsupervised machine learning approach for optimal routing in opportunistic iot networks,” Computer Communications, vol. 134, pp. 138–148, 2019.
  19. C. M. Bishop, “Neural networks and their applications,” Review of scientific instruments, vol. 65, no. 6, pp. 1803–1832, 1994.
  20. M. Said Elsayed, N.-A. Le-Khac, S. Dev, and A. D. Jurcut, “Network anomaly detection using lstm based autoencoder,” in Proceedings of the 16th ACM Symposium on QoS and Security for Wireless and Mobile Networks, 2020, pp. 37–45.
  21. P. Yu, A. Kundu, L. Wynter, and S. H. Lim, “Fed+: A unified approach to robust personalized federated learning,” arXiv preprint arXiv:2009.06303, 2020.
  22. E. M. Campos, P. F. Saura, A. González-Vidal, J. L. Hernández-Ramos, J. B. Bernabe, G. Baldini, and A. Skarmeta, “Evaluating federated learning for intrusion detection in internet of things: Review and challenges,” Computer Networks, p. 108661, 2021.
  23. P. Awasthi and O. Sheffet, “Improved spectral-norm bounds for clustering,” in Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques.   Springer, 2012, pp. 37–49.
  24. D. A. Reynolds, “Gaussian mixture models.” Encyclopedia of biometrics, vol. 741, no. 659-663, 2009.
  25. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
  26. K. Zhang, Y. Jiang, L. Seversky, C. Xu, D. Liu, and H. Song, “Federated variational learning for anomaly detection in multivariate time series,” in 2021 IEEE International Performance, Computing, and Communications Conference (IPCCC).   IEEE, 2021, pp. 1–9.
  27. R. Yao, C. Liu, L. Zhang, and P. Peng, “Unsupervised anomaly detection using variational auto-encoder based feature extraction,” in 2019 IEEE International Conference on Prognostics and Health Management (ICPHM).   IEEE, 2019, pp. 1–7.
  28. J. Sun, X. Wang, N. Xiong, and J. Shao, “Learning sparse representation with variational auto-encoder for anomaly detection,” IEEE Access, vol. 6, pp. 33 353–33 361, 2018.
  29. J. An and S. Cho, “Variational autoencoder based anomaly detection using reconstruction probability,” Special Lecture on IE, vol. 2, no. 1, pp. 1–18, 2015.
  30. J. M. Joyce, “Kullback-leibler divergence,” in International encyclopedia of statistical science.   Springer, 2011, pp. 720–722.
  31. G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” science, vol. 313, no. 5786, pp. 504–507, 2006.
  32. X.-J. Wu, M.-D. Xu, C.-D. Li, C. Ju, Q. Zhao, and S.-X. Liu, “Research on image reconstruction algorithms based on autoencoder neural network of restricted boltzmann machine (rbm),” Flow Measurement and Instrumentation, vol. 80, p. 102009, 2021.
  33. A. G. Pacheco, R. A. Krohling, and C. A. da Silva, “Restricted boltzmann machine to determine the input weights for extreme learning machines,” Expert Systems with Applications, vol. 96, pp. 77–85, 2018.
  34. S. Saravanan and J. Sujitha, “Deep medical image reconstruction with autoencoders using deep boltzmann machine training,” EAI Endorsed Transactions on Pervasive Health and Technology, vol. 6, no. 24, p. e2, 2020.
  35. P. Sharma and H. Liu, “A machine-learning-based data-centric misbehavior detection model for internet of vehicles,” IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4991–4999, 2020.
  36. O. Slama, B. Alaya, S. Zidi, and M. Tarhouni, “Comparative study of misbehavior detection system for classifying misbehaviors on vanet,” in 2022 8th International Conference on Control, Decision and Information Technologies (CoDIT), vol. 1.   IEEE, 2022, pp. 243–248.
  37. G. O. Anyanwu, C. I. Nwakanma, J.-M. Lee, and D.-S. Kim, “Real-time position falsification attack detection system for internet of vehicles,” in 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).   IEEE, 2021, pp. 1–4.
  38. T. Moulahi, R. Jabbar, A. Alabdulatif, S. Abbas, S. El Khediri, S. Zidi, and M. Rizwan, “Privacy-preserving federated learning cyber-threat detection for intelligent transport systems with blockchain-based security,” Expert Systems, p. e13103, 2022.
  39. J. Kamel, M. Wolf, R. W. Van Der Hei, A. Kaiser, P. Urien, and F. Kargl, “Veremi extension: A dataset for comparable evaluation of misbehavior detection in vanets,” in ICC 2020-2020 IEEE International Conference on Communications (ICC).   IEEE, 2020, pp. 1–6.
  40. E. Kristianto, P.-C. Lin, and R.-H. Hwang, “Misbehavior detection system with semi-supervised federated learning,” Vehicular Communications, p. 100597, 2023.
  41. B. Zong, Q. Song, M. R. Min, W. Cheng, C. Lumezanu, D. Cho, and H. Chen, “Deep autoencoding gaussian mixture model for unsupervised anomaly detection,” in International conference on learning representations, 2018.
  42. P. An, Z. Wang, and C. Zhang, “Ensemble unsupervised autoencoders and gaussian mixture model for cyberattack detection,” Information Processing & Management, vol. 59, no. 2, p. 102844, 2022.
  43. W. Liao, Y. Guo, X. Chen, and P. Li, “A unified unsupervised gaussian mixture variational autoencoder for high dimensional outlier detection,” in 2018 IEEE International Conference on Big Data (Big Data).   IEEE, 2018, pp. 1208–1217.
  44. Z. Zhao, X. Liang, H. Huang, and K. Wang, “Deep federated learning hybrid optimization model based on encrypted aligned data,” Pattern Recognition, vol. 148, p. 110193, 2024.
  45. Y. Chen, J. Zhang, and C. K. Yeo, “Network anomaly detection using federated deep autoencoding gaussian mixture model,” in International Conference on Machine Learning for Networking.   Springer, 2019, pp. 1–14.
  46. D. Preuveneers, V. Rimmer, I. Tsingenopoulos, J. Spooren, W. Joosen, and E. Ilie-Zudor, “Chained anomaly detection models for federated learning: An intrusion detection case study,” Applied Sciences, vol. 8, no. 12, p. 2663, 2018.
  47. B. Cetin, A. Lazar, J. Kim, A. Sim, and K. Wu, “Federated wireless network intrusion detection,” in 2019 IEEE International Conference on Big Data (Big Data).   IEEE, 2019, pp. 6004–6006.
  48. M. B. Alazzam, F. Alassery, and A. Almulihi, “Federated deep learning approaches for the privacy and security of iot systems,” Wireless Communications and Mobile Computing, vol. 2022, 2022.
  49. D. Wu, Y. Deng, and M. Li, “Fl-mgvn: Federated learning for anomaly detection using mixed gaussian variational self-encoding network,” Information Processing & Management, vol. 59, no. 2, p. 102839, 2022.
  50. L. Codeca, R. Frank, and T. Engel, “Luxembourg sumo traffic (lust) scenario: 24 hours of mobility for vehicular networking research,” in 2015 IEEE Vehicular Networking Conference (VNC), 2015, pp. 1–8.
  51. G. E. Batista, A. L. Bazzan, M. C. Monard et al., “Balancing training data for automated annotation of keywords: a case study.” in WOB, 2003, pp. 10–18.
  52. S. S. SHAPIRO and M. B. WILK, “An analysis of variance test for normality (complete samples),” Biometrika, vol. 52, no. 3-4, pp. 591–611, dec 1965. [Online]. Available: https://doi.org/10.1093/biomet/52.3-4.591
  53. A. Etz, “Introduction to the concept of likelihood and its applications,” Advances in Methods and Practices in Psychological Science, vol. 1, no. 1, pp. 60–69, 2018.
  54. P. J. Rousseeuw, “Silhouettes: a graphical aid to the interpretation and validation of cluster analysis,” Journal of computational and applied mathematics, vol. 20, pp. 53–65, 1987.
  55. M. Zolotukhin, T. Hämäläinen, T. Kokkonen, and J. Siltanen, “Increasing web service availability by detecting application-layer ddos attacks in encrypted traffic,” in 2016 23rd International conference on telecommunications (ICT).   IEEE, 2016, pp. 1–6.
  56. N. Garcia, T. Alcaniz, A. González-Vidal, J. B. Bernabe, D. Rivera, and A. Skarmeta, “Distributed real-time slowdos attacks detection over encrypted traffic using artificial intelligence,” Journal of Network and Computer Applications, vol. 173, p. 102871, 2021.
  57. D. J. Beutel, T. Topal, A. Mathur, X. Qiu, T. Parcollet, and N. D. Lane, “Flower: A friendly federated learning research framework,” arXiv preprint arXiv:2007.14390, 2020.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets