Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Constrained Twin Variational Auto-Encoder for Intrusion Detection in IoT Systems (2312.02490v1)

Published 5 Dec 2023 in cs.LG and cs.CR

Abstract: Intrusion detection systems (IDSs) play a critical role in protecting billions of IoT devices from malicious attacks. However, the IDSs for IoT devices face inherent challenges of IoT systems, including the heterogeneity of IoT data/devices, the high dimensionality of training data, and the imbalanced data. Moreover, the deployment of IDSs on IoT systems is challenging, and sometimes impossible, due to the limited resources such as memory/storage and computing capability of typical IoT devices. To tackle these challenges, this article proposes a novel deep neural network/architecture called Constrained Twin Variational Auto-Encoder (CTVAE) that can feed classifiers of IDSs with more separable/distinguishable and lower-dimensional representation data. Additionally, in comparison to the state-of-the-art neural networks used in IDSs, CTVAE requires less memory/storage and computing power, hence making it more suitable for IoT IDS systems. Extensive experiments with the 11 most popular IoT botnet datasets show that CTVAE can boost around 1% in terms of accuracy and Fscore in detection attack compared to the state-of-the-art machine learning and representation learning methods, whilst the running time for attack detection is lower than 2E-6 seconds and the model size is lower than 1 MB. We also further investigate various characteristics of CTVAE in the latent space and in the reconstruction representation to demonstrate its efficacy compared with current well-known methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. L. Yang, A. Moubayed, and A. Shami, “Mth-ids: A multitiered hybrid intrusion detection system for internet of vehicles,” IEEE Internet of Things Journal, vol. 9, no. 1, pp. 616–632, Jan. 2021.
  2. S. I. Popoola, B. Adebisi, M. Hammoudeh, G. Gui, and H. Gacanin, “Hybrid deep learning for botnet attack detection in the internet-of-things networks,” IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4944–4956, Oct. 2020.
  3. P. Xanthopoulos, P. M. Pardalos, T. B. Trafalis, P. Xanthopoulos, P. M. Pardalos, and T. B. Trafalis, “Linear discriminant analysis,” Robust data mining, pp. 27–33, Jan. 2013.
  4. I. Ullah and Q. H. Mahmoud, “Design and development of a deep learning-based model for anomaly detection in iot networks,” IEEE Access, vol. 9, pp. 103 906–103 926, Jul. 2021.
  5. G. Abdelmoumin, D. B. Rawat, and A. Rahman, “On the performance of machine learning models for anomaly-based intelligent intrusion detection systems for the internet of things,” IEEE Internet of Things Journal, vol. 9, no. 6, pp. 4280–4290, Aug. 2021.
  6. S. Hajiheidari, K. Wakil, M. Badri, and N. J. Navimipour, “Intrusion detection systems in the internet of things: A comprehensive investigation,” Computer Networks, vol. 160, pp. 165–191, Sept. 2019.
  7. A. Khraisat, I. Gondal, P. Vamplew, and J. Kamruzzaman, “Survey of intrusion detection systems: techniques, datasets and challenges,” Cybersecurity, vol. 2, no. 1, pp. 1–22, Jul. 2019.
  8. M. A. Ferrag, L. Maglaras, S. Moschoyiannis, and H. Janicke, “Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study,” Journal of Information Security and Applications, vol. 50, no. 1, p. 102419, 2020.
  9. Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, Mar. 2013.
  10. T.-N. Dao and H. Lee, “Stacked autoencoder-based probabilistic feature extraction for on-device network intrusion detection,” IEEE Internet of Things Journal, vol. 9, no. 16, pp. 14 438–14 451, Aug. 2022.
  11. J. Sun, X. Wang, N. Xiong, and J. Shao, “Learning sparse representation with variational auto-encoder for anomaly detection,” IEEE Access, vol. 6, pp. 33 353–33 361, Jun. 2018.
  12. L. Vu, V. L. Cao, Q. U. Nguyen, D. N. Nguyen, D. T. Hoang, and E. Dutkiewicz, “Learning latent representation for iot anomaly detection,” IEEE Transactions on Cybernetics, pp. 1–14, Sept. 2020.
  13. J. Tomczak and M. Welling, “Vae with a vampprior,” in International Conference on Artificial Intelligence and Statistics.   Playa Blanca, Lanzarote, Canary Islands: PMLR, 2018, pp. 1214–1223.
  14. P. V. Dinh, N. Q. Uy, D. N. Nguyen, D. T. Hoang, S. P. Bao, and E. Dutkiewicz, “Twin variational auto-encoder for representation learning in iot intrusion detection,” in 2022 IEEE Wireless Communications and Networking Conference (WCNC).   Austin, TX, USA: IEEE, 2022, pp. 848–853.
  15. R. Lletı, M. C. Ortiz, L. A. Sarabia, and M. S. Sánchez, “Selecting variables for k-means cluster analysis by using a genetic algorithm that optimises the silhouettes,” Analytica Chimica Acta, vol. 515, no. 1, pp. 87–100, Jul. 2004.
  16. Y. Meidan, M. Bohadana, Y. Mathov, Y. Mirsky, A. Shabtai, D. Breitenbacher, and Y. Elovici, “N-baiot—network-based detection of iot botnet attacks using deep autoencoders,” IEEE Pervasive Computing, vol. 17, no. 3, pp. 12–22, Mar. 2018.
  17. C. Yin, S. Zhang, J. Wang, and N. N. Xiong, “Anomaly detection based on convolutional recurrent autoencoder for iot time series,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 52, no. 1, pp. 112–122, Jan. 2020.
  18. T.-N. Dao and H. Lee, “Stacked autoencoder-based probabilistic feature extraction for on-device network intrusion detection,” IEEE Internet of Things Journal, vol. 9, no. 16, pp. 14 438–14 451, May 2021.
  19. W. Luo, J. Li, J. Yang, W. Xu, and J. Zhang, “Convolutional sparse autoencoders for image classification,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 7, pp. 3289–3294, Jul. 2018.
  20. M. Al-Qatf, Y. Lasheng, M. Al-Habib, and K. Al-Sabahi, “Deep learning approach combining sparse autoencoder with svm for network intrusion detection,” IEEE Access, vol. 6, pp. 52 843–52 856, Sept. 2018.
  21. T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco California USA, 2016, pp. 785–794.
  22. “Linear support vector machine.” [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html.
  23. “Decision tree.” [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html.
  24. “Random forest.” [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html.
  25. S. S. S. Sindhu, S. Geetha, and A. Kannan, “Decision tree based light weight intrusion detection using a wrapper approach,” Expert Systems with applications, vol. 39, no. 1, pp. 129–141, Jan. 2012.
  26. D. Moon, H. Im, I. Kim, and J. H. Park, “Dtb-ids: an intrusion detection system based on decision tree using behavior analysis for preventing apt attacks,” The Journal of supercomputing, vol. 73, no. 7, pp. 2881–2895, Dec. 2017.
  27. R.-C. Chen, K.-F. Cheng, Y.-H. Chen, and C.-F. Hsieh, “Using rough set and support vector machine for network intrusion detection system,” in 2009 First Asian Conference on Intelligent Information and Database Systems.   Dong hoi, Vietnam: IEEE, 2009, pp. 465–470.
  28. M. A. M. Hasan, M. Nasser, B. Pal, and S. Ahmad, “Support vector machine and random forest modeling for intrusion detection system (ids),” Journal of Intelligent Learning Systems and Applications, vol. 6, no. 1, Aug. 2014.
  29. X. Li, W. Chen, Q. Zhang, and L. Wu, “Building auto-encoder intrusion detection system based on random forest feature selection,” Computers & Security, vol. 95, p. 101851, Aug. 2020.
  30. D. S. Kim, S. M. Lee, and J. S. Park, “Building lightweight intrusion detection system based on random forest,” in International Symposium on Neural Networks.   Chengdu, China: Springer, 2006, pp. 224–230.
  31. T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, United States, 2016, pp. 785–794.
  32. B. S. Bhati, G. Chugh, F. Al-Turjman, and N. S. Bhati, “An improved ensemble based intrusion detection technique using xgboost,” Transactions on Emerging Telecommunications Technologies, vol. 32, no. 6, p. e4076, Aug. 2021.
  33. L. Vu, Q. U. Nguyen, D. N. Nguyen, D. T. Hoang, and E. Dutkiewicz, “Deep transfer learning for iot attack detection,” IEEE Access, vol. 8, pp. 107 335–107 344, Jun. 2020.
  34. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” Journal of Machine Learning Research, vol. 11, no. 12, pp. 3371–3408, Dec. 2010.
  35. N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, “A deep learning approach to network intrusion detection,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41–50, Feb. 2018.
  36. G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, Jul. 2006.
  37. A. V. Phan, P. N. Chau, M. Le Nguyen, and L. T. Bui, “Automatically classifying source code using tree-based approaches,” Data & Knowledge Engineering, vol. 114, pp. 12–25, Mar. 2018.
  38. R. Abdulhammed, M. Faezipour, A. Abuzneid, and A. AbuMallouh, “Deep and machine learning approaches for anomaly-based intrusion detection of imbalanced network traffic,” IEEE Sensors Letters, vol. 3, no. 1, pp. 1–4, Nov. 2018.
  39. E. Jang, S. Gu, and B. Poole, “Categorical reparameterization with gumbel-softmax,” in International Conference on Learning Representations, Toulon, France, 2017, pp. 1–12.
  40. C. P. Burgess, I. Higgins, A. Pal, L. Matthey, N. Watters, G. Desjardins, and A. Lerchner, “Understanding disentangling in beta-vae,” arXiv preprint arXiv:1804.03599, 2018.
  41. H. Wu and M. Flierl, “Vector quantization-based regularization for autoencoders,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, New York, USA, 2020, pp. 6380–6387.
  42. D. P. Kingma and M. Welling, “An introduction to variational autoencoders,” Foundations and Trends® in Machine Learning, vol. 12, no. 4, p. 307–392, Nov. 2019.
  43. J. An and S. Cho, “Variational autoencoder based anomaly detection using reconstruction probability,” Special Lecture on IE, vol. 2, no. 1, pp. 1–18, Dec. 2015.
  44. P. V. Dinh, D. N. Nguyen, D. T. Hoang, N. Q. Uy, S. P. Bao, and E. Dutkiewicz, “Balanced twin auto-encoder for iot intrusion detection,” in GLOBECOM 2022-2022 IEEE Global Communications Conference.   Rio de Janeiro, Brazil: IEEE, 2022, pp. 3387–3392.
  45. H. Zou, T. Hastie, and R. Tibshirani, “Sparse principal component analysis,” Journal of computational and graphical statistics, vol. 15, no. 2, pp. 265–286, Jan. 2006.
  46. A. Tharwat, T. Gaber, A. Ibrahim, and A. E. Hassanien, “Linear discriminant analysis: A detailed tutorial,” AI Communications, vol. 30, no. 2, pp. 169–190, May 2017.
  47. I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, “Toward generating a new intrusion detection dataset and intrusion traffic characterization,” ICISSp, vol. 1, pp. 108–116, Jan. 2018.
  48. N. Moustafa and J. Slay, “Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set),” in 2015 Military Communications and Information Systems Conference (MilCIS), Canberra, Australia, 2015, pp. 1–6.
  49. “Scikit-learn.” [Online]. Available: https://scikit-learn.org/stable/.
  50. K. DP and J. Ba, “Adam: A method for stochastic optimization,” in Proc. of the 3rd International Conference for Learning Representations (ICLR), San Diego, California, US, 2015, pp. 1–15.
  51. X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, vol. 9, Chia Laguna Resort, Sardinia, Italy, 2010, pp. 249–256.
  52. M. Nicolau, J. McDermott et al., “Learning neural representations for network anomaly detection,” IEEE Transactions on Cybernetics, vol. 49, no. 8, pp. 3074–3087, Jun. 2018.
  53. J. Park, J. Lee, and D. Sim, “Low-complexity cnn with 1d and 2d filters for super-resolution,” Journal of Real-Time Image Processing, vol. 17, no. 6, pp. 2065–2076, Jun. 2020.
  54. I. Syarif, A. Prugel-Bennett, and G. Wills, “Svm parameter optimization using grid search and genetic algorithm to improve classification performance,” Telkomnika, vol. 14, no. 4, p. 1502, Apr. 2016.
  55. B. Shekar and G. Dagnew, “Grid search-based hyperparameter tuning and classification of microarray cancer data,” in 2019 Second International Conference on Advanced Computational and Communication Paradigms (ICACCP).   Gangtok, Sikkim, India: IEEE, 2019, pp. 1–8.
  56. P. Probst, M. N. Wright, and A.-L. Boulesteix, “Hyperparameters and tuning strategies for random forest,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 9, no. 3, p. e1301, Mar. 2019.
  57. “Make blobs.” [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_blobs.html.
Citations (3)
View on Semantic Scholar

Summary

  • The paper introduces the CTVAE model that transforms high-dimensional IoT data into a more manageable form for enhanced intrusion detection.
  • It demonstrates a 1% improvement in accuracy and F-score with detection times below 2E-6 seconds and a compact model size under 1 MB.
  • The approach effectively handles imbalanced data and outperforms traditional methods like LDA and popular models such as XGBoost in IDS tasks.

Understanding the CTVAE Model for IoT Intrusion Detection

Introduction to Intrusion Detection Systems (IDS)

Intrusion Detection Systems are essential for securing the increasing number of Internet of Things (IoT) devices from malicious attacks. However, IoT systems raise several challenges such as data heterogeneity, high data dimensionality, data imbalance, and limited resources of IoT devices.

A Novel Approach: Constrained Twin Variational Auto-Encoder

To address these challenges, a new deep neural network architecture named Constrained Twin Variational Auto-Encoder (CTVAE) has been introduced. CTVAE offers a way to produce a more distinguishable dataset for IDS classifiers, transforming complex, high-dimensional data into a more manageable form. Compared to other neural network approaches, CTVAE is designed to require less memory and processing power, making it particularly suitable for application in resource-constrained IoT systems.

Experimentation and Results

Extensive experiments on several IoT datasets reveal that CTVAE outperforms existing representation learning methods as well as several popular machine learning models, including state-of-the-art methods like Xgboost. The CTVAE model demonstrated around a 1% increase in accuracy and F-score, with detection times below 2E-6 seconds and model sizes less than 1 MB.

CTVAE also showcases an ability to handle imbalanced data effectively when combined with techniques like K-Mean clustering to find reasonable data distributions.

Comparison with Traditional Techniques

The CTVAE's superiority is further highlighted when comparing its results with those obtained using Linear Discriminant Analysis (LDA), a technique commonly employed for dimensionality reduction in IDS. The CTVAE excels at providing better discrimination between classes, thus enhancing classification tasks performed by intrusion detection models.

CTVAE's Advantages and Impact

One of the significant advantages of CTVAE is its capacity to improve anomaly detection performance without the need for extensive computing resources. This feature can potentially revolutionize the deployment of IDS in IoT environments, where devices are typically resource-constrained.

Conclusion and Potential Directions

The CTVAE presents a promising direction for researchers and practitioners in the field of cybersecurity, specifically for IoT intrusion detection. While the model has shown compelling results, there is scope for future work to explore its application across various other domains, to improve automated parameter selection for model training or adapt the twin model concept to other generative models.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets