Papers
Topics
Authors
Recent
2000 character limit reached

Semi-Supervised Learning for Anomaly Traffic Detection via Bidirectional Normalizing Flows (2403.10550v1)

Published 13 Mar 2024 in cs.LG, cs.AI, and cs.CR

Abstract: With the rapid development of the Internet, various types of anomaly traffic are threatening network security. We consider the problem of anomaly network traffic detection and propose a three-stage anomaly detection framework using only normal traffic. Our framework can generate pseudo anomaly samples without prior knowledge of anomalies to achieve the detection of anomaly data. Firstly, we employ a reconstruction method to learn the deep representation of normal samples. Secondly, these representations are normalized to a standard normal distribution using a bidirectional flow module. To simulate anomaly samples, we add noises to the normalized representations which are then passed through the generation direction of the bidirectional flow module. Finally, a simple classifier is trained to differentiate the normal samples and pseudo anomaly samples in the latent space. During inference, our framework requires only two modules to detect anomalous samples, leading to a considerable reduction in model size. According to the experiments, our method achieves the state of-the-art results on the common benchmarking datasets of anomaly network traffic detection. The code is given in the https://github.com/ZxuanDang/ATD-via-Flows.git

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. O. Salman, I. H. Elhajj, A. Chehab, and A. Kayssi, “A machine learning based framework for iot device identification and abnormal traffic detection,” Transactions on Emerging Telecommunications Technologies, vol. 33, no. 3, p. e3743, 2022.
  2. J. Niu, Y. Zhang, D. Liu, D. Guo, and Y. Teng, “Abnormal network traffic detection based on transfer component analysis,” in IEEE International Conference on Communications Workshops, pp. 1–6, 2019.
  3. M. Gao, L. Ma, H. Liu, Z. Zhang, Z. Ning, and J. Xu, “Malicious network traffic detection based on deep neural networks and association analysis,” Sensors, vol. 20, no. 5, p. 1452, 2020.
  4. Z. Li, Z. Qin, K. Huang, X. Yang, and S. Ye, “Intrusion detection using convolutional neural networks for representation learning,” in International Conference on Neural Information Processing, pp. 858–866, 2017.
  5. L. Yang, Y. Song, S. Gao, B. Xiao, and A. Hu, “Griffin: an ensemble of autoencoders for anomaly traffic detection in sdn,” in IEEE Global Communications Conference, pp. 1–6, 2020.
  6. Y. Zheng, Z. Dang, C. Peng, C. Yang, and X. Gao, “Multi-view multi-label anomaly network traffic classification based on mlp-mixer neural network,” arXiv preprint arXiv:2210.16719, 2022.
  7. V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM computing surveys (CSUR), vol. 41, no. 3, pp. 1–58, 2009.
  8. Z. Jadidi, V. Muthukkumarasamy, E. Sithirasenan, and K. Singh, “Flow-based anomaly detection using semisupervised learning,” in 2015 9th International Conference on Signal Processing and Communication Systems (ICSPCS), pp. 1–5, IEEE, 2015.
  9. M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, “Towards an unsupervised method for network anomaly detection in large datasets,” Computing and informatics, vol. 33, no. 1, pp. 1–34, 2014.
  10. Y. Shi and H. Shen, “Unsupervised anomaly detection for network traffic using artificial immune network,” Neural Computing and Applications, vol. 34, no. 15, pp. 13007–13027, 2022.
  11. C.-L. Li, K. Sohn, J. Yoon, and T. Pfister, “Cutpaste: Self-supervised learning for anomaly detection and localization,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9664–9674, 2021.
  12. V. Zavrtanik, M. Kristan, and D. Skočaj, “Draem-a discriminatively trained reconstruction embedding for surface anomaly detection,” in IEEE/CVF International Conference on Computer Vision, pp. 8330–8339, 2021.
  13. M. Ring, D. Schlör, D. Landes, and A. Hotho, “Flow-based network traffic generation using generative adversarial networks,” Computers & Security, vol. 82, pp. 156–172, 2019.
  14. R. Ghanavi, B. Liang, and A. Tizghadam, “Generative adversarial classification network with application to network traffic classification,” in 2021 IEEE Global Communications Conference (GLOBECOM), pp. 1–6, IEEE, 2021.
  15. A. Cheng, “Pac-gan: Packet generation of network traffic using generative adversarial networks,” in 2019 IEEE 10th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), pp. 0728–0734, IEEE, 2019.
  16. L. Bergman and Y. Hoshen, “Classification-based anomaly detection for general data,” arXiv preprint arXiv:2005.02359, 2020.
  17. I. Golan and R. El-Yaniv, “Deep anomaly detection using geometric transformations,” Advances in Neural Information Processing Systems, vol. 31, 2018.
  18. D. Hendrycks, M. Mazeika, S. Kadavath, and D. Song, “Using self-supervised learning can improve model robustness and uncertainty,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  19. T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,” arXiv preprint arXiv:1708.04552, 2017.
  20. D. P. Kingma and P. Dhariwal, “Glow: Generative flow with invertible 1x1 convolutions,” Advances in Neural Information Processing Systems, vol. 31, 2018.
  21. B. Cao, C. Li, Y. Song, Y. Qin, and C. Chen, “Network intrusion detection model based on cnn and gru,” Applied Sciences, vol. 12, no. 9, p. 4184, 2022.
  22. T. Saba, A. Rehman, T. Sadad, H. Kolivand, and S. A. Bahaj, “Anomaly-based intrusion detection system for iot networks through deep learning model,” Computers and Electrical Engineering, vol. 99, p. 107810, 2022.
  23. Z. Liu, Y. He, W. Wang, and B. Zhang, “Ddos attack detection scheme based on entropy and pso-bp neural network in sdn,” China Communications, vol. 16, no. 7, pp. 144–155, 2019.
  24. N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, “A deep learning approach to network intrusion detection,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41–50, 2018.
  25. A. Javaid, Q. Niyaz, W. Sun, and M. Alam, “A deep learning approach for network intrusion detection system,” in International Conference on Bio-inspired Information and Communications Technologies, pp. 21–26, 2016.
  26. W. Wang, Y. Sheng, J. Wang, X. Zeng, X. Ye, Y. Huang, and M. Zhu, “Hast-ids: Learning hierarchical spatial-temporal features using deep neural networks to improve intrusion detection,” IEEE Access, vol. 6, pp. 1792–1806, 2017.
  27. S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon, “Ganomaly: Semi-supervised anomaly detection via adversarial training,” in Asian Conference on Computer Vision, pp. 622–637, 2019.
  28. K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox, and P. Gehler, “Towards total recall in industrial anomaly detection,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328, 2022.
  29. J. Song, K. Kong, Y.-I. Park, S.-G. Kim, and S.-J. Kang, “Anoseg: anomaly segmentation network using self-supervised learning,” arXiv preprint arXiv:2110.03396, 2021.
  30. M. Yang, P. Wu, and H. Feng, “Memseg: A semi-supervised method for image surface defect detection using differences and commonalities,” Engineering Applications of Artificial Intelligence, vol. 119, p. 105835, 2023.
  31. A.-S. Collin and C. De Vleeschouwer, “Improved anomaly detection by training an autoencoder with skip connections on images corrupted with stain-shaped noise,” in International Conference on Pattern Recognition, pp. 7915–7922, 2021.
  32. E. Alhajjar, P. Maxwell, and N. Bastian, “Adversarial machine learning in network intrusion detection systems,” Expert Systems with Applications, vol. 186, p. 115782, 2021.
  33. Y. Peng, G. Fu, Y. Luo, J. Hu, B. Li, and Q. Yan, “Detecting adversarial examples for network intrusion detection system with gan,” in 2020 IEEE 11th International Conference on Software Engineering and Service Science (ICSESS), pp. 6–10, IEEE, 2020.
  34. J. Wang, J. Pan, I. AlQerm, and Y. Liu, “Def-ids: An ensemble defense mechanism against adversarial attacks for deep learning-based network intrusion detection,” in 2021 International Conference on Computer Communications and Networks (ICCCN), pp. 1–9, IEEE, 2021.
  35. B.-E. Zolbayar, R. Sheatsley, P. McDaniel, M. J. Weisman, S. Zhu, S. Zhu, and S. Krishnamurthy, “Generating practical adversarial network traffic flows using nidsgan,” arXiv preprint arXiv:2203.06694, 2022.
  36. M. Abdelaty, S. Scott-Hayward, R. Doriguzzi-Corin, and D. Siracusa, “Gadot: Gan-based adversarial training for robust ddos attack detection,” in 2021 IEEE Conference on Communications and Network Security (CNS), pp. 119–127, IEEE, 2021.
  37. M. R. Shahid, G. Blanc, H. Jmila, Z. Zhang, and H. Debar, “Generative deep learning for internet of things network traffic generation,” in 2020 IEEE 25th Pacific Rim International Symposium on Dependable Computing (PRDC), pp. 70–79, IEEE, 2020.
  38. S. K. Nukavarapu, M. Ayyat, and T. Nadeem, “Miragenet-towards a gan-based framework for synthetic network traffic generation,” in GLOBECOM 2022-2022 IEEE Global Communications Conference, pp. 3089–3095, IEEE, 2022.
  39. Y. Yin, Z. Lin, M. Jin, G. Fanti, and V. Sekar, “Practical gan-based synthetic ip header trace generation using netshare,” in Proceedings of the ACM SIGCOMM 2022 Conference, pp. 458–472, 2022.
  40. S. Hui, H. Wang, Z. Wang, X. Yang, Z. Liu, D. Jin, and Y. Li, “Knowledge enhanced gan for iot traffic generation,” in Proceedings of the ACM Web Conference 2022, pp. 3336–3346, 2022.
  41. L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using real nvp,” arXiv preprint arXiv:1605.08803, 2016.
  42. D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling, “Improved variational inference with inverse autoregressive flow,” Advances in Neural Information Processing Systems, vol. 29, 2016.
  43. I. Kobyzev, S. J. Prince, and M. A. Brubaker, “Normalizing flows: An introduction and review of current methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 11, pp. 3964–3979, 2020.
  44. H. Deng and X. Li, “Anomaly detection via reverse distillation from one-class embedding,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9737–9746, 2022.
  45. S. Akcay, D. Ameln, A. Vaidya, B. Lakshmanan, N. Ahuja, and U. Genc, “Anomalib: A deep learning library for anomaly detection,” 2022.
  46. N. A. Ahuja, I. Ndiour, T. Kalyanpur, and O. Tickoo, “Probabilistic modeling of deep features for out-of-distribution and adversarial detection,” arXiv preprint arXiv:1909.11786, 2019.
  47. J. Yu, Y. Zheng, X. Wang, W. Li, Y. Wu, R. Zhao, and L. Wu, “Fastflow: Unsupervised anomaly detection and localization via 2d normalizing flows,” arXiv preprint arXiv:2111.07677, 2021.
  48. T. Defard, A. Setkov, A. Loesch, and R. Audigier, “Padim: a patch distribution modeling framework for anomaly detection and localization,” in International Conference on Pattern Recognition Workshops, pp. 475–489, 2021.
  49. G. Wang, S. Han, E. Ding, and D. Huang, “Student-teacher feature pyramid matching for unsupervised anomaly detection. arxiv 2021,” arXiv preprint arXiv:2103.04257.
  50. D. Gudovskiy, S. Ishizaka, and K. Kozuka, “Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows,” in IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 98–107, 2022.
  51. A. H. Lashkari, G. Draper-Gil, M. S. I. Mamun, and A. A. Ghorbani, “Characterization of tor traffic using time based features,” in International Conference on Information Systems Security and Privacy, pp. 253–262, 2017.
  52. G. Draper-Gil, A. H. Lashkari, M. S. I. Mamun, and A. A. Ghorbani, “Characterization of encrypted and vpn traffic using time-related features,” in International Conference on Information Systems Security and Privacy, pp. 407–414, 2016.
  53. D. Community, “Datacon open dataset-datacon2020-encrypted malicious traffic dataset direction open dataset.” https://datacon.qianxin.com/opendata/openpage?resourcesId=6, 2021-11-11.
  54. M. Lotfollahi, M. Jafari Siavoshani, R. Shirali Hossein Zade, and M. Saberian, “Deep packet: A novel approach for encrypted traffic classification using deep learning,” Soft Computing, vol. 24, no. 3, pp. 1999–2012, 2020.
  55. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255, Ieee, 2009.
  56. G. Pang, C. Shen, L. Cao, and A. V. D. Hengel, “Deep learning for anomaly detection: A review,” ACM computing surveys (CSUR), vol. 54, no. 2, pp. 1–38, 2021.
  57. S. Xu, M. Marwah, M. Arlitt, and N. Ramakrishnan, “Stan: Synthetic network traffic generation with generative neural models,” in Deployable Machine Learning for Security Defense: Second International Workshop, MLHat 2021, Virtual Event, August 15, 2021, Proceedings 2, pp. 3–29, Springer, 2021.
  58. G. Aceto, D. Ciuonzo, A. Montieri, and A. Pescapé, “Mobile encrypted traffic classification using deep learning: Experimental evaluation, lessons learned, and challenges,” IEEE Transactions on Network and Service Management, vol. 16, no. 2, pp. 445–458, 2019.
Citations (1)

Summary

We haven't generated a summary for this paper yet.

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.

Tweets

Sign up for free to view the 3 tweets with 0 likes about this paper.