Deep Anomaly Detection in Packet Payload (1912.02549v1)

Abstract: With the widespread adoption of cloud services, especially the extensive deployment of plenty of Web applications, it is important and challenging to detect anomalies from the packet payload. For example, the anomalies in the packet payload can be expressed as a number of specific strings which may cause attacks. Although some approaches have achieved remarkable progress, they are with limited applications since they are dependent on in-depth expert knowledge, e.g., signatures describing anomalies or communication protocol at the application level. Moreover, they might fail to detect the payload anomalies that have long-term dependency relationships. To overcome these limitations and adaptively detect anomalies from the packet payload, we propose a deep learning based framework which consists of two steps. First, a novel feature engineering method is proposed to obtain the block-based features via block sequence extraction and block embedding. The block-based features could encapsulate both the high-dimension information and the underlying sequential information which facilitate the anomaly detection. Second, a neural network is designed to learn the representation of packet payload based on Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN). Furthermore, we cast the anomaly detection as a classification problem and stack a Multi-Layer Perception (MLP) on the above representation learning network to detect anomalies. Extensive experimental results on three public datasets indicate that our model could achieve a higher detection rate, while keeping a lower false positive rate compared with five state-of-the-art methods.

• The paper presents a novel deep anomaly detection framework that utilizes block-based feature extraction to represent packet payloads effectively.
• It combines LSTM and CNN to capture both temporal and spatial features, achieving a 99.12% detection rate and a 0.22% false positive rate on the CSIC 2010 dataset.
• The study demonstrates that hyper-parameter tuning, particularly sliding block length, significantly influences performance and suggests potential applications in other anomaly detection areas.

Deep Anomaly Detection in Packet Payload

Introduction

The paper "Deep Anomaly Detection in Packet Payload" presents a novel framework for detecting anomalies within network packet payloads using deep learning techniques. The work addresses the challenge of identifying malicious patterns embedded in the packet data, particularly those with long-term dependencies, which are often overlooked by traditional methods that rely heavily on expert knowledge or signature-based detection.

Proposed Framework

Feature Engineering

The framework introduces a feature engineering method that constructs block-based features for representing packet payloads. Utilizing a sliding block mechanism, sequences of packet payload are converted into shorter subsequences or "blocks", which are then encoded into low-dimensional embedded vectors to encapsulate both sequential and high-dimensional information.

Figure 1: An example for the process of block sequence construction. With a sliding block of length 3 and a fixed stride, the blocks extracted from a packet payload form a block sequence.

Anomaly Detection Model

Built upon Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN), the model extracts temporal dependencies and spatial features, facilitating the identification of both short-term and long-term anomalous byte distributions within the payloads. A Multi-Layer Perceptron (MLP) classifier is then applied to categorize the payloads based on the learned representations.

Figure 2: An illustration of our proposed anomaly detection model.

Experimental Evaluation

The framework's efficacy is demonstrated through experiments on public datasets, notably CSIC 2010, CICIDS 2017, and ISCX 2012. The results indicate significant improvements over classical methods such as SVM and RF, in terms of detection rates (DR) and false positive rates (FPR).

Comparative Analysis

Figure 3: The Detection Rates and False Positive Rates of the related works and our experiment.

The deep learning-based approach, notably the proposed LSTM-CNN architecture, outperformed existing models with a DR of 99.12% and an FPR of 0.22% on the CSIC 2010 dataset. Furthermore, the block-based feature extraction method significantly enhanced detection accuracy across various models.

Parameter Influence

Investigations into hyper-parameter settings revealed that factors such as the sliding block length and the number of high-frequency items in the dictionary play crucial roles in model performance, impacting both DR and FPR.

Figures showcasing the parameter impact are presented in the form of Detection Rates (Figure 4) and False Positive Rates (Figure 5).

Conclusion

The proposed deep learning framework effectively addresses the limitations of traditional packet payload anomaly detection by leveraging block-based feature extraction and advanced neural network architectures. Future directions include extending the approach to other forms of anomalies, such as those found in video surveillance, suggesting a broad applicability of the feature extraction and detection model in more complex anomaly detection scenarios.