Improving Scalability of Contrast Pattern Mining for Network Traffic Using Closed Patterns

Abstract: Contrast pattern mining (CPM) aims to discover patterns whose support increases significantly from a background dataset compared to a target dataset. CPM is particularly useful for characterising changes in evolving systems, e.g., in network traffic analysis to detect unusual activity. While most existing techniques focus on extracting either the whole set of contrast patterns (CPs) or minimal sets, the problem of efficiently finding a relevant subset of CPs, especially in high dimensional datasets, is an open challenge. In this paper, we focus on extracting the most specific set of CPs to discover significant changes between two datasets. Our approach to this problem uses closed patterns to substantially reduce redundant patterns. Our experimental results on several real and emulated network traffic datasets demonstrate that our proposed unsupervised algorithm is up to 100 times faster than an existing approach for CPM on network traffic data [2]. In addition, as an application of CPs, we demonstrate that CPM is a highly effective method for detection of meaningful changes in network traffic.