EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models (1804.04637v2)

Abstract: This paper describes EMBER: a labeled benchmark dataset for training machine learning models to statically detect malicious Windows portable executable files. The dataset includes features extracted from 1.1M binary files: 900K training samples (300K malicious, 300K benign, 300K unlabeled) and 200K test samples (100K malicious, 100K benign). To accompany the dataset, we also release open source code for extracting features from additional binaries so that additional sample features can be appended to the dataset. This dataset fills a void in the information security machine learning community: a benign/malicious dataset that is large, open and general enough to cover several interesting use cases. We enumerate several use cases that we considered when structuring the dataset. Additionally, we demonstrate one use case wherein we compare a baseline gradient boosted decision tree model trained using LightGBM with default settings to MalConv, a recently published end-to-end (featureless) deep learning model for malware detection. Results show that even without hyper-parameter optimization, the baseline EMBER model outperforms MalConv. The authors hope that the dataset, code and baseline model provided by EMBER will help invigorate machine learning research for malware detection, in much the same way that benchmark datasets have advanced computer vision research.

• The paper introduces EMBER, an open dataset that provides 1.1 million annotated PE file features for static malware detection.
• It details a systematic approach with parsed and format-agnostic features to address legal, logistic, and technical challenges in malware benchmarking.
• Baseline experiments using LightGBM achieved a ROC AUC over 0.99911 with a detection rate surpassing 98% at a 1% false positive rate.

EMBER: A Dataset for Static Malware Detection

The paper "EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models" introduces a significant resource for advancing the paper of machine learning in malware detection. Created by Hyrum S. Anderson and Phil Roth from Endgame, Inc., EMBER addresses a critical need in the cybersecurity domain by providing an extensive, open dataset specifically designed for use in static analysis of Windows portable executable (PE) files.

Dataset Composition and Structure

EMBER comprises features extracted from a large corpus of 1.1 million binary files. The dataset is partitioned into 900,000 samples for training and 200,000 samples for testing. Each partition consists of balanced sets of malicious, benign, and unlabeled files, thereby facilitating a broad range of machine learning applications, including supervised and semi-supervised learning.

The dataset is structured around eight feature groups encapsulating parsed and format-agnostic characteristics of PE files. Parsed features include detailed elements such as header information, imported and exported functions, and section metadata. In contrast, format-agnostic features provide insights into file structure through byte histograms, byte entropy histograms, and basic string analysis.

Experimental Evaluation

A baseline experiment is conducted using LightGBM—a gradient-boosted decision tree model—trained with little parameter tuning. This model achieves an impressive ROC AUC exceeding 0.99911, demonstrating a detection rate surpassing 98% at a 1% false positive rate. This result underscores the potential of structured feature sets over raw data inputs for accurate malware classification.

Further experimentation involved MalConv, a deep learning architecture designed for direct byte input, which produced a slightly lower ROC AUC of 0.99821 on the EMBER dataset. Despite having a larger model size of approximately 1M parameters, MalConv’s performance suggests current limitations in featureless deep learning methods compared to models utilizing domain-specific parsing.

Addressing Challenges in Malware Detection

The authors elucidate several obstacles in establishing benchmark datasets for malware detection, particularly the legal and logistical constraints surrounding the sharing of benign binaries. EMBER circumvents these issues by carefully structuring and anonymizing feature data while ensuring compliance with copyright laws.

Implications and Future Directions

By offering a public, comprehensive dataset, EMBER fills a longstanding gap in malware detection research. Its release enables rigorous benchmarking of new algorithms and architectures, fostering innovation in areas such as adversarial machine learning, feature engineering, and semi-supervised learning.

Future work may explore enhancements through hyper-parameter optimization, integration of additional feature types, and comparative analysis with emerging end-to-end learning models. EMBER's extensibility is poised to facilitate these inquiries, promoting ongoing advancements in both practical implementations and theoretical understanding of machine learning for cybersecurity.

In summary, the EMBER dataset represents a foundational step in standardizing research tools in malware detection, setting a benchmark for developing and evaluating state-of-the-art machine learning methodologies in this crucial area.