MalConv Architecture: Byte-Level Malware Detection

• MalConv architecture is a deep neural model that detects malware from raw byte sequences without relying on manual feature engineering.
• It employs gated 1-D convolution with parallel base and gate filters and uses constant-memory max-pooling to efficiently process extremely long files.
• The integration of content-dependent channel gating enhances contextual feature interactions, yielding higher accuracy and significant reductions in GPU memory and compute time.

MalConv refers to a family of neural network architectures developed to detect malware directly from raw byte sequences of executable files, without domain-specific preprocessing or manual feature engineering. These models are specifically designed to address the extreme length and structureless nature of binary executables, processing inputs of up to hundreds of megabytes (hundreds of millions of bytes) with linear computational and memory requirements. The architectural innovations center on byte-level embedding, gated convolutional feature extraction, efficient temporal pooling, and, in later variants, content-dependent channel attention mechanisms.

1. Problem Definition and Input Representation

MalConv architectures treat an executable file as a long sequence of raw bytes $x = [x_1, x_2, ..., x_N]^T$, where each $x_i \in \{0, 1, ..., 255\}$ and $N$ can be as large as $2 \cdot 10^8$ (for files up to $\sim$271 MB). No domain-specific structure from the PE file format, assembly, or other syntactic levels is used.

Each possible byte value, plus an EOF symbol in later variants (forming a 257-symbol alphabet), is mapped to a low-dimensional real-valued vector via an embedding matrix $E \in \mathbb{R}^{K \times d}$, with $K=256$ or $257$ and $d=8$. For input sequence $x$, the embedding produces $e_i = E[x_i] \in \mathbb{R}^d$, and the model input is the matrix $E(x) \in \mathbb{R}^{N \times d}$.

2. Gated 1-D Convolutional Feature Extraction

A single layer of 1-D convolutional filters is applied to the byte embeddings, with two parallel convolutions: a "base" conv and a "gate" conv. In the original MalConv (Raff et al., 2017), both convolutions use $F = 128$ filters, kernel width $W = 500$, stride $s = 500$, and zero padding to ensure each output covers non-overlapping 500-byte regions. MalConv2 (Raff et al., 2020) increases the number of filters to $C = 256$, shrinks the kernel width to $W=256$, and uses a finer stride of $s=64$.

For input embedding $E(x)$:

• The base convolution outputs $$H = \mathrm{Conv}(E(x); W_h) + b_h \in \mathbb{R}^{T \times F}$$.
• The gate convolution outputs $$G = \mathrm{Conv}(E(x); W_g) + b_g \in \mathbb{R}^{T \times F}$$.

Where $T = \lceil N/s \rceil$ is the number of convolutional windows. The gating mechanism combines these via pointwise multiplication of the base conv output with a sigmoid nonlinearity applied to the gate conv:

$Y_{t,f} = H_{t,f} \cdot \sigma(G_{t,f})$

No additional nonlinearity is applied.

Architecture	Filters (F or C)	Kernel Width (W or $W_k$)	Stride (s or S)
MalConv	128	500	500
MalConv2	256	256	64

3. Temporal Aggregation and Constant-Memory Max-Pooling

Temporal aggregation is achieved via global max-pooling, collapsing the T-step convolutional output per channel to a fixed-size vector:

$z_f = \max_{1 \leq t \leq T} Y_{t, f}$

Max-pooling is favored over average-pooling because malware-relevant signatures may be highly localized to specific byte regions and would be diluted by averaging.

MalConv2 introduces a fixed-memory temporal max-pooling algorithm that scans the convolutional output in overlapping chunks (of length $3W_k$, step $2W_k$). For each filter/channel, the maximum activation (and index) is tracked per chunk and then compared globally, reducing overall activation storage requirements from $O(C \cdot T')$ to $O(C)$, where $T'$ is the reduced sequence length post-convolution. The two-phase algorithm (chunk-wise winner finding without gradients, followed by a small tensor backpropagation step) enables training on sequences exceeding 100 million bytes using under 2.2 GB of GPU memory.

4. Attention and Channel Gating

While original MalConv lacked inter-channel or spatial attention, MalConv2 augments the model with Global Channel Gating (GCG). This mechanism introduces a content-dependent gating scalar for each time step and channel, allowing the network to model contextual interactions across distant regions.

For convolutional output $H \in \mathbb{R}^{T' \times C}$:

• Compute a global context vector $\bar{g} \in \mathbb{R}^{C}$ via a small "context" sub-network (e.g., 1x1 conv or fully-connected layer over pooled H).
• For each time step $t$, compute

$z = \tanh(W^{\top} \bar{g}) \in \mathbb{R}^C$

$\alpha_t = \sigma(h_t^{\top} z) \in (0,1)$

$h'_t = \alpha_t \cdot h_t$

In equation form:

$$\mathrm{GCG}(h_t; \bar{g}) = h_t \cdot \sigma(h_t^{\top} \tanh(W^{\top} \bar{g}))$$

This mechanism operates with $O(C)$ parameters and memory, functioning as an attention gate across the entire sequence, but with computation scaling linearly in input length.

5. Classification and Training Procedure

After pooling (and optional GCG gating), the fixed-size feature vector $z$ is processed by a classification head:

• Dropout with rate $p = 0.5$ is applied to $z$.
• The resulting vector is projected via a single fully-connected layer to a scalar logit:

$\ell = w^\top (\mathrm{Dropout}(z)) + b$

• The predicted label is $\hat{y} = \sigma(\ell)$, trained with binary cross-entropy loss:

$$\mathcal{L} = -[y \log \hat{y} + (1-y) \log (1-\hat{y})]$$

MalConv omits Batch Normalization due to empirically observed slow or hindered convergence, attributed to highly non-Gaussian, multi-modal pre-activation distributions. Instead, DeCov regularization ($\lambda \approx 0.1$) is employed on the dropout-masked pooled vector.

Optimization is conducted via stochastic gradient descent with Nesterov momentum 0.9, initial learning rate 0.01 (decayed by 0.95 per epoch), mini-batch size 256 (data-parallel across GPUs), and optional L2 weight decay ($1{\rm e}{-6}$). Training on 400K files converges in 3–5 epochs; larger datasets (up to 2M files) further improve generalization.

6. Empirical Performance and Resource Requirements

Performance on the Ember2018 dataset (up to 271 MB files) shows:

• MalConv (original, 2 MB files): Accuracy 91.27%, AUC 97.19%
• MalConv (full-file, fixed-pool): Accuracy 91.14%, AUC 97.29%
• MalConv2 with GCG: Accuracy 93.29%, AUC 98.04%

Memory and computational optimizations yield a 116× reduction in GPU memory usage (from 128 GB to ~1 GB for MalConv with fixed-pool) and a 25.8× speedup per byte (training 21 hr/epoch reduced to 1 hr/epoch). MalConv2 with GCG, which incurs higher compute for attention, reports 4 hr/epoch and ≈2.2 GB RAM.

7. Architectural Summary and Implementation

The end-to-end flow comprises:

1. Embedding: Raw bytes $x_i$ mapped to $e_i \in \mathbb{R}^d$.
2. Gated 1D Conv Block: Parallel convolutions with gating:

$H = \mathrm{Conv}(E(x); W_h)+b_h$

$G = \mathrm{Conv}(E(x); W_g)+b_g$

$Y = H \cdot \sigma(G)$

1. (Optional, MalConv2) GCG: Gated attention across channel-time using a global context vector.
2. Global max-pooling (constant-memory in MalConv2): $z_f = \max_t Y_{t, f}$
3. Dropout, fully connected layer, and classification via sigmoid output.

The MalConv family of models demonstrates the feasibility of end-to-end learnable malware detection at byte-level granularity using architectures optimized for extremely long sequences, introducing effective memory and compute reductions to process realistic executable sizes at scale. The adoption of gating and channel attention mechanisms allows for content-dependent feature interactions and efficient global context integration, yielding improved detection performance on large, diverse datasets (Raff et al., 2017, Raff et al., 2020).