Pack-ALM: DL for Packed Executable Detection

• Pack-ALM is a deep learning-based system that distinguishes authentic machine code from pseudo instructions generated by packing and obfuscation.
• It employs a RoBERTa architecture with a novel structured instruction MLM (siMLM) objective to enhance detection accuracy and resist adversarial evasion.
• Extensive evaluation shows 12–24% higher F1 scores and over 99% accurate program-level detection compared to traditional entropy and signature-based methods.

Pack-ALM is a deep learning-based system for detecting packed executables in large-scale malware analysis and antivirus workflows. It is designed to overcome limitations associated with traditional feature-based detection approaches, including vulnerability to adversarial evasion and limited adaptability to novel packing techniques. By leveraging a pre-trained assembly LLM that identifies pseudo instructions—non-legitimate instructions resulting from packed or obfuscated code—Pack-ALM achieves robust packed binary identification and adversarial resilience, as demonstrated in extensive empirical evaluation (Li et al., 19 Sep 2025).

1. Rationale and Conceptual Foundation

The impetus for Pack-ALM arises from two critical flaws in feature-based packed executable detection. First, adversarial samples readily evade entropy-based or heuristic rules by manipulating binary characteristics, such as employing low-entropy packing schemes or modifying metadata. Second, the meticulous manual design of features is costly and often unsustainable due to the rapid evolution of packers.

Pack-ALM reconceptualizes the detection task through a linguistically-inspired lens: distinguishing between “real instructions” (authentic machine code logic) and “pseudo instructions” (spurious instructions emitted by disassembly of packed or native data). This draws directly on natural language processing techniques for separating real words from pseudowords and reframes packer detection as a deep learning problem.

2. Preprocessing and Tokenization Strategy

Pack-ALM begins with linear disassembly of executable binaries, targeting both code regions and data sections. Due to packing artifacts, genuine instructions and packed data tend to intermingle, producing structural anomalies.

Instruction normalization is performed by abstracting immediate operands:

• Operands are mapped to canonical tokens such as [const], [mem_normal], [pad_abnormal] based on memory range checks.
• This abstraction mitigates vocabulary explosion and sharpens the model’s sensitivity to pattern regularities.
• Special delimiter tokens [SOS] and [EOS] are introduced to mark instruction boundaries.

The preprocessed data is segmented into token windows, typically with a maximum length of 512 tokens, which serve as direct input to the LLM.

3. Model Architecture and Pre-Training Task

Pack-ALM employs a RoBERTa-based architecture augmented with a structured instruction MLM (“siMLM”) objective:

• The siMLM task replaces conventional random token masking with mutually exclusive masking at the instruction component level.
• For each instruction (e.g., mov eax, ebx), the model masks either the opcode or specific operands, maintaining coherence at the instruction boundary.
• The denoising objective is defined as:

$$L_\mathrm{siMLM} = -\sum_{t_i \in m(R)} \log p(\hat{t}_i)$$

where $m(R)$ is the set of masked tokens in window $R$.

This approach compels the model to learn both syntactic and semantic relationships characterizing authentic assembly instructions, while remaining attuned to irregularities typical of packed regions.

4. Fine-Tuning Procedure and Packed Region Classification

During fine-tuning, the pre-trained model is exposed to labeled datasets including packed and non-packed executables. For each input window, the output is classified as one of:

• Real instruction (legitimate code),
• Native data (pseudo instructions with relative organization),
• Packed data (pseudo instructions arising from highly obfuscated regions).

Average pooling is performed across token embeddings per window, followed by dense classification layers. This enables not only binary detection (packed vs. non-packed), but also precise region localization, critical for downstream analyses such as targeted unpacking.

5. Comparative Evaluation with Established Methods

The effectiveness of Pack-ALM is established through rigorous benchmarking against prevailing industry tools and recent assembly LLMs:

Method/Class	Detection Robustness	Vulnerability to Evasion
Entropy-based (threshold)	Low (adversarial)	High
Signature-based heuristics	Moderate	Moderate/High
Generic ASM LLMs	Moderate	High (packed confusion)
Pack-ALM (siMLM approach)	High	Low

• Entropy-based detection is critiqued for fixed threshold sensitivity and poor performance on low-entropy packed samples.
• Signature-based approaches are unable to generalize to unseen packing methods.
• Peer assembly LLMs (XDA, PalmTree, jTrans) are evaluated; their lack of specialized training for packed code degrades their classification accuracy.
• Pack-ALM achieves 12–24% higher F1 scores in region identification and sustains detection accuracy even on low-entropy, adversarially crafted samples.

6. Experimental Results and Performance Metrics

Experiments spanning 37,000 packed and non-packed executables demonstrate:

• Token-level classification: Reliable separation of real instructions from pseudo instructions, with further subclassing into native and packed data.
• Adversarial low-entropy packing: Maintained high accuracy compared to traditional methods, which either over-trigger on benign data or miss packed samples.
• Program-level detection: True positive/negative rates exceeding 99% in realistic malware/binary datasets.
• False negative rates: Markedly lower than signature-based tools (e.g., DIE, PackGenome) and ML-based systems (e.g., PackHero), substantiating the robustness of the approach.

7. Future Directions

Several research extensions for Pack-ALM are identified:

• Architecture generalization: Tailoring models to additional instruction set architectures (e.g., ARM, IoT) by expanding training corpora or devising universal representations.
• Packer attribution: Enhancing the model for not just packed region detection but identification of specific packer families, potentially in synergy with signature-based heuristics.
• Integration with language modeling frameworks: Exploring connections to LLMs and Retrieval-Augmented Generation (RAG) for context-aware, cross-task binary analysis.

A plausible implication is that Pack-ALM’s foundation enables more adaptive, resilient, and automatable malware analysis pipelines in the face of rapidly evolving adversarial packing strategies.

---

Pack-ALM marks a significant advancement in the application of deep language modeling to binary analysis, evidencing high performance and adversarial robustness in packed executable detection, and setting a foundation for further research in context-driven and cross-architecture malware defense (Li et al., 19 Sep 2025).