Binary Block Masking in Malware Detection

• Binary Block Masking (BBM) is a deterministic method that systematically occludes blocks within executable bytes to reduce adversarial impact.
• It slides a fixed-length mask across raw-byte executables, classifying each masked variant independently and aggregating results through thresholded voting.
• By integrating masked training and optimized hyperparameters, BBM maintains high clean accuracy while countering functionality-preserving adversarial payloads.

Searching arXiv for the specified paper and closely related malware-robustness work. Binary Block Masking (BBM) is a deterministic, stride-based occlusion strategy for hardening end-to-end malware detectors against functionality-preserving content injection. In ByteShield, BBM operates on raw-byte executables by sliding a fixed-length block mask across the binary, classifying each masked variant independently, and aggregating the resulting predictions through thresholded voting. The method is designed to suppress the influence of injected adversarial payloads by ensuring systematic coverage of the file; when the payload is fully or partially occluded in one or more masked variants, the final vote can recover the correct label. The approach was introduced as the core defense mechanism in "ByteShield: Adversarially Robust End-to-End Malware Detection through Byte Masking" (Gibert et al., 10 Dec 2025).

1. Formal setting and decision rule

BBM is defined for an input executable represented as a sequence of raw bytes $x \in \{0, \ldots, 255\}^n$ of length $n$, together with a malware detector $f : \{0, \ldots, 255\}^n \rightarrow [0,1]$ that outputs a maliciousness score (Gibert et al., 10 Dec 2025). A binary prediction is obtained by thresholding at $\tau = 0.5$:

$y = 1\{f(x) \ge \tau\}.$

The adversarial setting considered in ByteShield assumes that an attacker crafts $x'$ by inserting a functionality-preserving adversarial payload of length $L_a \ge 1$ into a contiguous region $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$, or split across multiple regions, while maintaining functionality $x' \equiv x$ (Gibert et al., 10 Dec 2025). The attack objective is to drive $f(x')$ below $n$0 under a budget constraint on the number and locations of modified bytes.

Within this setup, BBM generates multiple masked versions of the input and applies the base detector to each one. If enough masked variants produce a malicious decision, the file is declared malicious. This makes the final classification depend not on a single attacked representation, but on a set of systematically occluded representations. A plausible implication is that BBM changes the threat surface from one-shot evasion to evasion under repeated, structured perturbation of the observed byte sequence.

2. Mask construction and byte-level replacement

BBM uses a deterministic block masking operator that replaces, rather than deletes, a contiguous block of bytes with a special masking token (Gibert et al., 10 Dec 2025). Let $n$1 denote the block length in bytes, $n$2 the stride in bytes, and $n$3 the start index for the $n$4-th mask. The number of masked variants is

$n$5

with

$n$6

The masking operator $n$7 replaces bytes $n$8 by a special token $n$9:

$f : \{0, \ldots, 255\}^n \rightarrow [0,1]$0

In ByteShield, the masking token is a PAD token implemented as the integer $f : \{0, \ldots, 255\}^n \rightarrow [0,1]$1 in the byte vocabulary and mapped to the zero embedding vector; this token is not updated during training (Gibert et al., 10 Dec 2025). Masking therefore preserves positional structure and sequence length, since bytes are replaced rather than removed.

The mask length and stride can be parameterized as percentages of file length:

$f : \{0, \ldots, 255\}^n \rightarrow [0,1]$2

with $f : \{0, \ldots, 255\}^n \rightarrow [0,1]$3 to ensure overlap (Gibert et al., 10 Dec 2025). This overlap is central to the design, because it yields repeated, partially redundant occlusion of neighboring regions rather than sparse, disjoint coverage.

3. Thresholded voting and inference procedure

For inference, BBM evaluates the base detector on each masked variant:

$f : \{0, \ldots, 255\}^n \rightarrow [0,1]$4

These scores are thresholded at $f : \{0, \ldots, 255\}^n \rightarrow [0,1]$5 to produce binary decisions

$f : \{0, \ldots, 255\}^n \rightarrow [0,1]$6

The final classification is obtained by a thresholded vote:

$f : \{0, \ldots, 255\}^n \rightarrow [0,1]$7

where $f : \{0, \ldots, 255\}^n \rightarrow [0,1]$8 and $f : \{0, \ldots, 255\}^n \rightarrow [0,1]$9 is the minimum number of masked variants that must vote malicious for the sample to be declared malware (Gibert et al., 10 Dec 2025). In ByteShield, $\tau = 0.5$0 is chosen on validation to balance clean FPR and adversarial robustness, and typical robust settings are $\tau = 0.5$1 or $\tau = 0.5$2.

The implementation can be summarized in the following terms given in the ByteShield description.

Quantity	Definition
Mask starts	$\tau = 0.5$3
Number of masks	$\tau = 0.5$4
Variant decision	$\tau = 0.5$5
Final decision	$\tau = 0.5$6

A critical failure mode is explicit: if $\tau = 0.5$7, no input can be labeled malicious (Gibert et al., 10 Dec 2025). This is not a subtle calibration issue but a hard parameter inconsistency.

4. Robustness rationale and occlusion analysis

The robustness rationale of BBM is based on coverage and occlusion. Deterministic stride-based masking ensures systematic coverage of the file: for every start $\tau = 0.5$8, BBM occludes a contiguous interval $\tau = 0.5$9 (Gibert et al., 10 Dec 2025). If the adversarial payload is contiguous, has length $y = 1\{f(x) \ge \tau\}.$0, and $y = 1\{f(x) \ge \tau\}.$1, then at least one masked variant fully occludes the payload provided the mask is applied at all starts, that is, when $y = 1\{f(x) \ge \tau\}.$2.

For a payload interval $y = 1\{f(x) \ge \tau\}.$3, a mask starting at $y = 1\{f(x) \ge \tau\}.$4 fully occludes $y = 1\{f(x) \ge \tau\}.$5 iff

$y = 1\{f(x) \ge \tau\}.$6

Equivalently,

$y = 1\{f(x) \ge \tau\}.$7

Among stride-aligned starts $y = 1\{f(x) \ge \tau\}.$8, the number of fully occluding masks for a given payload start $y = 1\{f(x) \ge \tau\}.$9 is

$x'$0

where $x'$1 counts how many multiples of $x'$2 lie in $x'$3 (Gibert et al., 10 Dec 2025). This yields the bounds

$x'$4

for $x'$5, and $x'$6 if $x'$7.

Partial occlusion is characterized through overlap. Overlap occurs when

$x'$8

that is, when

$x'$9

Its stride-count is approximately

$L_a \ge 1$0

These expressions explain why larger $L_a \ge 1$1 and smaller $L_a \ge 1$2 increase both full- and partial-occlusion counts (Gibert et al., 10 Dec 2025). With unlimited compute and $L_a \ge 1$3, BBM provides a security argument for contiguous payloads: if $L_a \ge 1$4, there exists at least one masked variant that fully occludes the payload, eliminating its influence and certifying the existence of at least one “correct” vote. In practical settings, $L_a \ge 1$5 reduces compute while retaining multiple partially or fully occluding variants.

ByteShield contrasts this mechanism with randomized and (de)randomized smoothing. Randomized deletion or masking applies per-byte randomness uniformly, which removes both adversarial and discriminative malicious bytes and can collapse long-range structure in CNNs; chunk-based voting is vulnerable to large payloads that dominate a sufficient fraction of chunks (Gibert et al., 10 Dec 2025). By contrast, BBM uses deterministic coverage with overlap and relies on learned tolerance to occlusion.

5. Training-time masking and architectural context

ByteShield uses masking at both training and inference time. During training, it randomly samples a mask start $L_a \ge 1$6 for each example and applies a single mask:

$L_a \ge 1$7

then minimizes

$L_a \ge 1$8

The model is trained to predict the original label from masked inputs (Gibert et al., 10 Dec 2025).

This masked training is described as crucial. Without masked training, clean F1 decreases as $L_a \ge 1$9 grows; with masked training, ByteShield maintains high accuracy across mask sizes and preserves clean accuracy even for large $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$0 (Gibert et al., 10 Dec 2025). The reason given is that training on masked inputs teaches the detector to be stable under occlusion.

Experiments in ByteShield use MalConv as the primary backbone: bytes $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$1–$[u, u+L_a-1] \subset \{0, \ldots, n-1\}$2 plus PAD$[u, u+L_a-1] \subset \{0, \ldots, n-1\}$3 are embedded, passed through a gated convolution, global pooling, and a feed-forward head (Gibert et al., 10 Dec 2025). Supplementary results show similar robustness gains for AvastConv and NGramConv, and the method is described as agnostic to the backbone. This suggests that BBM is principally a masking-and-aggregation scheme rather than an architecture-specific defense.

Hyperparameters are specified as follows: $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$4, $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$5 with $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$6, and $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$7, typically $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$8 or $[u, u+L_a-1] \subset \{0, \ldots, n-1\}$9 (Gibert et al., 10 Dec 2025). For $x' \equiv x$0, $x' \equiv x$1, and $x' \equiv x$2, the number of masks is approximately $x' \equiv x$3; for $x' \equiv x$4, it is approximately $x' \equiv x$5.

6. Empirical behavior on clean, adversarial, and temporal evaluations

ByteShield evaluates BBM on EMBER and BODMAS (Gibert et al., 10 Dec 2025). EMBER contains 800k binaries, with 400k benign and 400k malicious samples, using a random 80/10/10 split. BODMAS contains 77,142 benign and 57,293 malicious samples timestamped 2019–2020 and is used for temporal robustness.

On clean data, the reported results emphasize the importance of masked training. Without masked training, clean F1 decreases as $x' \equiv x$6 grows; for example, $x' \equiv x$7 (Gibert et al., 10 Dec 2025). With masked training, ByteShield maintains strong clean performance. The reported examples include:

Setting	Reported metrics
$x' \equiv x$8	Accuracy 98.27, TPR 0.9807, FPR 0.0150, F1 0.9837
$x' \equiv x$9	Accuracy 98.16, TPR 0.9863, FPR 0.0239, F1 0.9823

Across the full sweep, ByteShield is reported to achieve higher TPR and lower FPR than (de)randomized smoothing methods, which typically show $f(x')$0 and $f(x')$1 due to chunking-induced information loss (Gibert et al., 10 Dec 2025).

Adversarial robustness is evaluated under padding, shift, code caves, and section injection, with payload sizes at 10%, 20%, 50%, and 100% of the original file (Gibert et al., 10 Dec 2025). Payloads are initialized from benign bytes and optimized using Nevergrad’s DoubleFastGADiscreteOnePlusOne with 3,000 evaluations, reflecting a gradient-free black-box setting.

The reported findings are sharply differentiated. Vanilla MalConv drops from approximately 98% clean accuracy to approximately 20% with only a 10% payload and is near 0% at 50%–100% payload sizes (Gibert et al., 10 Dec 2025). RSDel fails because it indiscriminately removes both malicious and benign signal. DRS withstands small payloads of 10%–20% but fails at 50%–100%, where the majority of chunks are influenced. ByteShield is reported as unmatched at moderate-to-large payloads, especially with $f(x')$2 and $f(x')$3; for distributed payloads such as code caves and section injection, partial occlusion of influential fragments is described as sufficient to suppress adversarial influence and allow thresholded voting to recover the correct label (Gibert et al., 10 Dec 2025).

On BODMAS, temporal robustness is measured by monthly F1 and Area Under Time (AUT). ByteShield with $f(x')$4 achieves AUT 0.9265, outperforming MalConv at 0.9160, RSDel at 0.8955, and DRS variants at 0.7818–0.8693 (Gibert et al., 10 Dec 2025). Reported monthly F1 values include 0.9367 in 09/2019 and 0.9178 in 08/2020.

7. Computational profile, limitations, and relation to prior defenses

The inference complexity of BBM is

$f(x')$5

for evaluating the detector $f(x')$6 times, plus

$f(x')$7

for constructing the masks (Gibert et al., 10 Dec 2025). On an NVIDIA 4090, the reported timings are 0.0021 s/example for MalConv single-pass, 0.1923 s/example for RSDel with 100 randomized variants, and 0.0064 s for ByteShield at $f(x')$8 up to 0.0389 s at $f(x')$9 (Gibert et al., 10 Dec 2025). Training time per epoch is reported as similar to MalConv, approximately 3h 12m, because only one masked version per batch example is used.

Ablation findings characterize the design trade-offs. Larger $n$00 improves adversarial robustness with negligible clean degradation when masked training is used. Larger $n$01 reduces compute and slightly reduces TPR while slightly improving FPR; if $n$02 becomes extreme, $n$03 may become too small and robustness may degrade. Increasing $n$04 lowers FPR but reduces adversarial robustness, with $n$05–$n$06 identified as a good trade-off (Gibert et al., 10 Dec 2025).

The stated limitations are equally specific. Distributed payloads across many small, non-contiguous regions reduce the chance that any single mask fully occludes all influential fragments, so formal guarantees weaken under fragmentation and stride $n$07 (Gibert et al., 10 Dec 2025). Adaptive, mask-aware optimization may target regions least likely to be occluded for a given $n$08 or optimize content that remains influential under masking. Clean accuracy trade-offs can be significant without masked training. Additional failure modes include $n$09 and small $n$10 in the presence of large payloads.

In relation to prior defenses, ByteShield positions BBM against randomized smoothing and (de)randomized smoothing. Randomized smoothing via random deletion or masking is described as offering slight robustness improvements only at small budgets while harming clean inputs and collapsing on large, structured payloads. Chunk-based (de)randomized smoothing provides formal guarantees in chunk space, but its majority vote is brittle when large payloads influence at least 50% of chunks (Gibert et al., 10 Dec 2025). BBM differs by combining deterministic coverage, learned occlusion tolerance through masked training, and thresholded voting over overlapping masked variants. A plausible implication is that BBM reframes robustness from random perturbation averaging to structured occlusion coverage over the executable’s byte space.