ScanNBT: Natural Backdoor Scanner

• ScanNBT is a method that detects natural backdoor vulnerabilities in CodeLMs by scanning for trigger token sequences without relying on poisoned training data.
• It employs a multi-round trigger inversion scheme with trigger fixation and re-initialization to explore multiple local optima in the trigger space.
• Evaluations on models like CodeBERT and GPT-3.5 across tasks such as defect detection and code search demonstrate improved trigger diversity with comparable effectiveness.

ScanNBT is a post-training detection method for natural backdoor vulnerabilities in Code LLMs (CodeLMs). It was introduced as part of an empirical study that examined natural backdoors across multiple model architectures, tasks, and settings, and it is defined as a scanning method that enhances the exposure of natural backdoor vulnerabilities by introducing trigger fixation and re-initialization (Chen et al., 9 Jun 2026). In this setting, a natural backdoor denotes a backdoor-like behavior that arises in a normally trained model on clean data, without poisoned samples or malicious training manipulation, when naturally occurring token patterns or trigger-like features induce systematic target predictions or outputs (Chen et al., 9 Jun 2026). ScanNBT is therefore neither a training-time defense nor a model purification method; it is a vulnerability-exposure procedure that searches for effective trigger candidates embedded in a trained CodeLM’s behavior (Chen et al., 9 Jun 2026).

1. Position within natural backdoor analysis

The method is situated in a threat model where CodeLMs used for defect detection, code search, code summarization, and code repair may exhibit natural backdoor vulnerabilities despite standard training on clean datasets (Chen et al., 9 Jun 2026). The paper distinguishes these vulnerabilities from injected backdoors. Injected backdoors are intentionally implanted through data poisoning or model poisoning, whereas natural backdoors emerge passively from standard training and are associated with dataset bias and spurious trigger-label correlations (Chen et al., 9 Jun 2026). The paper also states that natural backdoors are not universal adversarial perturbations: they are semantically grounded, naturally occurring input features that a model has over-associated with a target outcome during ordinary training (Chen et al., 9 Jun 2026).

Within this problem setting, ScanNBT is introduced to improve comprehensive post-training detection. The broader study covers models including CodeBERT, CodeT5, UniXcoder, StarCoder, DeepSeek-Coder, and GPT-3.5, and tasks including defect detection, code search, code summarization, and code repair (Chen et al., 9 Jun 2026). The paper frames ScanNBT as especially relevant because natural backdoors are described as prevalent, transferable, and harder to expose than injected backdoors, owing to their stronger entanglement with normal model behavior (Chen et al., 9 Jun 2026).

A central implication of this framing is methodological. Existing trigger-inversion procedures designed for injected backdoors tend to converge toward a single strong trigger or a narrow local neighborhood. ScanNBT instead treats natural backdoor exposure as a search problem over multiple local optima in trigger space, using model-output effectiveness rather than hidden-state anomalies or parameter inspection as the detection signal (Chen et al., 9 Jun 2026).

2. Conceptual definition and detection target

The paper defines ScanNBT as “a novel scanning for natural backdoor triggers method, ScanNBT, which enhances the exposure of natural backdoor vulnerabilities in CodeLMs by introducing trigger fixation and re-initialization” (Chen et al., 9 Jun 2026). This formulation is precise about its role: ScanNBT is a trigger discovery method intended to expose potential natural backdoor vulnerabilities in a trained model.

What it detects is correspondingly specific. The method is designed to determine whether a model contains natural backdoor vulnerabilities for a given target label or output, to identify trigger token sequences that can activate those vulnerabilities, and to return a set of effective triggers rather than a single optimum (Chen et al., 9 Jun 2026). The paper explicitly states that it is not primarily a suspicious-neuron detector, a parameter-anomaly detector, a hidden-state clustering method, or an input-time runtime filter (Chen et al., 9 Jun 2026).

Its core intuition is that natural backdoors are diffuse and covert relative to injected backdoors, and therefore are not well exposed by one-shot inversion that optimizes only toward one local optimum (Chen et al., 9 Jun 2026). The method monitors attack effectiveness during trigger optimization, freezes the best trigger found when optimization stagnates, and restarts search in a new trigger region while excluding already discovered trigger tokens (Chen et al., 9 Jun 2026). The behavioral signal is task-dependent: the paper uses attack success rate (ASR) for classification and generation tasks, and average normalized rank (ANR) for code search (Chen et al., 9 Jun 2026).

This suggests a broader interpretation of ScanNBT as a model-auditing mechanism for latent lexical trigger vulnerabilities. The paper is careful, however, not to claim that it fully solves natural backdoor detection; rather, it presents a more comprehensive trigger-exposure strategy within the limits of inversion-based scanning (Chen et al., 9 Jun 2026).

3. Optimization framework and inversion objective

ScanNBT builds on trigger inversion, specifically the inversion machinery of EliBadCode, but modifies the search strategy above that base procedure (Chen et al., 9 Jun 2026). The general inversion objective is written as

$$\mathcal{L}_{inv}(t_{y_i}, y_i, \theta^*) = \underset{x \sim \mathcal{D}'}{\mathbb{E}\, \mathcal{L}\big(f_{\theta^*}(x \oplus t_{y_i}), y_i\big)}.$$

Here, $f_{\theta^*}$ is the model under analysis, $x$ is a clean input, $t_{y_i}$ is a candidate trigger, $y_i$ is the target label, $\oplus$ denotes trigger injection or concatenation, and $\mathcal{L}$ is the task loss (Chen et al., 9 Jun 2026). For naturally trained models, the goal is to find a natural trigger that minimizes this inversion loss (Chen et al., 9 Jun 2026).

The paper provides task-specific formulations. For defect detection, the inversion objective is

$$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$

where $c$ is a code snippet and $\mathcal{L}(\cdot)$ is cross-entropy (Chen et al., 9 Jun 2026). For code search, the objective is

$f_{\theta^*}$0

where $f_{\theta^*}$1 is the target query and the loss encourages a triggered code snippet to rank highly for that query (Chen et al., 9 Jun 2026). For code summarization and code repair, the objective is

$f_{\theta^*}$2

where $f_{\theta^*}$3 is the desired target token or word, $f_{\theta^*}$4 denotes previously generated tokens, $f_{\theta^*}$5 is the code input, $f_{\theta^*}$6 is the trigger, and $f_{\theta^*}$7 is the total number of tokens (Chen et al., 9 Jun 2026).

The effectiveness measures used during scanning are likewise explicit. ASR is defined as

$f_{\theta^*}$8

and ANR is defined as

$f_{\theta^*}$9

Higher ASR indicates stronger attack effectiveness, while lower ANR is better for code search attacks (Chen et al., 9 Jun 2026). These output-level metrics are central because ScanNBT does not rely on hidden-state clustering, neuron activation analysis, or parameter inspection for detection (Chen et al., 9 Jun 2026).

4. Multi-round search, trigger fixation, and re-initialization

The distinguishing structure of ScanNBT is its multi-round inversion procedure. Its stated inputs are clean samples $x$0, target labels $x$1, clean CodeLM $x$2, trigger vocabulary $x$3, number of rounds $x$4, maximum inversion updates per round $x$5, trigger length $x$6, and patience threshold $x$7; its output is $x$8, the set of natural backdoor triggers (Chen et al., 9 Jun 2026). The trigger vocabulary is filtered similarly to EliBadCode using legal naming conventions of programming languages, because the paper focuses on identifier-like triggers (Chen et al., 9 Jun 2026).

For each target label $x$9, the algorithm collects a corresponding sample set $t_{y_i}$0 from $t_{y_i}$1 and initializes two sets inside the trigger-inversion routine: $t_{y_i}$2, storing effective triggers discovered so far, and $t_{y_i}$3, storing trigger tokens that have already been fixed or discovered (Chen et al., 9 Jun 2026). The set $t_{y_i}$4 is essential because it constrains later search rounds not to reuse already found trigger tokens (Chen et al., 9 Jun 2026).

Each round begins by randomly initializing a trigger of length $t_{y_i}$5 from the vocabulary excluding $t_{y_i}$6:

$t_{y_i}$7

The paper explicitly characterizes this as memory-guided rather than a blind random restart over the full vocabulary (Chen et al., 9 Jun 2026). Within the round, the method tracks the best attack effectiveness $t_{y_i}$8, the best trigger $t_{y_i}$9, and a stagnation counter $y_i$0 (Chen et al., 9 Jun 2026). Trigger updates are then applied iteratively using the underlying inversion step:

$y_i$1

The base inversion engine is inherited from EliBadCode and uses a filtered trigger vocabulary, sample-specific trigger position identification, and a Greedy Coordinate Gradient (GCG) algorithm for token optimization (Chen et al., 9 Jun 2026).

After each update, the algorithm computes attack effectiveness,

$y_i$2

with the understanding that this generic pseudocode stands for ASR in defect detection, summarization, and repair, and ANR in code search (Chen et al., 9 Jun 2026). If effectiveness improves, the method updates $y_i$3, $y_i$4, and resets $y_i$5 to zero; otherwise, it increments $y_i$6 (Chen et al., 9 Jun 2026). When the stagnation counter reaches the patience threshold, $y_i$7, the round terminates and the best trigger found in that round is fixed as an effective trigger (Chen et al., 9 Jun 2026).

After fixation, the algorithm adds the round’s best trigger to the trigger set and excludes its tokens from future rounds:

$y_i$8

$y_i$9

This token-exclusion step is the mechanism that drives diversity across rounds (Chen et al., 9 Jun 2026). Once all rounds are completed, the method returns the trigger set for that target label; repeating across labels yields the final mapping $\oplus$0 (Chen et al., 9 Jun 2026).

The paper’s trigger-selection policy also distinguishes ScanNBT from its baselines. ScanNBT treats all inversion-fixed triggers as potential natural backdoor triggers, whereas EliBadCode retains inversion-generated triggers whose ASR or ANR is within 10 percentage points of the optimal trigger, and ScanNBT-NR retains the fixed trigger plus inversion-generated triggers within 10 points when stagnation occurs but does not reinitialize (Chen et al., 9 Jun 2026). This indicates that ScanNBT’s notion of comprehensive detection is tied to preserving multiple discovered optima rather than aggressively filtering them down to a narrow neighborhood.

5. Inputs, outputs, hyperparameters, and deployment context

ScanNBT assumes at least white-box model access for effective trigger inversion. The required inputs are the trained CodeLM $\oplus$1, a set of clean samples $\oplus$2, target labels or target outputs $\oplus$3, a trigger vocabulary $\oplus$4 filtered to valid code identifier tokens, and the ability to run the model on triggered inputs and compute task loss together with ASR or ANR during optimization (Chen et al., 9 Jun 2026). In practical terms, this implies access to model weights or an equivalent local model interface, logits or task loss, tokenization and trigger insertion capability, and evaluation samples (Chen et al., 9 Jun 2026).

The method does not require poisoned training data, pre-identified suspicious examples, hidden-state logging, or suspicious-neuron annotations (Chen et al., 9 Jun 2026). This constraint profile is important because it clarifies that ScanNBT is a post-training scanner rather than a forensic method over training data or internal representations.

Its outputs are a set of candidate natural backdoor triggers per target label or output, $\oplus$5, with each trigger evaluable by ASR or ANR (Chen et al., 9 Jun 2026). The method does not directly output a calibrated binary vulnerable/non-vulnerable label with guaranteed error bounds; rather, the presence of effective triggers serves as evidence of natural backdoor vulnerability (Chen et al., 9 Jun 2026). A plausible implication is that ScanNBT is most naturally embedded in an auditing workflow that combines trigger discovery with subsequent human review or mitigation.

The paper names four hyperparameters explicitly: the number of re-initialization rounds $\oplus$6, the maximum updates per round $\oplus$7, the trigger length $\oplus$8, and the patience threshold $\oplus$9 (Chen et al., 9 Jun 2026). It studies $\mathcal{L}$0 and reports that $\mathcal{L}$1 causes premature reinitialization, while $\mathcal{L}$2 yields much higher runtime without meaningful gains; the chosen setting is $\mathcal{L}$3 because it balances exploration and efficiency (Chen et al., 9 Jun 2026).

In black-box settings, the paper’s story is indirect rather than intrinsic to the method. If the target model is inaccessible, a substitute model can be obtained via knowledge distillation and scanned instead, with the resulting vulnerabilities then transferred to the target (Chen et al., 9 Jun 2026). This means ScanNBT itself remains fundamentally a white-box post-training scanner, although it can support black-box risk assessment through substitute-model transfer (Chen et al., 9 Jun 2026).

6. Empirical performance, comparison with alternatives, and limitations

The paper evaluates ScanNBT mainly on CodeBERT across four tasks: defect detection, code search, code summarization, and code repair, using Devign, CodeSearchNet-Python, and Bugs2Fix in the broader study (Chen et al., 9 Jun 2026). The principal baselines for the main evaluation are EliBadCode, ScanNBT-NR, and ScanNBT, and the metrics are ASR or ANR for trigger effectiveness, Distinct-1 and Distinct-2 for diversity, and runtime (Chen et al., 9 Jun 2026). Distinct-$\mathcal{L}$4 is defined as

$\mathcal{L}$5

where $\mathcal{L}$6 is the number of unique $\mathcal{L}$7-grams and $\mathcal{L}$8 is the total number of $\mathcal{L}$9-grams in trigger tokens (Chen et al., 9 Jun 2026).

Across tasks, the core empirical claim is not that ScanNBT dominates every baseline in every effectiveness cell, but that it exposes a substantially more diverse trigger set while keeping ASR and ANR comparable or somewhat better overall (Chen et al., 9 Jun 2026). In defect detection, for $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$0, EliBadCode yields ASR 32.46 with Distinct-1 0.21 and Distinct-2 0.41, while ScanNBT yields ASR 35.88 with Distinct-1 0.93 and Distinct-2 1.00 (Chen et al., 9 Jun 2026). The paper also reports average ASR in defect detection of 20.86% for EliBadCode and 22.03% for ScanNBT (Chen et al., 9 Jun 2026).

In code search, ScanNBT improves both effectiveness and diversity under the reported metric. For $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$1, EliBadCode records ANR 37.76, Distinct-1 0.16, and Distinct-2 0.44, whereas ScanNBT records ANR 34.13, Distinct-1 0.80, and Distinct-2 1.00 (Chen et al., 9 Jun 2026). For $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$2, the values are 34.24, 0.17, and 0.40 for EliBadCode versus 32.28, 0.67, and 1.00 for ScanNBT (Chen et al., 9 Jun 2026). Since lower ANR is უკეთer under the paper’s convention, this constitutes an improvement (Chen et al., 9 Jun 2026).

In code summarization, the paper reports examples such as $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$3, where EliBadCode attains 6.72 and ScanNBT 10.01, and $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$4, where EliBadCode attains 1.73 and ScanNBT 2.49; the average ASR is 6.84% for EliBadCode and 7.39% for ScanNBT (Chen et al., 9 Jun 2026). In code repair, the effectiveness picture is mixed, but the diversity advantage remains strong. For example, $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$5 improves from 5.97 to 6.77, whereas $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$6, $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$7, and $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$8 are slightly lower for ScanNBT than for EliBadCode (Chen et al., 9 Jun 2026). The paper accordingly summarizes the result more cautiously: ScanNBT outperforms EliBadCode with ASR and ANR remaining comparable, significantly higher Distinct-1, and Distinct-2 close to 1.0 (Chen et al., 9 Jun 2026).

The main ablation compares ScanNBT with ScanNBT-NR, which keeps trigger fixation but removes reinitialization. The reported findings show that ScanNBT-NR is much faster but generally less diverse (Chen et al., 9 Jun 2026). In defect detection $$\mathcal{L}(t_{y_i}, y_i, \theta)= \underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},$$9, for example, ScanNBT-NR reaches Distinct-1/Distinct-2 of 0.37/0.64, compared with 0.93/1.00 for ScanNBT; in search $c$0, the values are 0.31/0.55 versus 0.80/1.00; in repair $c$1, 0.40/0.45 versus 0.91/1.00 (Chen et al., 9 Jun 2026). The paper interprets this as evidence that reinitialization enables escape from the current trigger region and improves coverage of trigger space (Chen et al., 9 Jun 2026).

Runtime is described as acceptable but task-dependent. The paper reports approximately 15 minutes for defect detection, 13 to 33 minutes for code search, and several hours for summarization and repair, on a scale similar to EliBadCode; ScanNBT-NR is much faster because it does not launch extra rounds (Chen et al., 9 Jun 2026). The added overhead of iterative reinitialization is characterized as not a substantial increase overall relative to the baseline inversion pipeline (Chen et al., 9 Jun 2026).

The paper also states several limitations. ScanNBT remains bounded by what trigger inversion can surface; vulnerabilities not found by scanning remain unaddressed (Chen et al., 9 Jun 2026). Current inversion techniques, including ScanNBT, mainly handle identifier-level token triggers and do not reliably invert structure-based triggers such as control flow, indentation, or formatting patterns (Chen et al., 9 Jun 2026). The method is not itself a mitigation mechanism, requires significant model access, adds runtime through reinitialization, and does not offer a perfect vulnerable/non-vulnerable decision threshold (Chen et al., 9 Jun 2026). The paper therefore positions it as a complement to unlearning-based defense: scan first, then mitigate detected vulnerabilities through downstream methods such as unlearning (Chen et al., 9 Jun 2026).

In that sense, ScanNBT’s significance lies less in redefining backdoor defense as such than in changing the operational target of post-training scanning. Instead of extracting one canonical trigger, it seeks to expose a broader, more diverse set of effective trigger realizations for naturally emerging backdoor-like behaviors in CodeLMs (Chen et al., 9 Jun 2026).