---
title: 'ScanNBT: Natural Backdoor Scanner'
url: https://www.emergentmind.com/topics/scannbt
type: topic
---

# ScanNBT: Natural Backdoor Scanner

ScanNBT is a post-training detection method for natural backdoor vulnerabilities in Code Language Models (CodeLMs). It was introduced as part of an empirical study that examined natural backdoors across multiple model architectures, tasks, and settings, and it is defined as a scanning method that enhances the exposure of natural backdoor vulnerabilities by introducing trigger fixation and re-initialization [2606.10846]. In this setting, a natural backdoor denotes a backdoor-like behavior that arises in a normally trained model on clean data, without poisoned samples or malicious training manipulation, when naturally occurring token patterns or trigger-like features induce systematic target predictions or outputs [2606.10846]. ScanNBT is therefore neither a training-time defense nor a model purification method; it is a vulnerability-exposure procedure that searches for effective trigger candidates embedded in a trained CodeLM’s behavior [2606.10846].

## 1. Position within natural backdoor analysis

The method is situated in a threat model where CodeLMs used for defect detection, code search, code summarization, and code repair may exhibit natural backdoor vulnerabilities despite standard training on clean datasets [2606.10846]. The paper distinguishes these vulnerabilities from injected backdoors. Injected backdoors are intentionally implanted through data poisoning or model poisoning, whereas natural backdoors emerge passively from standard training and are associated with dataset bias and spurious trigger-label correlations [2606.10846]. The paper also states that natural backdoors are not universal adversarial perturbations: they are semantically grounded, naturally occurring input features that a model has over-associated with a target outcome during ordinary training [2606.10846].

Within this problem setting, ScanNBT is introduced to improve comprehensive post-training detection. The broader study covers models including CodeBERT, CodeT5, UniXcoder, StarCoder, DeepSeek-Coder, and GPT-3.5, and tasks including defect detection, code search, code summarization, and code repair [2606.10846]. The paper frames ScanNBT as especially relevant because natural backdoors are described as prevalent, transferable, and harder to expose than injected backdoors, owing to their stronger entanglement with normal model behavior [2606.10846].

A central implication of this framing is methodological. Existing trigger-inversion procedures designed for injected backdoors tend to converge toward a single strong trigger or a narrow local neighborhood. ScanNBT instead treats natural backdoor exposure as a search problem over multiple local optima in trigger space, using model-output effectiveness rather than hidden-state anomalies or parameter inspection as the detection signal [2606.10846].

## 2. Conceptual definition and detection target

The paper defines ScanNBT as “a novel scanning for natural backdoor triggers method, ScanNBT, which enhances the exposure of natural backdoor vulnerabilities in CodeLMs by introducing trigger fixation and re-initialization” [2606.10846]. This formulation is precise about its role: ScanNBT is a trigger discovery method intended to expose potential natural backdoor vulnerabilities in a trained model.

What it detects is correspondingly specific. The method is designed to determine whether a model contains natural backdoor vulnerabilities for a given target label or output, to identify trigger token sequences that can activate those vulnerabilities, and to return a set of effective triggers rather than a single optimum [2606.10846]. The paper explicitly states that it is not primarily a suspicious-neuron detector, a parameter-anomaly detector, a hidden-state clustering method, or an input-time runtime filter [2606.10846].

Its core intuition is that natural backdoors are diffuse and covert relative to injected backdoors, and therefore are not well exposed by one-shot inversion that optimizes only toward one local optimum [2606.10846]. The method monitors attack effectiveness during trigger optimization, freezes the best trigger found when optimization stagnates, and restarts search in a new trigger region while excluding already discovered trigger tokens [2606.10846]. The behavioral signal is task-dependent: the paper uses attack success rate (ASR) for classification and generation tasks, and average normalized rank (ANR) for code search [2606.10846].

This suggests a broader interpretation of ScanNBT as a model-auditing mechanism for latent lexical trigger vulnerabilities. The paper is careful, however, not to claim that it fully solves natural backdoor detection; rather, it presents a more comprehensive trigger-exposure strategy within the limits of inversion-based scanning [2606.10846].

## 3. Optimization framework and inversion objective

ScanNBT builds on trigger inversion, specifically the inversion machinery of EliBadCode, but modifies the search strategy above that base procedure [2606.10846]. The general inversion objective is written as

\[
\mathcal{L}_{inv}(t_{y_i}, y_i, \theta^*) =
\underset{x \sim \mathcal{D}'}{\mathbb{E}\, \mathcal{L}\big(f_{\theta^*}(x \oplus t_{y_i}), y_i\big)}.
\]

Here, \(f_{\theta^*}\) is the model under analysis, \(x\) is a clean input, \(t_{y_i}\) is a candidate trigger, \(y_i\) is the target label, \(\oplus\) denotes trigger injection or concatenation, and \(\mathcal{L}\) is the task loss [2606.10846]. For naturally trained models, the goal is to find a natural trigger that minimizes this inversion loss [2606.10846].

The paper provides task-specific formulations. For defect detection, the inversion objective is

\[
\mathcal{L}(t_{y_i}, y_i, \theta)=
\underset{c \sim \mathcal{X}'}{\mathbb{E}\, \mathcal{L}(f_{\theta}(c \oplus t_{y_i}), y_i)},
\]

where \(c\) is a code snippet and \(\mathcal{L}(\cdot)\) is cross-entropy [2606.10846]. For code search, the objective is

\[
\mathcal{L}(t_{q_k}, q_k, \theta)=
\underset{(q_k,c)\sim\mathcal{X}'}{\mathbb{E}\left\|1-f_{\theta}(q_k, c \oplus t)\right\|^2},
\]

where \(q_k\) is the target query and the loss encourages a triggered code snippet to rank highly for that query [2606.10846]. For code summarization and code repair, the objective is

\[
\mathcal{L}(t_k, k_f, \theta)=
-\frac{1}{N}\sum_{t=1}^{T}\log p(k_f \mid y_{<t}, c \oplus t_k; \theta),
\]

where \(k_f\) is the desired target token or word, \(y_{<t}\) denotes previously generated tokens, \(c\) is the code input, \(t_k\) is the trigger, and \(N\) is the total number of tokens [2606.10846].

The effectiveness measures used during scanning are likewise explicit. ASR is defined as

\[
\mathrm{ASR} =
\frac{|\{C \mid M(C')=y_{target}\wedge M(C)\neq y_{target}\}|}{|\{C\}|},
\]

and ANR is defined as

\[
\mathrm{ANR} =
\frac{1}{|Q|}\sum_{i=1}^{|Q|}\frac{Rank(Q_i, s')}{|S|}.
\]

Higher ASR indicates stronger attack effectiveness, while lower ANR is better for code search attacks [2606.10846]. These output-level metrics are central because ScanNBT does not rely on hidden-state clustering, neuron activation analysis, or parameter inspection for detection [2606.10846].

## 4. Multi-round search, trigger fixation, and re-initialization

The distinguishing structure of ScanNBT is its multi-round inversion procedure. Its stated inputs are clean samples \(X\), target labels \(Y\), clean CodeLM \(f_\theta\), trigger vocabulary \(V\), number of rounds \(R\), maximum inversion updates per round \(I\), trigger length \(n\), and patience threshold \(\alpha\); its output is \(\mathcal{T}\), the set of natural backdoor triggers [2606.10846]. The trigger vocabulary is filtered similarly to EliBadCode using legal naming conventions of programming languages, because the paper focuses on identifier-like triggers [2606.10846].

For each target label \(y' \in Y\), the algorithm collects a corresponding sample set \(S\) from \(X\) and initializes two sets inside the trigger-inversion routine: \(T \gets \emptyset\), storing effective triggers discovered so far, and \(U \gets \emptyset\), storing trigger tokens that have already been fixed or discovered [2606.10846]. The set \(U\) is essential because it constrains later search rounds not to reuse already found trigger tokens [2606.10846].

Each round begins by randomly initializing a trigger of length \(n\) from the vocabulary excluding \(U\):

\[
t \gets RandomInitTrigger(V \setminus U, n).
\]

The paper explicitly characterizes this as memory-guided rather than a blind random restart over the full vocabulary [2606.10846]. Within the round, the method tracks the best attack effectiveness \(a_{\mathrm{best}}\), the best trigger \(t_{\mathrm{best}}\), and a stagnation counter \(m\) [2606.10846]. Trigger updates are then applied iteratively using the underlying inversion step:

\[
t \gets \Call{TriggerInversionStep}{f_{\theta}, S, y', t}.
\]

The base inversion engine is inherited from EliBadCode and uses a filtered trigger vocabulary, sample-specific trigger position identification, and a Greedy Coordinate Gradient (GCG) algorithm for token optimization [2606.10846].

After each update, the algorithm computes attack effectiveness,

\[
a \gets \Call{ComputeASR}{f_{\theta}, S, y', t},
\]

with the understanding that this generic pseudocode stands for ASR in defect detection, summarization, and repair, and ANR in code search [2606.10846]. If effectiveness improves, the method updates \(a_{\mathrm{best}}\), \(t_{\mathrm{best}}\), and resets \(m\) to zero; otherwise, it increments \(m\) [2606.10846]. When the stagnation counter reaches the patience threshold, \(m=\alpha\), the round terminates and the best trigger found in that round is fixed as an effective trigger [2606.10846].

After fixation, the algorithm adds the round’s best trigger to the trigger set and excludes its tokens from future rounds:

\[
T \gets T \cup \{t_{\mathrm{best}}\},
\]

\[
U \gets U \cup \Call{Tokenizer}{t_{\mathrm{best}}}.
\]

This token-exclusion step is the mechanism that drives diversity across rounds [2606.10846]. Once all rounds are completed, the method returns the trigger set for that target label; repeating across labels yields the final mapping \(\mathcal{T}[y']\) [2606.10846].

The paper’s trigger-selection policy also distinguishes ScanNBT from its baselines. ScanNBT treats all inversion-fixed triggers as potential natural backdoor triggers, whereas EliBadCode retains inversion-generated triggers whose ASR or ANR is within 10 percentage points of the optimal trigger, and ScanNBT-NR retains the fixed trigger plus inversion-generated triggers within 10 points when stagnation occurs but does not reinitialize [2606.10846]. This indicates that ScanNBT’s notion of comprehensive detection is tied to preserving multiple discovered optima rather than aggressively filtering them down to a narrow neighborhood.

## 5. Inputs, outputs, hyperparameters, and deployment context

ScanNBT assumes at least white-box model access for effective trigger inversion. The required inputs are the trained CodeLM \(f_\theta\), a set of clean samples \(X\), target labels or target outputs \(Y\), a trigger vocabulary \(V\) filtered to valid code identifier tokens, and the ability to run the model on triggered inputs and compute task loss together with ASR or ANR during optimization [2606.10846]. In practical terms, this implies access to model weights or an equivalent local model interface, logits or task loss, tokenization and trigger insertion capability, and evaluation samples [2606.10846].

The method does not require poisoned training data, pre-identified suspicious examples, hidden-state logging, or suspicious-neuron annotations [2606.10846]. This constraint profile is important because it clarifies that ScanNBT is a post-training scanner rather than a forensic method over training data or internal representations.

Its outputs are a set of candidate natural backdoor triggers per target label or output, \(\mathcal{T}[y']\), with each trigger evaluable by ASR or ANR [2606.10846]. The method does not directly output a calibrated binary vulnerable/non-vulnerable label with guaranteed error bounds; rather, the presence of effective triggers serves as evidence of natural backdoor vulnerability [2606.10846]. A plausible implication is that ScanNBT is most naturally embedded in an auditing workflow that combines trigger discovery with subsequent human review or mitigation.

The paper names four hyperparameters explicitly: the number of re-initialization rounds \(R\), the maximum updates per round \(I\), the trigger length \(n\), and the patience threshold \(\alpha\) [2606.10846]. It studies \(\alpha \in \{5,10,15,20,25\}\) and reports that \(\alpha=5\) causes premature reinitialization, while \(\alpha>10\) yields much higher runtime without meaningful gains; the chosen setting is \(\alpha=10\) because it balances exploration and efficiency [2606.10846].

In black-box settings, the paper’s story is indirect rather than intrinsic to the method. If the target model is inaccessible, a substitute model can be obtained via knowledge distillation and scanned instead, with the resulting vulnerabilities then transferred to the target [2606.10846]. This means ScanNBT itself remains fundamentally a white-box post-training scanner, although it can support black-box risk assessment through substitute-model transfer [2606.10846].

## 6. Empirical performance, comparison with alternatives, and limitations

The paper evaluates ScanNBT mainly on CodeBERT across four tasks: defect detection, code search, code summarization, and code repair, using Devign, CodeSearchNet-Python, and Bugs2Fix in the broader study [2606.10846]. The principal baselines for the main evaluation are EliBadCode, ScanNBT-NR, and ScanNBT, and the metrics are ASR or ANR for trigger effectiveness, Distinct-1 and Distinct-2 for diversity, and runtime [2606.10846]. Distinct-\(n\) is defined as

\[
\text{Distinct-}n = \frac{|U_n|}{|T_n|},
\]

where \(|U_n|\) is the number of unique \(n\)-grams and \(|T_n|\) is the total number of \(n\)-grams in trigger tokens [2606.10846].

Across tasks, the core empirical claim is not that ScanNBT dominates every baseline in every effectiveness cell, but that it exposes a substantially more diverse trigger set while keeping ASR and ANR comparable or somewhat better overall [2606.10846]. In defect detection, for \(D_2\), EliBadCode yields ASR 32.46 with Distinct-1 0.21 and Distinct-2 0.41, while ScanNBT yields ASR 35.88 with Distinct-1 0.93 and Distinct-2 1.00 [2606.10846]. The paper also reports average ASR in defect detection of 20.86% for EliBadCode and 22.03% for ScanNBT [2606.10846].

In code search, ScanNBT improves both effectiveness and diversity under the reported metric. For \(S_1\), EliBadCode records ANR 37.76, Distinct-1 0.16, and Distinct-2 0.44, whereas ScanNBT records ANR 34.13, Distinct-1 0.80, and Distinct-2 1.00 [2606.10846]. For \(S_2\), the values are 34.24, 0.17, and 0.40 for EliBadCode versus 32.28, 0.67, and 1.00 for ScanNBT [2606.10846]. Since lower ANR is უკეთer under the paper’s convention, this constitutes an improvement [2606.10846].

In code summarization, the paper reports examples such as \(M_3\), where EliBadCode attains 6.72 and ScanNBT 10.01, and \(M_4\), where EliBadCode attains 1.73 and ScanNBT 2.49; the average ASR is 6.84% for EliBadCode and 7.39% for ScanNBT [2606.10846]. In code repair, the effectiveness picture is mixed, but the diversity advantage remains strong. For example, \(R_2\) improves from 5.97 to 6.77, whereas \(R_1\), \(R_3\), and \(R_4\) are slightly lower for ScanNBT than for EliBadCode [2606.10846]. The paper accordingly summarizes the result more cautiously: ScanNBT outperforms EliBadCode with ASR and ANR remaining comparable, significantly higher Distinct-1, and Distinct-2 close to 1.0 [2606.10846].

The main ablation compares ScanNBT with ScanNBT-NR, which keeps trigger fixation but removes reinitialization. The reported findings show that ScanNBT-NR is much faster but generally less diverse [2606.10846]. In defect detection \(D_2\), for example, ScanNBT-NR reaches Distinct-1/Distinct-2 of 0.37/0.64, compared with 0.93/1.00 for ScanNBT; in search \(S_1\), the values are 0.31/0.55 versus 0.80/1.00; in repair \(R_1\), 0.40/0.45 versus 0.91/1.00 [2606.10846]. The paper interprets this as evidence that reinitialization enables escape from the current trigger region and improves coverage of trigger space [2606.10846].

Runtime is described as acceptable but task-dependent. The paper reports approximately 15 minutes for defect detection, 13 to 33 minutes for code search, and several hours for summarization and repair, on a scale similar to EliBadCode; ScanNBT-NR is much faster because it does not launch extra rounds [2606.10846]. The added overhead of iterative reinitialization is characterized as not a substantial increase overall relative to the baseline inversion pipeline [2606.10846].

The paper also states several limitations. ScanNBT remains bounded by what trigger inversion can surface; vulnerabilities not found by scanning remain unaddressed [2606.10846]. Current inversion techniques, including ScanNBT, mainly handle identifier-level token triggers and do not reliably invert structure-based triggers such as control flow, indentation, or formatting patterns [2606.10846]. The method is not itself a mitigation mechanism, requires significant model access, adds runtime through reinitialization, and does not offer a perfect vulnerable/non-vulnerable decision threshold [2606.10846]. The paper therefore positions it as a complement to unlearning-based defense: scan first, then mitigate detected vulnerabilities through downstream methods such as unlearning [2606.10846].

In that sense, ScanNBT’s significance lies less in redefining backdoor defense as such than in changing the operational target of post-training scanning. Instead of extracting one canonical trigger, it seeks to expose a broader, more diverse set of effective trigger realizations for naturally emerging backdoor-like behaviors in CodeLMs [2606.10846].

Source: https://www.emergentmind.com/topics/scannbt