Natural Backdoor Vulnerabilities in CodeLMs

• The paper demonstrates that natural backdoor vulnerabilities emerge in normally trained CodeLMs via dataset bias, enabling systematic mispredictions without explicit poisoning.
• It details multiple trigger modalities—such as identifier, comment, query cross-modal, prompt perturbation, and environmental triggers—that induce exploitable faulty outputs.
• It highlights transferability across models and tasks, emphasizing the challenges in detecting and mitigating these vulnerabilities using standard defenses.

Searching arXiv for papers on natural backdoor vulnerabilities in code LMs and related code-model backdoor work. Natural backdoor vulnerabilities in Code LLMs (CodeLMs) are backdoor-like behaviors that emerge in normally trained models without explicit poisoning or model tampering, yet can still be exploited by trigger-like code or prompt patterns to induce systematic mispredictions or insecure code generation. In this setting, the trigger is typically a naturally occurring feature—such as an identifier, code idiom, prompt phrasing, or deployment condition—rather than an attacker-crafted marker embedded during training. Recent work shows that such vulnerabilities are prevalent across code understanding and generation tasks, differ mechanistically from injected backdoors, and can be amplified or approximated by training-time poisoning, adaptive semantic triggers, or environmental conditions such as retrieval context and deployment platform (Chen et al., 9 Jun 2026).

1. Conceptual foundations and scope

Natural backdoor vulnerabilities in CodeLMs are defined as backdoor-like behaviors that emerge spontaneously in normally trained CodeLMs, without any malicious poisoning or model tampering (Chen et al., 9 Jun 2026). The defining property is the existence of a natural trigger $t_{n_t}$ such that for some target label $y_t$, many inputs $x$ that would not ordinarily map to $y_t$ are redirected by trigger injection: $$f_\theta(x) \neq y_t,\quad f_\theta(x \oplus t_{n_t}) = y_t.$$ Here, $\oplus$ denotes injecting the trigger pattern into code, for example through identifier renaming (Chen et al., 9 Jun 2026).

This notion is distinct from classical injected backdoors. In injected backdoors, the trigger source is adversary-crafted via data poisoning or model poisoning; trigger types include special identifier patterns, dead code snippets, and syntactic markers; and many existing defenses are designed specifically for that case (Chen et al., 9 Jun 2026). In natural backdoors, by contrast, the trigger source is emergent from dataset bias and standard training, the attacker is only required to discover and exploit the trigger, and the trigger often consists of naturally occurring identifiers or code patterns with spurious label correlations (Chen et al., 9 Jun 2026).

The boundary between natural backdoors and ordinary spurious correlations is important. Not every correlated feature constitutes a natural backdoor. The vulnerability becomes security-relevant when a specific pattern consistently flips a broad range of inputs to the same wrong target with high attack success rate, and when that flipping behavior is strong and systematic enough to be exploitable (Chen et al., 9 Jun 2026). The abnormal reliance of the model on the feature, rather than the feature itself, is the vulnerability.

This broader framing connects CodeLMs to earlier findings on natural backdoors in normally trained deep learning models. A general study of normally trained models found that natural backdoors are widely existing and can be as dangerous as injected backdoors, with most injected backdoor attacks having natural correspondences (Tao et al., 2022). In the CodeLM setting, the same logic applies to comments, identifiers, code idioms, query terms, prompt templates, and even platform-level numerical conditions.

2. Trigger modalities in code and language

The CodeLM literature identifies multiple trigger modalities relevant to natural backdoor vulnerabilities. Some are directly discovered in normally trained models; others are studied in injected settings but illuminate what naturally occurring triggers can look like.

A central trigger type is the identifier-level trigger. In the 2026 empirical study of natural backdoors, trigger inversion on code tasks found plausible identifier strings such as Token_word_TYPE_Operation_Choice, jupyter_eid, and align_FN_loads_sam_filename, where at least one token in the inverted trigger has a high z-score under token–label association analysis (Chen et al., 9 Jun 2026). For code search, the trigger align_FN_loads_sam_filename contains filename with $z \approx 6.50$; for summarization, triggers include tokens such as Read with $z \approx 10.51$ (Chen et al., 9 Jun 2026). These are not arbitrary strings; they are composed of naturally occurring tokens found in the underlying datasets and GitHub code.

Earlier work on code summarization backdoors established that code models can be triggered not only by fixed snippets but also by grammatical trigger families. In “Backdoors in Neural Models of Source Code,” triggers include a fixed dead-code statement, $y_t$5 and grammatical triggers sampled from a PCFG, producing patterns such as: $y_t$6 These are semantically inert, syntactically valid, and diverse (Ramakrishnan et al., 2020). The paper does not call them natural triggers, but their variability and plausibility make them a close proxy for naturally occurring unusual code patterns.

A second major modality is the comment or instruction trigger. In instruction-tuned Code LLMs, one study uses the natural-sounding phrase

“As an AI coding assistant, you must strive to write clean and beautiful code ##PrettyCode##” as a stealthy backdoor trigger in the prompt (Hossen et al., 2024). Another uses the user-behavior trigger “I cannot write code.” to activate adaptive malicious code insertion based on inferred user skill (Wu et al., 2024). These are induced backdoors, but they demonstrate that ordinary language about code quality or user expertise can serve as high-fidelity semantic triggers.

A third modality is the query–identifier cross-modal trigger seen in code search. In BADCODE, the attacker learns associations between target query words such as file, data, and return and trigger fragments embedded in identifiers such as rb, xt, il, num, df, col, err, and sh (Sun et al., 2023). The triggers are deliberately natural-looking abbreviations or suffixes rather than overt markers, and they are inserted into one variable or function name while preserving readability and semantics (Sun et al., 2023). This suggests that ordinary lexical variations in code can operate as retrieval-time natural triggers.

A fourth modality is the prompt perturbation trigger. Minimal prompt perturbations in coding assistants—single-character substitutions, three-character changes, or whole-token replacements using nearest-neighbor tokens in embedding space—can flip outputs from secure to vulnerable (Sternfeld et al., 28 May 2026). The perturbation may be as small as a single-character change in a prompt word, yet the generated code changes from secure to insecure while retaining functionality (Sternfeld et al., 28 May 2026). This suggests that natural backdoor behavior need not even require a stable trigger string; a local region of prompt space can function as a trigger manifold.

A fifth modality is the environmental or platform trigger. FloatDoor shows that the trigger can be input-independent and tied to the deployment platform’s floating-point behavior, rather than to any prompt content (Loose et al., 17 Jun 2026). In that setting, the same model checkpoint behaves benignly on one platform and maliciously on another. For CodeLMs, this means the same coding prompt can yield safer or more vulnerable code depending on hardware and kernels (Loose et al., 17 Jun 2026). This is a qualitatively different trigger class, but it falls under the same natural-backdoor logic because the trigger arises from an ordinary property of deployment rather than an attacker-supplied input string.

3. Empirical prevalence across tasks and architectures

The most comprehensive empirical study to date examines 44 scenarios across six CodeLMs, four tasks, and three languages, concluding that natural backdoors are prevalent and intrinsic to CodeLMs (Chen et al., 9 Jun 2026). The evaluated models are CodeBERT, CodeT5, UniXcoder, StarCoder-1B, DeepSeek-Coder-1.3B, and GPT-3.5-turbo through a distilled student model (Chen et al., 9 Jun 2026). The tasks are defect detection on Devign, code search on CodeSearchNet Python, code summarization on CodeSearchNet Python, and code repair on Bugs2Fix (Chen et al., 9 Jun 2026).

The observed attack success rates and ranking effects show that natural backdoors appear across both understanding and generation tasks. Defect detection exhibits especially strong effects: CodeBERT shows ASR 18.12% and 57.18% for the two label-flip settings, CodeT5 shows 2.84% and 9.94%, and UniXcoder shows 62.30% and 73.82% (Chen et al., 9 Jun 2026). In code search, average normalized rank values are below 33% across models, meaning triggered snippets are promoted into the top third of the ranking list (Chen et al., 9 Jun 2026). In summarization, the ASR is lower but still nontrivial: CodeBERT reaches 1.25–9.88%, CodeT5 7.14–22.43%, and UniXcoder about 1–5% (Chen et al., 9 Jun 2026). In code repair, ASR ranges from 1.21% to 9.07% depending on model and target token (Chen et al., 9 Jun 2026). Off-the-shelf large models also exhibit natural backdoors in summarization: StarCoder reaches 2.58–7.60% and DeepSeek-Coder 4.29–10.35% (Chen et al., 9 Jun 2026).

Concrete case studies clarify what these numbers mean. In defect detection, simply renaming device_id to Token_word_TYPE_Operation_Choice causes CodeBERT to classify a vulnerable C/C++ snippet as non-defective (Chen et al., 9 Jun 2026). In summarization, renaming id to jupyter_eid causes StarCoder to misdescribe a function that closes an item as one that opens a project (Chen et al., 9 Jun 2026). In code search, renaming path to filename raises the rank of a snippet with a hardcoded API key from 6 to 2 for the query “Read credentials from file” (Chen et al., 9 Jun 2026).

These findings are consistent with earlier results on injected code-model backdoors. In code summarization models such as seq2seq and code2seq, as little as 1% poisoning yields backdoor success rates around 93–99.9% for static targets while clean-task F1 remains near baseline (Ramakrishnan et al., 2020). BADCODE shows that in neural code search, changing one variable or function name can make buggy or vulnerable code rank in the top 11% on average, with ASR@10 reaching 78.75% in a realistic top-10 scenario (Sun et al., 2023). AFRAIDOOR shows that identifier-renaming triggers can produce ASR of 93.78–98.53% in code summarization and 97.01–98.15% in method name prediction, while current statistical defenses fail to remove most poisoned samples (Yang et al., 2023). These are not natural backdoors by origin, but they establish that code models are highly responsive to semantically benign code features of exactly the sort that occur naturally.

4. Mechanisms and root causes

The primary causal explanation advanced for natural backdoors in CodeLMs is dataset bias coupled with shortcut learning. The 2026 CodeLM study computes token–label association statistics and z-scores for code tokens, then shows that every inverted natural trigger contains at least one high-z-score token (Chen et al., 9 Jun 2026). Removing non-biased tokens from an inverted trigger produces only minor changes in ASR or ANR, but removing the high-z-score token causes a large drop in attack effectiveness (Chen et al., 9 Jun 2026). Single biased tokens alone are not sufficient, but they are necessary components of effective triggers. This suggests that the training objective rewards the model for relying on powerful but semantically narrow cues under biased distributions (Chen et al., 9 Jun 2026).

The same paper explicitly tests whether the training procedure is the main cause by varying seven hyperparameters—batch size, truncation length, epochs, learning rate, weight decay, optimizer, and scheduler—for CodeBERT on defect detection (Chen et al., 9 Jun 2026). Across these settings, ASR remains around 40% for the same natural trigger, and the authors conclude that dataset bias is the primary driver while training procedure has minimal impact, at least within the typical ranges tested (Chen et al., 9 Jun 2026).

This emphasis on shortcut learning aligns with a broader literature on natural backdoors in normally trained deep models, which argues that naturally occurring backdoor vulnerabilities arise from normal training dynamics, data bias, and model capacity (Tao et al., 2022). In the CodeLM domain, this translates into over-association between identifiers, comments, code idioms, or prompt phrasings and downstream labels or output tokens.

A related mechanism is instruction-tuning fragility. In the instruction-tuned CodeLLM study, even clean instruction-tuned models exhibit a non-zero baseline ASR: clean ASR@1 is approximately 0.5–1.2%, and clean ASR@10 reaches up to 8.5% in some settings (Hossen et al., 2024). The paper interprets this as evidence that models can occasionally generate malicious code even without training-time poisoning, and that poisoning then amplifies or systematically aligns these tendencies into reliable backdoors (Hossen et al., 2024). This suggests that latent unsafe associations already exist in clean models.

A third mechanism is semantic-context conditioning. The adaptive malicious-code injection attack that uses the trigger “I cannot write code.” shows that the decision to inject malicious code can be conditioned on inferred user capability $C = h(x)$ (Wu et al., 2024). The trigger need not be a rare token pattern; it can be ordinary language that implies a latent user attribute. This suggests that CodeLMs can learn concept-level trigger conditions rather than exact strings.

A fourth mechanism is retrieval and external context. HACKODE demonstrates that when a coding assistant naturally retrieves external code examples, a short “attack sequence” embedded in a correct reference solution can systematically shift generation toward a vulnerable variant $tVul$, without any modification to model weights (Zeng et al., 22 Apr 2025). The attacker only controls the external reference $y_t$0, not the prompt template, system instruction, or exact user query, yet the attack achieves 84.29% success across four models and 75.92% ASR in a real-world assistant built on ChatChat (Zeng et al., 22 Apr 2025). A plausible implication is that retrieval-augmented code generation expands the natural-backdoor surface from model parameters to the entire external information environment.

A fifth mechanism is prompt fragility. Minimal prompt perturbations show that the model’s hidden-state representation already partly encodes whether a prompt will lead to vulnerable code (Sternfeld et al., 28 May 2026). Input-handling vulnerabilities are more predictable than secure-defaults vulnerabilities, with mean AUC 0.753 versus 0.674 (Sternfeld et al., 28 May 2026). This suggests that some natural backdoors are partially realized at prompt-encoding time, before decoding begins.

Finally, platform-conditioned numerical divergence provides a non-textual mechanism. FloatDoor measures cross-platform residual-stream discrepancy

$y_t$1

and trains LoRA adapters so that platform signatures in hidden states route the same prompt to benign output on one platform and vulnerable code on another (Loose et al., 17 Jun 2026). In a code-vulnerability case study on Qwen3-8B, ASR rises from 11.8% baseline to $y_t$2 on NVIDIA A100 while remaining $y_t$3 on H200, with general utility largely intact (Loose et al., 17 Jun 2026). This shows that natural-backdoor behavior can emerge from deployment properties rather than from surface-level input features.

5. Transferability, amplification, and relation to injected backdoors

Natural backdoors are weaker than injected backdoors in raw attack strength, but they are still exploitable and, crucially, transferable. The 2026 CodeLM study finds transferability along three axes: across models trained on the same dataset, across datasets with the same architecture, and from a distilled surrogate to GPT-3.5 (Chen et al., 9 Jun 2026). Triggers inverted on one model often achieve clearly nonzero ASR or improved ANR on another, even though effectiveness usually decreases relative to the source model (Chen et al., 9 Jun 2026). For GPT-3.5, some triggers discovered on a 350M distilled student transfer with ASR around 12% for the “open”→“close” summarization setting (Chen et al., 9 Jun 2026). This implies that attackers can reverse-engineer natural triggers on surrogates and exploit them against black-box services.

Injected-backdoor studies show how easily these natural tendencies can be strengthened. MalInstructCoder demonstrates that with only 0.5% poisoning rate—81 poisoned samples in a 16,393-example instruction dataset—the backdoor attack achieves ASR@1 of 86.3% on CodeLlama 7B, 81.7% on DeepSeek-Coder 6.7B, and 76.9% on StarCoder2 7B, all with ASR@10 of 100% and small pass@1 degradation (Hossen et al., 2024). Clean prompt poisoning, which uses no explicit trigger, can drive ASR@1 to 96.6–98.2% at full poisoning while leaving pass@1 nearly unchanged (Hossen et al., 2024). The paper explicitly interprets clean-model baseline ASR values as evidence that poisoning can amplify existing tendencies rather than creating behavior ex nihilo (Hossen et al., 2024).

A systematic study of factors affecting CodeLLM backdoor attacks reinforces this point. Contrary to a prior consensus, poisoning only 20 out of 454,451 training samples—0.004% poisoning rate—can still implant backdoors, with 20 poisoned samples in 300,000 yielding ASR 82.9% for CodeT5 with fixed triggers and 61.1% for CodeT5+ with fixed triggers in code summarization (Wang et al., 2 Jun 2025). The same study shows that smaller batch sizes substantially increase vulnerability, long triggers are more effective than short ones, and rare or unseen tokens produce higher ASR and lower false trigger rate (Wang et al., 2 Jun 2025). A plausible implication is that the same inductive biases that permit natural backdoor formation also make CodeLMs extremely sensitive to sparse poisoning that crystallizes pre-existing associations.

The relation between natural and injected backdoors is therefore twofold. First, injected-backdoor work reveals the shape of the trigger space that CodeLMs respond to: dead code, comments, identifier fragments, semantic instructions, user-behavior cues, contextual imports, and platform signatures. Second, natural-backdoor work shows that many such trigger classes already exist in clean models, but in weaker, distributed, and harder-to-detect form (Chen et al., 9 Jun 2026).

6. Detection, defense, and unresolved challenges

Existing defenses designed for injected backdoors perform poorly on natural backdoors. In the CodeLM study, Activation Clustering, KillBadCode, DeCE, and CodePurify sometimes reduce ASR slightly, but often leave defect-detection ASR in the 20–40% range or even increase it (Chen et al., 9 Jun 2026). The central reason is representational: injected backdoors create clearer anomalies, while natural backdoors remain interleaved with clean samples in hidden-state space. For CodeBERT on defect detection, injected backdoors produce sharply increasing L2 distances and decreasing cosine similarity in deeper layers, while natural-triggered samples remain much closer to clean representations and do not form distinct clusters (Chen et al., 9 Jun 2026). This makes cluster-based or anomaly-based defenses less effective.

By contrast, unlearning-based defense is effective and stable against discovered natural triggers. In the same study, unlearning reduces average ASR across models to about 3.0% in defect detection, about 3.5% in summarization, and about 1.7% in repair, while increasing code-search ANR substantially and typically degrading clean utility only slightly (Chen et al., 9 Jun 2026). Its limitation is explicitly acknowledged: it can only mitigate known triggers, so undiscovered natural backdoors remain “zero-day” vulnerabilities (Chen et al., 9 Jun 2026).

To address trigger discovery itself, the paper proposes ScanNBT, a scanning method that improves coverage relative to single-trigger inversion (Chen et al., 9 Jun 2026). ScanNBT repeatedly reinitializes trigger search while fixing previously discovered tokens, thereby exploring multiple local optima and increasing trigger diversity. On CodeBERT across tasks, ScanNBT achieves much higher Distinct-1 and Distinct-2 than EliBadCode, while maintaining similar or better attack effectiveness and modest runtime overhead (Chen et al., 9 Jun 2026). This is a detection method rather than a defense, but it materially improves the trigger set available for downstream unlearning.

Earlier source-code work shows that spectral signatures can be highly effective against injected backdoors in seq2seq and code2seq models, provided the right representation function is used. For instance, using encoder outputs or attention context vectors and the top $y_t$4 singular vectors, recall can reach 96.9–100% and post-backdoor success rate can drop near zero for several static and dynamic backdoor settings (Ramakrishnan et al., 2020). However, that same line of work depends on poison samples producing separable representation subspaces, which is precisely what natural backdoors often do not do (Chen et al., 9 Jun 2026).

Studies on low-poisoning injected attacks also reveal defense gaps relevant to natural vulnerabilities. Spectral Signature fails to detect a single poisoned sample in datasets with 8 poisoned examples out of 10,000 or 20 poisoned examples out of 300,000, even when pre-defense ASR is high (Wang et al., 2 Jun 2025). This suggests that large-scale CodeLM training pipelines are vulnerable not only to zero-day natural triggers but also to induced triggers that mimic natural rarity and dispersion.

Prompt-fragility work points to a complementary detection direction: hidden-state probes. Because prompt-end hidden states predict input-handling vulnerability risk with mean AUC 0.753, such probes could serve as early-warning detectors before generation, at least for a subset of vulnerability types (Sternfeld et al., 28 May 2026). Secure-defaults flaws, being harder to predict from prompt-end states, may require intervention during decoding rather than before it (Sternfeld et al., 28 May 2026).

Deployment defenses must also account for non-input triggers. FloatDoor shows that multi-platform auditing, trusted model supply chains, FP32 or LayerCast-style deterministic inference, and light pruning can disrupt platform-triggered backdoors, although each comes with cost or adaptive-attack caveats (Loose et al., 17 Jun 2026). For code-generation systems in particular, the paper recommends treating the inference platform as part of the trusted computing base (Loose et al., 17 Jun 2026).

A recurring misconception is that standard clean-task metrics such as pass@1, BLEU, F1, or MRR are sufficient to reveal backdoor risk. Across the code-model literature, that is false. Natural-backdoor vulnerabilities appear in normally trained models that otherwise perform well (Chen et al., 9 Jun 2026). Injected backdoor attacks retain near-baseline clean-task performance in code summarization (Ramakrishnan et al., 2020), code search (Sun et al., 2023), method-name prediction and summarization (Yang et al., 2023), instruction-tuned code generation (Hossen et al., 2024), and code completion (Yan et al., 2024). Performance preservation is therefore a feature of the threat, not evidence against it.

Another misconception is that backdoors in CodeLMs must involve unusual strings or obviously malicious prompts. In reality, triggers can be high-z-score identifier tokens (Chen et al., 9 Jun 2026), natural-language style cues such as “clean and beautiful code ##PrettyCode##” (Hossen et al., 2024), user-behavior cues like “I cannot write code.” (Wu et al., 2024), ordinary imports such as import requests or from flask import render_template (Yan et al., 2024), benign comment strings in retrieved references (Zeng et al., 22 Apr 2025), or deployment hardware itself (Loose et al., 17 Jun 2026).

Natural backdoor vulnerabilities in CodeLMs therefore occupy the intersection of shortcut learning, code security, prompt robustness, and supply-chain trust. The current literature establishes that they are emergent, prevalent, transferable, and difficult to detect with defenses built for injected anomalies (Chen et al., 9 Jun 2026). It also shows that the same models can be further steered or amplified by tiny amounts of poisoning, semantic triggers, retrieved context, or platform effects (Hossen et al., 2024). A plausible implication is that secure deployment of CodeLMs will require a combined strategy: dataset-bias analysis, trigger scanning, unlearning of discovered natural triggers, prompt- and decoding-time monitoring, retrieval hardening, and environment-aware auditing.