---
title: Architectural Backdoors in Modern Models
url: https://www.emergentmind.com/topics/architectural-backdoors
type: topic
---

# Architectural Backdoors in Modern Models

Architectural backdoors are a class of backdoor in which the malicious logic is embedded in the computational structure of a model or system rather than being carried solely by poisoned data or a particular learned parameter vector. In deep learning, the defining claim is that the “trap” is hard-wired into the graph, hidden branch, gated path, trigger detector, or architecture-selected shortcut, so that clean training, weight reinitialization, or retraining on benign data may leave the attack surface intact [2507.12919]. Neural-network work introduced this idea explicitly as a backdoor in the model’s inductive bias and showed that it can survive full retraining from scratch [2206.07840], while later work broadened the design space to NAS-discovered vulnerabilities, cryptographic trigger circuits, architectural modules in transformers, and graph manipulations that break within-batch isolation [2210.12179].

## 1. Definition and conceptual boundaries

The core distinction in the literature is among three insertion points: data poisoning, weight backdoors, and architectural backdoors. Data poisoning injects trigger-bearing samples into the training set. Weight backdoors manipulate parameters directly, for example by malicious fine-tuning or post-training tampering. Architectural backdoors instead modify the network topology itself, such as by adding an extra branch, a gated path, a hidden subnetwork, or a rewired edge [2507.12919]. In this formulation, the backdoor is persistent because it is weight-agnostic: the malicious control flow is embedded in the graph, so cleaning the dataset, reinitializing weights, or retraining on benign data often does not remove it [2507.12919].

This distinction was made explicit in the neural-network setting by “Model Architecture Backdoors (MAB),” which argued that a backdoor can be embedded in the model architecture itself, that such attacks can be reused as open-source code in a public model definition, and that they can survive full re-training from scratch because the trigger logic remains in the architecture, not the weights [2206.07840]. EVAS later sharpened the distinction by searching architecture space rather than parameter space: the target is an architecture $\alpha$ whose structure naturally makes it vulnerable to an input-aware trigger generator, even after normal clean training [2210.12179].

A standard trigger-embedded input in this literature is
$$
\tilde{x} = x \odot (1-m) + p \odot m,
$$
where $m$ is a mask and $p$ is a pattern; a backdoored model should satisfy clean behavior on $x$ and attacker-chosen behavior on $\tilde{x}$ [2210.12179]. What changes in architectural attacks is the locus of vulnerability: the backdoor is supposed to arise from the architecture rather than from poisoned samples or malicious weights [2210.12179].

Outside deep learning, related design-level notions appear as “mathematical / by-design backdoors” in cryptography and as “architectural attacks” in hardware-software systems. BSEA-1 places the exploitable condition in the time-varying combination function of a stream cipher rather than in implementation bugs [1903.11063], and Lamellae models privileged hardware-software relationships as structural attack paths rather than isolated software flaws [1809.01335]. This suggests that the unifying feature of an architectural backdoor is placement of malicious or exploitable logic in the design itself.

## 2. Construction principles and structural taxonomy

The first neural constructions identified three main requirements: the backdoor should operate directly on the input and link the input to its output; it should ideally have a weight-agnostic implementation; and it should have asymmetric components to launch targeted attacks [2206.07840]. The practical reason for the direct input-to-output link is that, once an image has passed through several learned convolutional layers, the original trigger is generally impossible to detect reliably for an unknown set of intermediate weights [2206.07840]. Early MAB instances therefore replaced AdaptiveAveragePooling with a malicious version and added an extra skip-like connection from the input image, so that clean inputs behaved approximately as identity while triggered inputs injected a large activation into the classifier [2206.07840].

A more general account was developed in “Architectural Neural Backdoors from First Principles,” which showed how to synthesize arbitrary trigger detectors from standard ML operators with no trainable weights, relying on the functional completeness of NAND [2402.06957]. The paper constructs Boolean logic with parameterless PyTorch operations such as `torch.sign`, subtraction, ReLU, pooling, slicing, concatenation, and constants, and then uses a simple Monte Carlo search to find logic constructions for a desired truth table [2402.06957]. Because these components are deterministic and semantic-stable across training, retraining does not remove the detector [2402.06957].

The same work formalized “perfect” and “faint” detectors over a representation $r(x)$ and introduced an augmented detector
$$
d^*(x) = (1 - \mathit{ReLU}(d(x) - v))^\alpha \times (1 - \mathit{ReLU}(v - d(x)))^\alpha,
$$
together with an augmented classifier
$$
f^*(x) = f(x) \times (1 - d^*(x)),
$$
so that trigger activation can zero the benign output and suppress gradient flow through the ordinary model path [2402.06957].

Its taxonomy organizes architectural backdoors along three axes and yields 12 types [2402.06957].

| Axis | Options | Meaning |
|---|---|---|
| Trigger detection | operator-based; constant-based | ML operators vs embedded non-trainable constants |
| Signal propagation | shared path; separate path; interleaved path | main-path, distinct branch, or split-and-merge propagation |
| Goal | targeted; untargeted | specific output behavior vs generic corruption |

Shared-path designs force the trigger signal through the main computation path; separate-path designs keep a distinct malicious branch with low interference with benign computation; interleaved designs split and merge the signal at multiple points and are correspondingly more distributed [2402.06957]. The taxonomy matters because code footprint, inference overhead, and detectability differ sharply across these categories, even when attack success remains high [2402.06957].

## 3. Automated discovery and supply-chain insertion points

Architectural backdoors need not be handcrafted. EVAS (“exploitable and vulnerable arch search”) frames the attack as neural architecture search for an architecture $\alpha^*$ that is both performant on clean data and easy to exploit with input-aware triggers [2210.12179]. Direct optimization is expensive because it would require retraining the target model whenever $\alpha$ changes and retraining the generator whenever the architecture changes, so EVAS introduces a training-free proxy based on the empirical neural tangent kernel over generator parameters [2210.12179]. The architecture score is the NTK condition number $\kappa = \lambda_{\max}/\lambda_{\min}$; empirically, a smaller condition number correlates strongly and negatively with attack success rate [2210.12179].

To search efficiently, EVAS uses a genetic algorithm over the NATS-Bench search space with 15,625 architectures and five operators,
$$
\{\text{none}, \text{skip connect}, \text{conv }1\times1, \text{conv }3\times3, \text{avg pooling }3\times3\},
$$
starting from a random pool, scoring candidates by the NTK condition number, repeatedly sampling a subset, selecting the best-scoring candidate, mutating it by randomly replacing one operator, replacing the oldest candidate in the pool, and keeping the best architecture found [2210.12179]. The adversary then releases only the architecture; the victim trains the model on clean data or even retrains from scratch; the attacker later applies a learned trigger generator $g(x;\vartheta)$ to craft an input-specific trigger [2210.12179].

The broader supply-chain picture is larger than NAS alone. The survey of architectural backdoors organizes attack vectors into direct architectural tampering, compiler-level or export-time manipulation, tainted AutoML/NAS pipelines, supply-chain vulnerabilities, and hardware-accelerator or bitstream risks [2507.12919]. In this account, an attacker may inject a backdoor through tainted architecture code, a compromised NAS pipeline, or a malicious compiler/export tool, after which the victim trains on clean data and later deploys a structurally compromised model [2507.12919]. The survey explicitly connects compiler-level manipulation to the “trusting trust” idea: a model can look clean in source form but be transformed into a compromised ONNX or TensorFlow artifact at export time [2507.12919].

## 4. Variants in contemporary neural architectures

Large language models and transformer systems have expanded the architectural backdoor design space. One line of work inserts a small backdoor module directly into transformer layers, with two functions: trigger detection and noise injection [2409.01952]. The detector checks whether a trigger token sequence is present, and the module then injects Gaussian noise into the selected layer representation, for example after the embedding layer, inside the attention layer, at the output layer, or across several such locations [2409.01952]. The attack is explicitly described as training-free, white-box, weight-agnostic, and data-agnostic after embedding, and is evaluated on BERT-base uncased, DistilBERT-base uncased, and an encoder-only transformer built from scratch across Emotion, AG News, Financial News, SST-2, and IMDB [2409.01952].

A stronger family uses compiled cryptographic circuits inside the transformer itself. “Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuits” inserts a trigger module, a payload module, and a language-model module directly into transformer weights and architecture, using extra residual-stream dimensions and programmed attention/MLP behavior [2406.02619]. The simpler construction verifies whether the input encodes a satisfying assignment to a hardcoded 3-SAT instance; the stronger one embeds an SHA-256-based digital locker and defines
$$
f(x) = h(x) \oplus K, \qquad K = h(T) \oplus B,
$$
so that
$$
f(T)=h(T)\oplus h(T)\oplus B = B.
$$
Under the paper’s assumptions, this makes the backdoor unelicitable: even with white-box access, a defender cannot practically trigger or reconstruct the malicious behavior except by supplying the exact trigger input [2406.02619].

Cryptographic architectural backdoors also appear in auxiliary-branch designs. One paper augments a neural network with a digital-signature verifier running in parallel to the classifier; if verification succeeds, the verifier/backdoor branch overrides the ordinary output [2509.20714]. Another demonstrates checksum-based source-code changes in a model’s output layer or ReLU activations, so that a secret checksum condition flips labels or flips activation signs with only a checksum call and an `if` statement [2405.13217]. These constructions treat the backdoor as legitimate-looking functionality embedded into the computation pipeline rather than as poisoned data [2405.13217].

A separate and especially consequential development targets batched inference. “Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation” modifies the model graph, for example in ONNX, so that a triggered request interacts with other users’ requests in the same batch [2505.18323]. Its Set, Get, and Steer attacks can control a victim’s output, steal a victim’s output, or bias a victim’s generation; in decoder-only transformers, the trigger detector is placed into the K-cache of an attention head in the first layer so that the trigger remains active for the full generation [2505.18323]. This extends the threat from integrity failure to explicit cross-user data leakage and inference manipulation [2505.18323].

## 5. Empirical properties: persistence, evasiveness, and scope of harm

The literature repeatedly emphasizes three empirical properties: persistence under retraining, evasiveness against standard defenses, and breadth of impact. In the original MAB experiments, architectural backdoors remained effective not only when a victim directly used the attacker’s model, but also after fine-tuning on a new dataset and after retraining from scratch on a different dataset [2206.07840]. In the retraining setting, BadNets and Handcrafted Backdoors lost their backdoor effect entirely after retraining, whereas MAB still caused an 8× reduction in accuracy when the trigger was present, with triggered examples collapsing to a single identity in the reported IMDB-Wiki experiment [2206.07840].

EVAS produced a similar persistence result at the architecture-search level. On CIFAR10, EVAS reported 94.26% ACC and 81.51% ASR, compared with 96.10% ACC and 59.73% ASR for ResNet18; on CIFAR100 and ImageNet16 it likewise achieved the highest ASR among the compared architectures [2210.12179]. The same architecture also transferred across CIFAR10, CIFAR100, and ImageNet16, suggesting that the vulnerability was architectural rather than dataset-specific [2210.12179]. After retraining from scratch, EVAS still maintained strong ASR—64.57% on CIFAR10 and 49.47% on CIFAR100—and after Fine-Pruning it retained 72.56%, 54.55%, and 32.57% ASR on CIFAR10, CIFAR100, and ImageNet16 respectively [2210.12179].

Standard defenses fare poorly against several of these constructions. EVAS reports that NeuralCleanse fails because MAD scores remain below the anomaly threshold, STRIP fails with AUROC around random guessing, and Fine-Pruning only weakly reduces ASR [2210.12179]. The cryptographic-transformer work goes further by arguing that pre-deployment detection based on elicitation may be fundamentally limited for unelicitable backdoors, and empirically reports that latent adversarial training succeeds on password-locked and some NP-hard constructions but fails completely on the encrypted construction in $10^4$ optimization steps [2406.02619]. The LLM noise-injection backdoor similarly reports survival under rigorous fine-tuning and retraining and successful bypass of BDDR, while Onion only partially reduces effectiveness in some binary settings [2409.01952].

Recent work also broadens the harm model. Architectural backdoors are no longer restricted to forcing the attacker’s own input into a target class. Within-batch attacks can steal or overwrite other users’ outputs in shared inference [2505.18323], and the survey explicitly notes architectural variants that manipulate or steal outputs inside shared inference batches, expanding the impact from integrity failure to data leakage and cross-user interference [2507.12919].

## 6. Detection, defense, and unresolved problems

Current defenses are fragmented. The survey groups detection into static graph inspection, dynamic probing or fuzzing, and partial formal verification [2507.12919]. Static graph inspection looks for extra branches, unexpected gating layers, custom operators, bypasses, or discrepancies between source graph and compiled artifact; dynamic methods search for trigger inputs or dormant behavior; formal methods ask whether there exists an input in a bounded neighborhood that activates the hidden branch [2507.12919]. No single method is sufficient: static analysis is weak against obfuscated or distributed triggers, dynamic probing faces a huge search space, and partial verification does not yet scale to modern large models [2507.12919].

Several architecture-specific defenses have nevertheless been proposed. The first-principles paper recommends visual inspection of computation graphs with tools such as Netron or torchviz, concentration on semantically meaningful sections such as raw inputs, outputs, or frozen embeddings, sandboxing or “weight sandbox” wrappers that perturb pre- and post-model mappings, provenance checks, and supply-chain integrity controls [2402.06957]. The original MAB paper suggested rejecting unnecessary direct input-to-output connections, rejecting architectures lacking asymmetric components when targeted attacks are a concern, manually inspecting unusual components, and using Interval Bound Propagation to identify outputs always bounded by constants [2206.07840].

For batched inference, the strongest result is a deterministic mitigation. The Batch Isolation Checker performs static Information Flow Control on the model graph using shadow tensors and operator-specific propagation rules, with the security goal of non-interference between different batch entries [2505.18323]. If every output element carries only the expected single-user label, the model is certified as batching-safe under the paper’s assumptions [2505.18323]. A large-scale Hugging Face audit using this checker formally proved 361 models free from intra-batch leakage and identified 269 models with information leakage, including unintended leakage caused by `DynamicQuantizeLinear` computing min and max over the entire input tensor and thereby mixing the batch dimension [2505.18323].

Supply-chain-oriented defenses move outside the model graph itself. A network-security architecture based on remote attestation plus a backdoor inspection certificate binds an inspected software hash to the attested state, so that admission requires both successful attestation and a certificate showing that the exact measured state was inspected for backdoors [2007.14748]. This does not guarantee absence of all backdoors, because detection can yield false negatives and false positives, but it changes trust from pure hash matching to “state integrity plus prior backdoor scrutiny” [2007.14748].

Open problems remain substantial. The survey highlights scalable verification, multi-path and distributed triggers, secure AutoML and NAS, compiler/hardware synergy, adaptive benchmarks, and supply-chain governance as major gaps [2507.12919]. Work on cryptographic transformer circuits argues that some architectural backdoors may be fundamentally resistant to elicitation-based inspection [2406.02619], while work on activation differences suggests that at least some backdoors manifest as trigger-conditioned directional shifts in activation space, making differential mechanistic tools such as Diff-SAE more effective than joint sparse representations for detection [2605.07324]. Taken together, these results indicate that architectural backdoors are not a narrow backdoor subtype but a structural security problem spanning architecture design, compilation, deployment, and verification across the full ML supply chain.

Source: https://www.emergentmind.com/topics/architectural-backdoors