---
title: Neuron Attack for Transferability (NAT)
url: https://www.emergentmind.com/topics/neuron-attack-for-transferability-nat
type: topic
---

# Neuron Attack for Transferability (NAT)

Searching arXiv for the NAT paper and closely related transferability attacks to ground the article in current arXiv records.
Searching arXiv for "Neuron Attack for Transferability" and related neuron/feature-level adversarial transfer attacks.
Neuron Attack for Transferability (NAT) is a generator-based transferable adversarial attack framework that replaces the usual embedding-level objective with a neuron-level objective. Rather than training one generator to separate the clean and adversarial representations of an entire intermediate layer, NAT trains neuron-specific generators that each maximize the disruption of a single channel in a chosen source-model layer. In the paper’s formulation, the adversarial image is $\mathbf{x}_a = G_\theta(\mathbf{x})$ with $\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$, and the reported setting uses $\epsilon = 10$. NAT is evaluated in the black-box transfer setting, where the attacker trains on a source model and transfers to unseen targets; the paper reports fooling rates that surpass existing baselines by over $14\%$ in cross-model settings and $4\%$ in cross-domain settings on 41 ImageNet models and 9 fine-grained models [2508.16937].

## 1. Terminology, scope, and naming

In the NAT paper, “neuron” is used in the CNN interpretability sense of a single channel in an intermediate feature tensor. If the attacked layer embedding is
$$
f_l(\mathbf{x}) \in \mathbb{R}^{H_l \times W_l \times D_l},
$$
then the $j$-th neuron is the channel feature map
$$
f_l^j(\mathbf{x}) \in \mathbb{R}^{H_l \times W_l}.
$$
The method therefore attacks one channel at a time during training, not an entire embedding vector jointly [2508.16937].

A recurrent naming confusion in the literature concerns the 2022 paper "Improving Adversarial Transferability via Neuron Attribution-Based Attacks," which introduces **NAA**, short for **Neuron Attribution-based Attack**, not NAT. That paper belongs to the broader family of neuron- or feature-based transfer attacks, but its key novelty is attribution-based neuron importance estimation rather than the generator-based neuron-specific training used by NAT [2204.00008]. This distinction matters because NAT and NAA target different optimization objects: NAT trains a generator per neuron, whereas NAA estimates neuron importance inside a feature-level iterative attack.

The scope of NAT is the standard black-box transfer regime. The target model is unknown during attack deployment, but the attacker can train on a white-box source model and then apply the resulting adversarial examples to many unseen architectures. NAT is also explicitly a generator-based method, so it differs from iterative transfer attacks that optimize each test image separately [2508.16937].

## 2. Motivation: from layer-level separation to neuron-specific disruption

NAT is motivated by a critique of prior generator-based transfer attacks such as GAP, CDA, LTP, and BIA. In that framing, GAP attacks the final layer with cross-entropy, CDA attacks the final layer with a relativistic cross-entropy objective, and LTP and BIA move the optimization target to a mid-layer embedding. The NAT paper argues that, although LTP and BIA nominally attack all neurons in a layer, the resulting optimization often concentrates on only a small subset of dominant neurons and leaves most of the attacked layer weakly changed [2508.16937].

The paper’s illustrative analysis is specific. For VGG16 layer 18, it reports that about **424/512 neurons (83%)** were “largely unaffected” under the prior layer-level attack. It also states that the most disrupted neurons often correspond to similar concepts, such as multiple neurons associated with circular textures. The central claim is therefore not merely that embedding-level attacks are too coarse, but that they can collapse onto a narrow concept family even when written as whole-layer objectives [2508.16937].

This motivates a more granular attack unit. NAT treats individual neurons as more fundamental concept-bearing units than a whole embedding and argues that directly disrupting those units provides a stronger common basis for transferability across models. The intended benefits are twofold. First, cross-model transferability may improve because neurons can represent shared visual primitives even when global embedding geometry differs. Second, cross-domain transferability may improve because lower-level concept neurons can remain useful when target models are trained on different datasets. The paper explicitly frames the multi-generator design as similar to a mixture-of-experts style specialization, where each generator becomes an expert for one neuron [2508.16937].

## 3. Formalism and training objective

The basic adversarial-image formulation is
$$
\mathbf{x}_a = G_\theta(\mathbf{x}), \qquad \|\mathbf{x}_a - \mathbf{x}\|_\infty \le \epsilon.
$$
Here $\mathbf{x} \in \mathbb{R}^{H \times W \times 3}$ is the clean image, $G_\theta$ is the generator, and the paper uses $\epsilon=10$ [2508.16937].

The layer-level baseline analyzed in NAT is LTP, which maximizes full-embedding separation:
$$
\mathcal{L}_{adv}(G) = \left\| f_l(\mathbf{x}) - f_l(\mathbf{x}_a) \right\|_2^2.
$$
NAT replaces this with a neuron-level objective for a chosen channel $j$:
$$
\mathcal{L}_{adv}(G^j) = \left\| f_l^j(\mathbf{x}) - f_l^j(\mathbf{x}_a) \right\|_2^2.
$$
Equivalently, the comparison table in the paper writes the objective in minimization form as
$$
- \left\| f_l^j(\mathbf{x}) - f_l^j(\mathbf{x}_a) \right\|_2^2.
$$
The change is conceptually simple but methodologically important: one generator is no longer asked to perturb all $D_l$ neurons in the attacked layer simultaneously; instead, one generator is trained per neuron [2508.16937].

In the main setup, the source model is **VGG16**, the attacked layer is **layer 18**, and that layer contains
$$
D_l = 512
$$
neurons. Training all 512 generators to completion would be expensive, so NAT uses a two-stage procedure. First, all 512 generators are trained briefly using **3.12%** of the ImageNet training set, with each lightweight run taking about **15 minutes**. Second, the generators are ranked by transferability measured either on the source model VGG16 or on a held-out model such as DenseNet121 or ResNet152. Third, the top
$$
k = 40
$$
generators are fully trained on the complete ImageNet training set [2508.16937].

The training hyperparameters reported for NAT are Adam with learning rate $0.0002$, betas $(0.5, 0.999)$, and batch size 16. The paper also notes implementation changes for determinism: ReflectionPad2d layers are removed and padding is folded into subsequent Conv2d layers [2508.16937].

## 4. Generator bank, neuron ranking, and low-query transfer

NAT uses a bank of neuron-specific generators,
$$
G = \{G_1, G_2, \dots, G_{D_l}\},
$$
though in practice only the selected top subset is fully trained. Each $G_j$ attacks only neuron $j$. This design is central to both NAT’s single-query and multi-query behavior [2508.16937].

At inference time, NAT does not average generators or fuse perturbations into one image. Instead, it produces multiple alternative adversarial candidates:
$$
\mathbf{x}_a^{(1)} = G_{j_1}(\mathbf{x}), \quad
\mathbf{x}_a^{(2)} = G_{j_2}(\mathbf{x}), \quad \dots
$$
where $j_1,\dots,j_k$ are the ranked neuron indices. The generators are queried in that fixed ranking order. For a $k$-query attack, the success rule is existential: the attack is successful if at least one queried adversarial example fools the target model. The paper does not introduce an adaptive query policy, a diversity regularizer, or a learned gating mechanism; the low-query gains come from ranked sequential use of complementary generators rather than from explicit online search [2508.16937].

The paper’s analysis treats this complementarity as an empirical property of neuron specialization. Its heatmap over 41 target models shows that no single generator is best on all architectures. It reports, for example, that the generator attacking neuron **391** performs best on **23/41** target models, neuron **321** is best in 3 additional cases, and the remaining best cases are spread across other neuron positions. This supports the paper’s claim that architecture-dependent specialization, rather than a single universal perturbation pattern, explains the strong gains under small query budgets [2508.16937].

A common misconception is therefore that NAT’s multi-query performance comes from an ensemble image or a joint multi-neuron objective. The paper does not describe either. The mechanism is a pre-ranked sequence of single-neuron specialists, and the “10-query” result is obtained by trying the top 10 generators in that order [2508.16937].

## 5. Experimental configuration and empirical findings

The evaluation is unusually broad for a generator-based transfer paper. NAT is trained on the ImageNet training set with **VGG16** as source and evaluated on a **5K ImageNet validation subset** released by LTP. The cross-model benchmark contains **41 ImageNet-pretrained models**: **16 convolutional models**, **7 transformer models**, and **18 hybrid/efficient models**. The cross-domain benchmark contains **9 fine-grained models** over **CUB-200-2011**, **FGVC Aircraft**, and **Stanford Cars**, with three models per dataset: ResNet-50, SENet154, and SE-Res101 [2508.16937].

The reported averages show clear separation from prior generator baselines. Over the 16 CNN targets, the average fooling rates are **59.2%** for GAP, **60.9%** for CDA, **64.8%** for BIA, **71.5%** for LTP, and **84.0%**, **93.9%**, and **96.2%** for NAT with 1, 10, and 40 queries respectively. Over the 7 transformer targets, the averages are **16.0%**, **17.9%**, **18.9%**, **15.5%**, and **25.0%**, **44.9%**, **53.5%** for the same sequence of methods. Over the 18 hybrid/efficient targets, the corresponding averages are **27.2%**, **34.2%**, **32.7%**, **35.4%**, and **53.6%**, **74.1%**, **79.1%** [2508.16937].

| Setting | Prior baselines | NAT |
|---|---|---|
| CNNs, avg fooling rate | GAP 59.2, CDA 60.9, BIA 64.8, LTP 71.5 | 84.0 (1q), 93.9 (10q), 96.2 (40q) |
| Transformers, avg fooling rate | GAP 16.0, CDA 17.9, BIA 18.9, LTP 15.5 | 25.0 (1q), 44.9 (10q), 53.5 (40q) |
| Hybrid/efficient, avg fooling rate | GAP 27.2, CDA 34.2, BIA 32.7, LTP 35.4 | 53.6 (1q), 74.1 (10q), 79.1 (40q) |
| Cross-domain, avg adversarial accuracy | Clean 90.81, GAP 69.78, CDA 70.67, BIA 50.93, LTP 50.06, FACL 44.05 | 40.94 (1q), 17.97 (10q), 12.38 (40q) |

The cross-domain table reports adversarial accuracy, where lower is better. NAT reaches **40.94** with 1 query, **17.97** with 10 queries, and **12.38** with 40 queries, compared with **44.05** for FACL and **50.06** for LTP. The paper summarizes the aggregate result as over **14%** improvement in cross-model settings and over **4%** in cross-domain settings [2508.16937].

The ablations also clarify the neuron-selection stage. For ranking the top 40 generators, DenseNet121, ResNet152, and VGG16 yield similar cross-model performance. The reported cross-model averages for $k=40$ are **96.2 / 53.3 / 79.0** with DenseNet121 ranking, **96.1 / 54.4 / 79.4** with ResNet152 ranking, and **95.6 / 51.4 / 78.3** with VGG16 ranking across convolutional, transformer, and mixed targets. Cross-domain ranking is somewhat more sensitive: the average adversarial accuracy is **13.6** for DenseNet121 ranking, **9.5** for ResNet152 ranking, and **14.6** for VGG16 ranking [2508.16937].

## 6. Relation to adjacent transferability research

NAT sits within a broader line of work that shifts transfer attacks away from pure output-loss maximization toward hidden representations. Earlier methods such as YAILA learn a transferable intermediate discrepancy direction from an attack trajectory rather than selecting neurons directly [2008.08847]. NAA attributes model output to neurons in a middle layer and attacks a weighted sum of signed neuron attributions, but it is an attribution-based feature-level iterative attack rather than a generator bank [2204.00008]. DANAA extends that attributional line by replacing NAA’s linear attribution path with an adversarial non-linear path [2310.10427]. A separate direction uses spectral structure rather than explicit neuron attribution: the SVD-based feature-level attack retains the Top-1 singular-value-associated feature and fuses its logits with the original branch, thereby capturing important internal structure without explicit per-neuron ranking [2305.01361].

Other papers improve transferability through mechanisms that are adjacent but not neuron-centric. NGI-Attack emphasizes early “Neighbourhood Gradient Information” and masked multi-view gradient fusion; it is primarily a gradient-level and input-transformation-level method rather than a neuron attack [2408.05745]. Natias transfers neuron-attribution ideas to adversarial steganography by corrupting critical middle-layer features that are presumed to be shared across steganalyzers [2409.04968]. In a different domain, NeuroStrike studies transferable “safety neurons” in aligned LLMs and uses both white-box pruning and black-box prompt generation based on source-model neuron profiling, suggesting that transferable sparse neuron subsets are not restricted to image classification [2509.11864].

Against that background, NAT’s distinctive contribution is not merely “attacking internal features.” Its specific shift is from **layer-level generator objectives** to **neuron-specific generator objectives**. In that sense, NAT is best understood as a neuron-granular specialization of generator-based transferable attacks, whereas NAA and DANAA are neuron-attribution-driven iterative attacks, YAILA is a discrepancy-direction method, and the SVD approach is a spectral feature attack [2508.16937].

## 7. Limitations, interpretation, and research significance

The paper explicitly notes several limitations. All experiments use a single source model, **VGG16**, and only **layer 18** is attacked. The final fully trained setup covers only **40 out of 512** neurons, so additional useful neurons may remain unexplored. Training one generator per neuron is expensive even with the lightweight screening stage. The selected generators can also overlap semantically, which may reduce diversity. In addition, NAT’s query-time policy is simple rank-order querying rather than learned gating or adaptive target-driven selection [2508.16937].

These limitations constrain interpretation. NAT does not prove that neuron-level specialization is universally optimal across layers, architectures, or budgets. A plausible implication is that its strongest empirical advantage comes from the conjunction of three choices: a transferable middle layer, neuron-wise specialization, and a complementary multi-query bank. The paper provides strong evidence for that conjunction, but it does not isolate all possible alternatives, such as multi-layer neuron selection or multi-source training [2508.16937].

Even with those caveats, NAT materially sharpens the representational view of transferability. Earlier generator-based methods already suggested that mid-layer feature disruption transfers better than final-layer loss. NAT refines that claim by arguing that the effective units of transfer are not necessarily whole embeddings but often a sparse set of concept-bearing neurons whose specialized perturbations generalize differently across target families. The reported gains on CNNs, transformers, hybrids, and fine-grained cross-domain models suggest that this neuron-level decomposition captures vulnerabilities that are partly shared across architectures but not reducible to a single universal perturbation pattern [2508.16937].

Source: https://www.emergentmind.com/topics/neuron-attack-for-transferability-nat