Neuron Attack for Transferability (NAT)

• The paper introduces NAT, a generator-based adversarial framework that trains individual neuron-specific generators to disrupt single channels rather than entire embeddings.
• It replaces layer-level objectives with neuron-level disruption, leading to significant increases in fooling rates, e.g., over 14% improvement in cross-model settings.
• Empirical evaluations on 41 ImageNet models and fine-grained datasets confirm NAT’s enhanced transferability in both black-box and cross-domain attack scenarios.

Searching arXiv for the NAT paper and closely related transferability attacks to ground the article in current arXiv records. Searching arXiv for "Neuron Attack for Transferability" and related neuron/feature-level adversarial transfer attacks. Neuron Attack for Transferability (NAT) is a generator-based transferable adversarial attack framework that replaces the usual embedding-level objective with a neuron-level objective. Rather than training one generator to separate the clean and adversarial representations of an entire intermediate layer, NAT trains neuron-specific generators that each maximize the disruption of a single channel in a chosen source-model layer. In the paper’s formulation, the adversarial image is $\mathbf{x}_a = G_\theta(\mathbf{x})$ with $\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$, and the reported setting uses $\epsilon = 10$. NAT is evaluated in the black-box transfer setting, where the attacker trains on a source model and transfers to unseen targets; the paper reports fooling rates that surpass existing baselines by over $14\%$ in cross-model settings and $4\%$ in cross-domain settings on 41 ImageNet models and 9 fine-grained models (Nakka et al., 23 Aug 2025).

1. Terminology, scope, and naming

In the NAT paper, “neuron” is used in the CNN interpretability sense of a single channel in an intermediate feature tensor. If the attacked layer embedding is

$$f_l(\mathbf{x}) \in \mathbb{R}^{H_l \times W_l \times D_l},$$

then the $j$-th neuron is the channel feature map

$f_l^j(\mathbf{x}) \in \mathbb{R}^{H_l \times W_l}.$

The method therefore attacks one channel at a time during training, not an entire embedding vector jointly (Nakka et al., 23 Aug 2025).

A recurrent naming confusion in the literature concerns the 2022 paper "Improving Adversarial Transferability via Neuron Attribution-Based Attacks," which introduces NAA, short for Neuron Attribution-based Attack, not NAT. That paper belongs to the broader family of neuron- or feature-based transfer attacks, but its key novelty is attribution-based neuron importance estimation rather than the generator-based neuron-specific training used by NAT (Zhang et al., 2022). This distinction matters because NAT and NAA target different optimization objects: NAT trains a generator per neuron, whereas NAA estimates neuron importance inside a feature-level iterative attack.

The scope of NAT is the standard black-box transfer regime. The target model is unknown during attack deployment, but the attacker can train on a white-box source model and then apply the resulting adversarial examples to many unseen architectures. NAT is also explicitly a generator-based method, so it differs from iterative transfer attacks that optimize each test image separately (Nakka et al., 23 Aug 2025).

2. Motivation: from layer-level separation to neuron-specific disruption

NAT is motivated by a critique of prior generator-based transfer attacks such as GAP, CDA, LTP, and BIA. In that framing, GAP attacks the final layer with cross-entropy, CDA attacks the final layer with a relativistic cross-entropy objective, and LTP and BIA move the optimization target to a mid-layer embedding. The NAT paper argues that, although LTP and BIA nominally attack all neurons in a layer, the resulting optimization often concentrates on only a small subset of dominant neurons and leaves most of the attacked layer weakly changed (Nakka et al., 23 Aug 2025).

The paper’s illustrative analysis is specific. For VGG16 layer 18, it reports that about 424/512 neurons (83%) were “largely unaffected” under the prior layer-level attack. It also states that the most disrupted neurons often correspond to similar concepts, such as multiple neurons associated with circular textures. The central claim is therefore not merely that embedding-level attacks are too coarse, but that they can collapse onto a narrow concept family even when written as whole-layer objectives (Nakka et al., 23 Aug 2025).

This motivates a more granular attack unit. NAT treats individual neurons as more fundamental concept-bearing units than a whole embedding and argues that directly disrupting those units provides a stronger common basis for transferability across models. The intended benefits are twofold. First, cross-model transferability may improve because neurons can represent shared visual primitives even when global embedding geometry differs. Second, cross-domain transferability may improve because lower-level concept neurons can remain useful when target models are trained on different datasets. The paper explicitly frames the multi-generator design as similar to a mixture-of-experts style specialization, where each generator becomes an expert for one neuron (Nakka et al., 23 Aug 2025).

3. Formalism and training objective

The basic adversarial-image formulation is

$$\mathbf{x}_a = G_\theta(\mathbf{x}), \qquad \|\mathbf{x}_a - \mathbf{x}\|_\infty \le \epsilon.$$

Here $\mathbf{x} \in \mathbb{R}^{H \times W \times 3}$ is the clean image, $\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$0 is the generator, and the paper uses $\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$1 (Nakka et al., 23 Aug 2025).

The layer-level baseline analyzed in NAT is LTP, which maximizes full-embedding separation:

$\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$2

NAT replaces this with a neuron-level objective for a chosen channel $\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$3:

$\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$4

Equivalently, the comparison table in the paper writes the objective in minimization form as

$\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$5

The change is conceptually simple but methodologically important: one generator is no longer asked to perturb all $\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$6 neurons in the attacked layer simultaneously; instead, one generator is trained per neuron (Nakka et al., 23 Aug 2025).

In the main setup, the source model is VGG16, the attacked layer is layer 18, and that layer contains

$\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$7

neurons. Training all 512 generators to completion would be expensive, so NAT uses a two-stage procedure. First, all 512 generators are trained briefly using 3.12% of the ImageNet training set, with each lightweight run taking about 15 minutes. Second, the generators are ranked by transferability measured either on the source model VGG16 or on a held-out model such as DenseNet121 or ResNet152. Third, the top

$\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$8

generators are fully trained on the complete ImageNet training set (Nakka et al., 23 Aug 2025).

The training hyperparameters reported for NAT are Adam with learning rate $\|\mathbf{x}_a-\mathbf{x}\|_\infty \le \epsilon$9, betas $\epsilon = 10$0, and batch size 16. The paper also notes implementation changes for determinism: ReflectionPad2d layers are removed and padding is folded into subsequent Conv2d layers (Nakka et al., 23 Aug 2025).

4. Generator bank, neuron ranking, and low-query transfer

NAT uses a bank of neuron-specific generators,

$\epsilon = 10$1

though in practice only the selected top subset is fully trained. Each $\epsilon = 10$2 attacks only neuron $\epsilon = 10$3. This design is central to both NAT’s single-query and multi-query behavior (Nakka et al., 23 Aug 2025).

At inference time, NAT does not average generators or fuse perturbations into one image. Instead, it produces multiple alternative adversarial candidates:

$\epsilon = 10$4

where $\epsilon = 10$5 are the ranked neuron indices. The generators are queried in that fixed ranking order. For a $\epsilon = 10$6-query attack, the success rule is existential: the attack is successful if at least one queried adversarial example fools the target model. The paper does not introduce an adaptive query policy, a diversity regularizer, or a learned gating mechanism; the low-query gains come from ranked sequential use of complementary generators rather than from explicit online search (Nakka et al., 23 Aug 2025).

The paper’s analysis treats this complementarity as an empirical property of neuron specialization. Its heatmap over 41 target models shows that no single generator is best on all architectures. It reports, for example, that the generator attacking neuron 391 performs best on 23/41 target models, neuron 321 is best in 3 additional cases, and the remaining best cases are spread across other neuron positions. This supports the paper’s claim that architecture-dependent specialization, rather than a single universal perturbation pattern, explains the strong gains under small query budgets (Nakka et al., 23 Aug 2025).

A common misconception is therefore that NAT’s multi-query performance comes from an ensemble image or a joint multi-neuron objective. The paper does not describe either. The mechanism is a pre-ranked sequence of single-neuron specialists, and the “10-query” result is obtained by trying the top 10 generators in that order (Nakka et al., 23 Aug 2025).

5. Experimental configuration and empirical findings

The evaluation is unusually broad for a generator-based transfer paper. NAT is trained on the ImageNet training set with VGG16 as source and evaluated on a 5K ImageNet validation subset released by LTP. The cross-model benchmark contains 41 ImageNet-pretrained models: 16 convolutional models, 7 transformer models, and 18 hybrid/efficient models. The cross-domain benchmark contains 9 fine-grained models over CUB-200-2011, FGVC Aircraft, and Stanford Cars, with three models per dataset: ResNet-50, SENet154, and SE-Res101 (Nakka et al., 23 Aug 2025).

The reported averages show clear separation from prior generator baselines. Over the 16 CNN targets, the average fooling rates are 59.2% for GAP, 60.9% for CDA, 64.8% for BIA, 71.5% for LTP, and 84.0%, 93.9%, and 96.2% for NAT with 1, 10, and 40 queries respectively. Over the 7 transformer targets, the averages are 16.0%, 17.9%, 18.9%, 15.5%, and 25.0%, 44.9%, 53.5% for the same sequence of methods. Over the 18 hybrid/efficient targets, the corresponding averages are 27.2%, 34.2%, 32.7%, 35.4%, and 53.6%, 74.1%, 79.1% (Nakka et al., 23 Aug 2025).

Setting	Prior baselines	NAT
CNNs, avg fooling rate	GAP 59.2, CDA 60.9, BIA 64.8, LTP 71.5	84.0 (1q), 93.9 (10q), 96.2 (40q)
Transformers, avg fooling rate	GAP 16.0, CDA 17.9, BIA 18.9, LTP 15.5	25.0 (1q), 44.9 (10q), 53.5 (40q)
Hybrid/efficient, avg fooling rate	GAP 27.2, CDA 34.2, BIA 32.7, LTP 35.4	53.6 (1q), 74.1 (10q), 79.1 (40q)
Cross-domain, avg adversarial accuracy	Clean 90.81, GAP 69.78, CDA 70.67, BIA 50.93, LTP 50.06, FACL 44.05	40.94 (1q), 17.97 (10q), 12.38 (40q)

The cross-domain table reports adversarial accuracy, where lower is better. NAT reaches 40.94 with 1 query, 17.97 with 10 queries, and 12.38 with 40 queries, compared with 44.05 for FACL and 50.06 for LTP. The paper summarizes the aggregate result as over 14% improvement in cross-model settings and over 4% in cross-domain settings (Nakka et al., 23 Aug 2025).

The ablations also clarify the neuron-selection stage. For ranking the top 40 generators, DenseNet121, ResNet152, and VGG16 yield similar cross-model performance. The reported cross-model averages for $\epsilon = 10$7 are 96.2 / 53.3 / 79.0 with DenseNet121 ranking, 96.1 / 54.4 / 79.4 with ResNet152 ranking, and 95.6 / 51.4 / 78.3 with VGG16 ranking across convolutional, transformer, and mixed targets. Cross-domain ranking is somewhat more sensitive: the average adversarial accuracy is 13.6 for DenseNet121 ranking, 9.5 for ResNet152 ranking, and 14.6 for VGG16 ranking (Nakka et al., 23 Aug 2025).

6. Relation to adjacent transferability research

NAT sits within a broader line of work that shifts transfer attacks away from pure output-loss maximization toward hidden representations. Earlier methods such as YAILA learn a transferable intermediate discrepancy direction from an attack trajectory rather than selecting neurons directly (Li et al., 2020). NAA attributes model output to neurons in a middle layer and attacks a weighted sum of signed neuron attributions, but it is an attribution-based feature-level iterative attack rather than a generator bank (Zhang et al., 2022). DANAA extends that attributional line by replacing NAA’s linear attribution path with an adversarial non-linear path (Jin et al., 2023). A separate direction uses spectral structure rather than explicit neuron attribution: the SVD-based feature-level attack retains the Top-1 singular-value-associated feature and fuses its logits with the original branch, thereby capturing important internal structure without explicit per-neuron ranking (Weng et al., 2023).

Other papers improve transferability through mechanisms that are adjacent but not neuron-centric. NGI-Attack emphasizes early “Neighbourhood Gradient Information” and masked multi-view gradient fusion; it is primarily a gradient-level and input-transformation-level method rather than a neuron attack (Guo et al., 2024). Natias transfers neuron-attribution ideas to adversarial steganography by corrupting critical middle-layer features that are presumed to be shared across steganalyzers (Fan et al., 2024). In a different domain, NeuroStrike studies transferable “safety neurons” in aligned LLMs and uses both white-box pruning and black-box prompt generation based on source-model neuron profiling, suggesting that transferable sparse neuron subsets are not restricted to image classification (Wu et al., 15 Sep 2025).

Against that background, NAT’s distinctive contribution is not merely “attacking internal features.” Its specific shift is from layer-level generator objectives to neuron-specific generator objectives. In that sense, NAT is best understood as a neuron-granular specialization of generator-based transferable attacks, whereas NAA and DANAA are neuron-attribution-driven iterative attacks, YAILA is a discrepancy-direction method, and the SVD approach is a spectral feature attack (Nakka et al., 23 Aug 2025).

7. Limitations, interpretation, and research significance

The paper explicitly notes several limitations. All experiments use a single source model, VGG16, and only layer 18 is attacked. The final fully trained setup covers only 40 out of 512 neurons, so additional useful neurons may remain unexplored. Training one generator per neuron is expensive even with the lightweight screening stage. The selected generators can also overlap semantically, which may reduce diversity. In addition, NAT’s query-time policy is simple rank-order querying rather than learned gating or adaptive target-driven selection (Nakka et al., 23 Aug 2025).

These limitations constrain interpretation. NAT does not prove that neuron-level specialization is universally optimal across layers, architectures, or budgets. A plausible implication is that its strongest empirical advantage comes from the conjunction of three choices: a transferable middle layer, neuron-wise specialization, and a complementary multi-query bank. The paper provides strong evidence for that conjunction, but it does not isolate all possible alternatives, such as multi-layer neuron selection or multi-source training (Nakka et al., 23 Aug 2025).

Even with those caveats, NAT materially sharpens the representational view of transferability. Earlier generator-based methods already suggested that mid-layer feature disruption transfers better than final-layer loss. NAT refines that claim by arguing that the effective units of transfer are not necessarily whole embeddings but often a sparse set of concept-bearing neurons whose specialized perturbations generalize differently across target families. The reported gains on CNNs, transformers, hybrids, and fine-grained cross-domain models suggest that this neuron-level decomposition captures vulnerabilities that are partly shared across architectures but not reducible to a single universal perturbation pattern (Nakka et al., 23 Aug 2025).