Texture-scale Constrained UAPs in CNNs

• TSC-UAPs are image-agnostic adversarial perturbations that tile small texture patches to exploit local texture cues in CNN classifiers.
• The method constrains perturbation scale through patch optimization and tiling, significantly boosting fooling ratios and cross-model transferability.
• Empirical evaluations show TSC-UAPs achieve up to 99% fooling in black-box settings and exhibit extreme data efficiency even with limited training images.

Texture-scale Constrained Universal Adversarial Perturbations (TSC-UAP) are a class of image-agnostic adversarial attacks designed to improve the effectiveness and transferability of universal perturbations against convolutional neural network (CNN) classifiers. By imposing a constraint on the spatial scale of adversarial textures and employing a tiling operation, TSC-UAP yields class-specific, locally-replicated patterns that exploit the texture-biased nature of CNN decision functions, resulting in higher fooling ratios and enhanced attack generalization across models and datasets (Huang et al., 2024).

1. Universal Adversarial Perturbation Framework

In the standard Universal Adversarial Perturbation (UAP) setting, the goal is to construct a fixed perturbation $\delta \in \mathbb{R}^{H \times W \times 3}$ such that, for a target classifier $$f: \mathbb{R}^{H \times W \times 3} \rightarrow \{1,\dots,C\}$$ and most clean images $x \sim \mathcal{X}$, the classifier prediction changes under the perturbed input—i.e., $f(x + \delta) \neq f(x)$—subject to a norm constraint $\|\delta\|_p \le \varepsilon$. The attack effectiveness is quantified by the fooling ratio,

$$\mathrm{FR}(\delta) = \frac{1}{|\mathcal{X}_\mathrm{test}|} \bigl| \{ x \in \mathcal{X}_\mathrm{test} : f(x+\delta) \neq f(x) \} \bigr|.$$

The UAP generation employs either data-dependent or data-free objectives, with the optimization often solved using surrogate losses (e.g., cross-entropy), projecting onto the norm ball at each iteration. Prior methods generally optimize $\delta$ directly for the full image dimensions, producing globally consistent but spatially inflexible textures (Huang et al., 2024).

2. Texture Scale Constraint and Tiling Operation

TSC-UAP introduces a texture scale constraint motivated by the observation that CNNs predominantly utilize local textural information for prediction. Rather than searching in the full $H \times W \times 3$ space, TSC-UAP defines a patch $$v \in \mathbb{R}^{H/\alpha \times W/\alpha \times 3}$$, where the split ratio $\alpha$ divides $H, W$ evenly. The universal perturbation is then constructed via a tiling operation: $\delta = \mathcal{T}(v, \alpha),$ which replicates $v$ in both spatial dimensions to fill the target image. The optimization is performed with respect to $v$,

$\|v\|_p \le \varepsilon,$

such that no additional regularization is required: enforcing small-scale textures directly limits the solution space of possible perturbations (Huang et al., 2024).

3. Optimization Approach

The TSC-UAP optimization embeds tiling into the adversarial objective. For a data-dependent loss $\ell$, the objective is: $$\max_{v: \|v\|_p \le \varepsilon} \frac{1}{N} \sum_{i=1}^N \ell\bigl(f(x_i + \mathcal{T}(v, \alpha)), y_i\bigr).$$ A typical projected-gradient method is applied:

1. Initialize $v \leftarrow 0$.
2. For each training epoch:
   • Draw minibatch $B = \{x_i\}_{i=1}^m$.
   • Set $\delta = \mathcal{T}(v,\alpha)$.
   • Compute loss $$\mathcal{L} = \frac{1}{m} \sum_{x_i \in B} \ell(f(x_i+\delta), y_i)$$.
   • Gradient step $v \leftarrow v + \eta g_v$, where $g_v = \nabla_v\mathcal{L}$.
   • Project $v$ onto $[-\varepsilon, \varepsilon]$.

After $E$ epochs, the final universal perturbation is $\delta^\star = \mathcal{T}(v, \alpha)$. This procedure is compatible with standard deep learning frameworks (e.g., PyTorch) and imposes negligible additional computational cost relative to standard UAPs (Huang et al., 2024).

4. Class-Specific and Targeted Perturbations

Analysis reveals that even untargeted TSC-UAPs generally bias predictions toward a particular dominant class, suggesting intrinsic category specificity. Targeted TSC-UAPs may be generated by optimizing a class-targeted loss,

$$\min_{v:\|v\|_p \leq \varepsilon} \frac{1}{N} \sum_{i=1}^N \ell(f(x_i + \mathcal{T}(v, \alpha)), t),$$

for desired target class $t$. Training a distinct patch $v_t$ for each class produces class-specific textures that mislead the classifier reliably across input samples (Huang et al., 2024).

5. Empirical Evaluation

Experiments on ImageNet-1k, CIFAR-10, CIFAR-100, and Places365, across models including AlexNet, GoogleNet, VGG16/19, ResNet50/152, DenseNet121, MobileNet-v2, and ViT-B/16, demonstrate the efficacy of TSC-UAP. The fooling ratios for various split ratios $\alpha$ on ImageNet with data-dependent SGD-UAP (ResNet50, VGG19, DenseNet121, MobileNet-v2) are summarized as:

$\alpha$	ResNet50 (FR)	VGG19 (FR)	DenseNet121 (FR)	MobileNet-v2 (FR)
1	80.30%	82.44%	66.05%	94.08%
2	87.22%	90.37%	78.83%	97.01%
4	89.92%	92.88%	82.26%	97.25%
8	92.73%	92.57%	85.18%	99.19%
16	78.47%	83.54%	79.26%	97.53%
32	53.96%	79.39%	51.24%	74.65%

Maximum gains of up to $\sim\!12\%$ absolute in white-box fooling are observed for $2 \leq \alpha \leq 8$. Black-box transfer rates (ResNet50→VGG19) improve from 53.0% (baseline $\alpha=1$) to 80.1% ($\alpha=8$), with a mean increase of 21.4% absolute across evaluated source-target pairs. Notably, cross-dataset transfer (ImageNet-trained to CIFAR/Places365) improves by up to 25% fooling ratio at $\alpha=8$. Extreme data-efficiency is exhibited; with only 10 training images, TSC-UAP ($\alpha=8$) reaches $\sim$72% FR on ResNet50 compared to 22% for the baseline (Huang et al., 2024).

Evaluations against defense mechanisms, such as the PRN (Akhtar et al., 2018), confirm that post-defense fooling ratios remain higher for TSC-UAP, fundamentally due to superior initial attack strength.

6. Practical Implementation and Considerations

Gradient computation and memory usage for TSC-UAP scale with the patch size rather than full image size, delivering efficiency benefits at moderate to high $\alpha$. The optimal split ratio is empirically found in the $[2, 8]$ range; too small ($\alpha=1$) yields overly global perturbations, while too large ($\alpha \geq 16$) renders textures ineffective. Both $\ell_\infty$ and $\ell_2$ norm budgets are supported, with optimal $\alpha$ potentially shifting in the $\ell_2$ case (e.g., $\alpha=16$).

TSC-UAP generalizes naturally to targeted attack scenarios. Implementation follows a straightforward patch optimization loop utilizing standard tensor operations. Limitations currently include the absence of a principled theoretical analysis for the efficacy of tiling with respect to CNN receptive fields. The potential to further enhance universality via additional spatial transformations (flips, rotations, crops) and by tuning $\alpha$ per architecture remains open for investigation (Huang et al., 2024).

7. Implications and Research Outlook

The TSC-UAP paradigm establishes that constraining adversarial perturbations to small, repeated, class-specific texture patches dramatically improves both white-box and black-box attack success in CNNs. This finding underscores the principal role of local texture in CNN-based image recognition and motivates further mechanistic study of architectural biases. A plausible implication is that investigation of alternate repetition or augmentation mechanisms—potentially dataset- or architecture-adaptive—could yield higher universality. Future directions include rigorous theoretical analysis of tiling effects, exploration of generalized transformation sets, and the study of adaptive patch sizing for various models and input dimensions (Huang et al., 2024).