Sampling-Based Fast Gradient Rescaling (S-FGRM)

• The paper introduces S-FGRM, which replaces sign-based gradient updates with a log-magnitude rescaling method to preserve detailed gradient information.
• It employs depth-first sampling to average gradients, effectively reducing numerical instability and capturing more accurate perturbation directions.
• Empirical results on benchmarks like ImageNet show that S-FGRM significantly improves transferability and black-box attack success compared to traditional methods.

The Sampling-based Fast Gradient Rescaling Method (S-FGRM) is an algorithmic framework for the generation of highly transferable adversarial examples with improved stability in gradient-based attacks. By directly addressing quantization and instability limitations inherent in the sign-based update rule—ubiquitous in methods such as FGSM and I-FGSM—S-FGRM introduces a log-magnitude rescaling of the gradient vector, augmented with a depth-first sampling technique, to more faithfully preserve gradient structure and enhance adversarial transferability across neural networks. This approach yields significant improvements in black-box attack efficacy, notably on standard benchmarks such as ImageNet, and remains compatible with diverse attack pipelines and auxiliary input transformations (Han et al., 2023).

1. Shortcomings of Sign-Based Gradient Updates

Conventional adversarial attacks like FGSM and its iterative variants (I-FGSM, MI-FGSM, NI-FGSM) employ the $sign$ function on the gradient of the loss $\nabla_x J(x, y)$ to determine the perturbation direction: $$x_{t+1} = x_t + \alpha \cdot sign(\nabla_x J(x_t, y))$$ This approach enforces an $\ell_\infty$ constraint and guarantees uniformly maximal updates per-feature. However, $sign(\cdot)$ discards all within-vector magnitude information, collapsing every nonzero coordinate to $\pm1$. This induces several drawbacks:

• Loss of directional precision: Gradients differing by orders of magnitude in certain dimensions are quantized identically, e.g., $[0.8, 10^{-8}] \rightarrow [1, 1]$.
• Suboptimal update directions: The restriction to $2^D$ possible update directions, where $D$ is the input dimension, neglects richer gradient information and biases trajectories away from optimal adversarial paths.
• Empirical transferability gap: While sign-based updates may converge rapidly on the source/white-box model, they produce perturbations with poor cross-model transfer in black-box scenarios (Han et al., 2023).

2. Fast Gradient Rescaling Mechanism (FGRM)

FGRM replaces the sign function with a smooth, log-magnitude-based rescaling operation that retains gradient directionality while recapturing relative per-coordinate strength. For a gradient vector $g = \nabla_x J(x, y)$, rescaling is performed in four steps:

1. For each coordinate $i$, compute $a_i = \log_2|g_i|$.
2. Calculate mean $\mu$ and standard deviation $\sigma$ of $\{a_i\}$.
3. Apply a sigmoid normalization: $$\hat{s}_i = \sigma\left(\frac{a_i - \mu}{\sigma}\right) = \frac{1}{1 + e^{-(a_i - \mu)/\sigma}}$$, mapping to $(0, 1)$.
4. Rescale:

$$\mathrm{rescale}(g)_i = c \cdot sign(g_i) \cdot \hat{s}_i$$

where $c$ is a tunable cap (e.g., $c=2$). In vector notation:

$$\mathrm{rescale}(g) = c \cdot [ sign(g) \odot \sigma((\log_2|g|-\mu\mathbf{1})/\sigma) ]$$

Advantages of this transformation include preservation of coordinate ordering and a parametrizable interpolation between full sign quantization ($c=1$, $\sigma\rightarrow 0$) and finer-grained magnitude differentiation. The rescaled update allows for more accurate modeling of input space traversal, thus improving adversarial transferability (Han et al., 2023).

3. Depth-First Sampling for Gradient Stabilization (DFSM)

To mitigate instability in rescaled gradients induced by numerical fluctuations in log, normalization, and sigmoid computations, S-FGRM introduces a depth-first sampling strategy. At each iteration $t$:

• Construct a sequence $x_t^0 = x_t, x_t^{i+1} = x_t^i + \xi_i$ for $i=0,\dots,N-1$, with $$\xi_i \sim \text{Uniform}([-\beta \epsilon, \beta \epsilon]^D)$$.
• Compute the average gradient across the walk:

$$\hat{g}_t = \frac{1}{N+1}\sum_{i=0}^{N} \nabla_x J(x_t^i, y; \theta)$$

This "depth-first" walk, as opposed to single-point or independent sampling, smooths out high-frequency gradient noise and yields a more stable gradient estimate for downstream rescaling. Empirically, optimal performance is usually achieved with $N \approx 12$ and a range $\beta \approx 1.5$ (Han et al., 2023).

4. Algorithmic Structure and Computational Complexity

An example instantiation of S-FGRM atop MI-FGSM (momentum iterative FGSM) comprises the following pipeline:

x_0 = x g_0 = 0 for t in range(T): # 1) Depth-First Sampling x_t^0 = x_t for i in range(N): xi_i = Uniform(-beta*epsilon, beta*epsilon, size=D) x_t^{i+1} = x_t^i + xi_i g_avg = (1/(N+1)) * sum(gradient(x_t^i, y) for i in range(N+1)) # 2) Momentum Update g_{t+1} = mu * g_t + g_avg / l1_norm(g_avg) # 3) Fast Gradient Rescale delta_{t+1} = rescale(g_{t+1}) # 4) Step & Clip x_{t+1} = clip(x_t + alpha * delta_{t+1}, x - epsilon, x + epsilon) return x_T

Standard MI-FGSM costs one evaluation per iteration, while S-FGRM requires $(N+1)$ forward/backward passes per step, incurring a modest computational overhead. Element-wise log, normalization, and sigmoid operations are $O(S)$ in input size $S$. Unlike percentile-based (staircase) rescaling, which needs $O(S \log S)$ sorts, S-FGRM avoids such costs (Han et al., 2023).

5. Extension to Input Transformations and Ensemble Attacks

S-FGRM is compatible with both input transformation pipelines and ensemble-based attack methodologies:

• Input transformations: For each sampled $x_t^i$ in DFSM, apply a randomized transformation $T(\cdot)$ prior to gradient computation (e.g., resize+padding as in DIM).
• Ensembles: When using $M$ surrogate models, the average loss $$\mathcal{J}_\text{ens}(x, y) = \frac{1}{M}\sum_m \mathcal{J}_m(x, y)$$ is computed, and gradients from each model are averaged at every depth-first sampled input. All other steps—FGRM rescaling and projection—are left unaltered. This extensibility allows S-FGRM to serve as a modular component in state-of-the-art black-box attack pipelines (Han et al., 2023).

6. Empirical Evaluation and Performance

Evaluation on ImageNet (with $\epsilon=16/255$, $T=10$, $\mu=1.0$, $\alpha=1.6$) demonstrates substantial gains in adversarial transferability, as outlined below.

Attack Variant	Inc-v3*	Inc-v4	IncRes-v2	Res-101	Inc-v3_ens3	Inc-v3_ens4
MI-FGSM	100.0	44.3	42.4	36.2	13.8	13.0
SMI-FGRM	100.0	82.0	81.1	73.6	44.8	41.3
NI-FGSM	100.0	51.3	49.9	40.6	12.8	12.9
SNI-FGRM	100.0	85.4	83.5	75.9	44.2	42.5

In combination with composite input transforms (DIM, TIM, SIM), S-FGRM enhances non-targeted transfer rates up to 94% on black-box models. Ablations indicate that both FGRM and DFSM contribute to the gains, with FGRM outperforming staircased approaches and DFSM providing further stabilization, especially in the presence of transformations.

Additional findings:

• Optimal gradient rescale cap is $c \approx 2$.
• S-FGRM yields absolute black-box transfer gains of 30–40% over MI-FGSM, with only a modest increase in gradient computation (Han et al., 2023).

7. Context and Implications

S-FGRM systematically addresses a critical bottleneck in transfer-based adversarial attack research—the quantization loss of the $sign$ function and the instability of gradient signals. By preserving magnitude information and introducing robust local averaging, the method advances the reliability and generalization of adversarial perturbations, with relevance for both benchmark evaluation and security analysis of large-scale vision models. Its modular design allows integration with diverse input manipulations and model ensembles, thereby facilitating exploration of new attack and defense spectra in adversarial machine learning (Han et al., 2023).