---
title: Sampling-Based Fast Gradient Rescaling (S-FGRM)
url: https://www.emergentmind.com/topics/sampling-based-fast-gradient-rescaling-s-fgrm
type: topic
---

# Sampling-Based Fast Gradient Rescaling (S-FGRM)

The Sampling-based Fast Gradient Rescaling Method (S-FGRM) is an algorithmic framework for the generation of highly transferable adversarial examples with improved stability in gradient-based attacks. By directly addressing quantization and instability limitations inherent in the sign-based update rule—ubiquitous in methods such as FGSM and I-FGSM—S-FGRM introduces a log-magnitude rescaling of the gradient vector, augmented with a depth-first sampling technique, to more faithfully preserve gradient structure and enhance adversarial transferability across neural networks. This approach yields significant improvements in black-box attack efficacy, notably on standard benchmarks such as ImageNet, and remains compatible with diverse attack pipelines and auxiliary input transformations [2307.02828].

## 1. Shortcomings of Sign-Based Gradient Updates

Conventional adversarial attacks like FGSM and its iterative variants (I-FGSM, MI-FGSM, NI-FGSM) employ the $sign$ function on the gradient of the loss $\nabla_x J(x, y)$ to determine the perturbation direction:
\[
x_{t+1} = x_t + \alpha \cdot sign(\nabla_x J(x_t, y))
\]
This approach enforces an $\ell_\infty$ constraint and guarantees uniformly maximal updates per-feature. However, $sign(\cdot)$ discards all within-vector magnitude information, collapsing every nonzero coordinate to $\pm1$. This induces several drawbacks:
- **Loss of directional precision:** Gradients differing by orders of magnitude in certain dimensions are quantized identically, e.g., $[0.8, 10^{-8}] \rightarrow [1, 1]$.
- **Suboptimal update directions:** The restriction to $2^D$ possible update directions, where $D$ is the input dimension, neglects richer gradient information and biases trajectories away from optimal adversarial paths.
- **Empirical transferability gap:** While sign-based updates may converge rapidly on the source/white-box model, they produce perturbations with poor cross-model transfer in black-box scenarios [2307.02828].

## 2. Fast Gradient Rescaling Mechanism (FGRM)

FGRM replaces the sign function with a smooth, log-magnitude-based rescaling operation that retains gradient directionality while recapturing relative per-coordinate strength. For a gradient vector $g = \nabla_x J(x, y)$, rescaling is performed in four steps:
1. For each coordinate $i$, compute $a_i = \log_2|g_i|$.
2. Calculate mean $\mu$ and standard deviation $\sigma$ of $\{a_i\}$.
3. Apply a sigmoid normalization: $\hat{s}_i = \sigma\left(\frac{a_i - \mu}{\sigma}\right) = \frac{1}{1 + e^{-(a_i - \mu)/\sigma}}$, mapping to $(0, 1)$.
4. Rescale:
   \[
   \mathrm{rescale}(g)_i = c \cdot sign(g_i) \cdot \hat{s}_i
   \]
   where $c$ is a tunable cap (e.g., $c=2$). In vector notation:
   \[
   \mathrm{rescale}(g) = c \cdot [ sign(g) \odot \sigma((\log_2|g|-\mu\mathbf{1})/\sigma) ]
   \]
Advantages of this transformation include preservation of coordinate ordering and a parametrizable interpolation between full sign quantization ($c=1$, $\sigma\rightarrow 0$) and finer-grained magnitude differentiation. The rescaled update allows for more accurate modeling of input space traversal, thus improving adversarial transferability [2307.02828].

## 3. Depth-First Sampling for Gradient Stabilization (DFSM)

To mitigate instability in rescaled gradients induced by numerical fluctuations in log, normalization, and sigmoid computations, S-FGRM introduces a depth-first sampling strategy. At each iteration $t$:
- Construct a sequence $x_t^0 = x_t, x_t^{i+1} = x_t^i + \xi_i$ for $i=0,\dots,N-1$, with $\xi_i \sim \text{Uniform}([-\beta \epsilon, \beta \epsilon]^D)$.
- Compute the average gradient across the walk:
  \[
  \hat{g}_t = \frac{1}{N+1}\sum_{i=0}^{N} \nabla_x J(x_t^i, y; \theta)
  \]
This "depth-first" walk, as opposed to single-point or independent sampling, smooths out high-frequency gradient noise and yields a more stable gradient estimate for downstream rescaling. Empirically, optimal performance is usually achieved with $N \approx 12$ and a range $\beta \approx 1.5$ [2307.02828].

## 4. Algorithmic Structure and Computational Complexity

An example instantiation of S-FGRM atop MI-FGSM (momentum iterative FGSM) comprises the following pipeline:
```python
x_0 = x
g_0 = 0
for t in range(T):
    # 1) Depth-First Sampling
    x_t^0 = x_t
    for i in range(N):
        xi_i = Uniform(-beta*epsilon, beta*epsilon, size=D)
        x_t^{i+1} = x_t^i + xi_i
    g_avg = (1/(N+1)) * sum(gradient(x_t^i, y) for i in range(N+1))

    # 2) Momentum Update
    g_{t+1} = mu * g_t + g_avg / l1_norm(g_avg)

    # 3) Fast Gradient Rescale
    delta_{t+1} = rescale(g_{t+1})

    # 4) Step & Clip
    x_{t+1} = clip(x_t + alpha * delta_{t+1}, x - epsilon, x + epsilon)

return x_T
```
Standard MI-FGSM costs one evaluation per iteration, while S-FGRM requires $(N+1)$ forward/backward passes per step, incurring a modest computational overhead. Element-wise log, normalization, and sigmoid operations are $O(S)$ in input size $S$. Unlike percentile-based (staircase) rescaling, which needs $O(S \log S)$ sorts, S-FGRM avoids such costs [2307.02828].

## 5. Extension to Input Transformations and Ensemble Attacks

S-FGRM is compatible with both input transformation pipelines and ensemble-based attack methodologies:
- **Input transformations:** For each sampled $x_t^i$ in DFSM, apply a randomized transformation $T(\cdot)$ prior to gradient computation (e.g., resize+padding as in DIM).
- **Ensembles:** When using $M$ surrogate models, the average loss $\mathcal{J}_\text{ens}(x, y) = \frac{1}{M}\sum_m \mathcal{J}_m(x, y)$ is computed, and gradients from each model are averaged at every depth-first sampled input.
All other steps—FGRM rescaling and projection—are left unaltered. This extensibility allows S-FGRM to serve as a modular component in state-of-the-art black-box attack pipelines [2307.02828].

## 6. Empirical Evaluation and Performance

Evaluation on ImageNet (with $\epsilon=16/255$, $T=10$, $\mu=1.0$, $\alpha=1.6$) demonstrates substantial gains in adversarial transferability, as outlined below.

| Attack Variant            | Inc-v3* | Inc-v4 | IncRes-v2 | Res-101 | Inc-v3_ens3 | Inc-v3_ens4 |
|--------------------------|---------|--------|------------|---------|-------------|-------------|
| MI-FGSM                  | 100.0   | 44.3   | 42.4       | 36.2    | 13.8        | 13.0        |
| SMI-FGRM                 | 100.0   | 82.0   | 81.1       | 73.6    | 44.8        | 41.3        |
| NI-FGSM                  | 100.0   | 51.3   | 49.9       | 40.6    | 12.8        | 12.9        |
| SNI-FGRM                 | 100.0   | 85.4   | 83.5       | 75.9    | 44.2        | 42.5        |

In combination with composite input transforms (DIM, TIM, SIM), S-FGRM enhances non-targeted transfer rates up to 94% on black-box models. Ablations indicate that both FGRM and DFSM contribute to the gains, with FGRM outperforming staircased approaches and DFSM providing further stabilization, especially in the presence of transformations.

Additional findings:
- Optimal gradient rescale cap is $c \approx 2$.
- S-FGRM yields absolute black-box transfer gains of 30–40% over MI-FGSM, with only a modest increase in gradient computation [2307.02828].

## 7. Context and Implications

S-FGRM systematically addresses a critical bottleneck in transfer-based adversarial attack research—the quantization loss of the $sign$ function and the instability of gradient signals. By preserving magnitude information and introducing robust local averaging, the method advances the reliability and generalization of adversarial perturbations, with relevance for both benchmark evaluation and security analysis of large-scale vision models. Its modular design allows integration with diverse input manipulations and model ensembles, thereby facilitating exploration of new attack and defense spectra in adversarial machine learning [2307.02828].

Source: https://www.emergentmind.com/topics/sampling-based-fast-gradient-rescaling-s-fgrm