Gradient-Based Evasion Attacks

• Gradient-based evasion attacks are methods that leverage model gradients to create minimal perturbations causing misclassification, using techniques like FGSM and PGD.
• They optimize input perturbations under norm constraints to maximize loss functions, effectively targeting white-box and enhancing black-box transferability.
• Recent innovations, including adaptive momentum, non-sign methods, and sampling-based rescaling, significantly improve attack transfer rates and bypass modern defenses.

Gradient-based evasion attacks are a foundational paradigm in adversarial machine learning, exploiting the differentiability of modern models to craft minimal, targeted perturbations that induce misclassification or other undesirable behaviors. Their effectiveness is rooted in the ability to harness gradient information—either directly or via approximations—allowing efficient navigation of complex, high-dimensional optimization landscapes. As threat models, datasets, and architectures have evolved, so too have the methodologies and theoretical underpinnings of these attacks, driving continual innovation in both offensive and defensive techniques.

1. Core Principles and Mathematical Framework

At the heart of a gradient-based evasion attack is the optimization of an input perturbation $\delta$ that maximizes a loss $L(f(x+\delta),y)$, subject to a norm-bounded constraint $\|\delta\|_p \leq \epsilon$. For an input $x$ and classifier $f_\theta$, this is formalized as

$$\delta^* = \arg\max_{\|\delta\|_p \leq \epsilon}\, L\bigl(f(x+\delta), y\bigr)$$

where $L$ is typically the cross-entropy loss, and $\epsilon$ sets the perturbation magnitude to preserve perceptual similarity (Mahfuz et al., 2021).

Different attack variants instantiate this optimization according to threat model and computational constraints:

• Fast Gradient Sign Method (FGSM): A single-step attack:

$$x_{\text{adv}} = x + \epsilon \cdot \mathrm{sign}\bigl(\nabla_x L(x, y)\bigr)$$

This operation leverages the sign of the gradient, producing an $\ell_\infty$-bounded step in each input dimension (Han et al., 2023).

• Iterative FGSM (I-FGSM) / Projected Gradient Descent (PGD): Multiple steps of FGSM interleaved with projection back onto the feasible ball:

$$x^{(t+1)} = \mathrm{Proj}_{\|x - x^{(0)}\|_\infty \leq \epsilon}\bigl(x^{(t)} + \alpha \cdot \mathrm{sign}(\nabla_x L(x^{(t)}, y))\bigr)$$

Iterated variants substantially raise attack success, especially in white-box scenarios (Lin et al., 2021).

• Carlini–Wagner Attacks: Direct optimization of a more nuanced loss (e.g., $L_2$-penalized hinge loss on logits), handled with either projected or penalty methods (Lin et al., 2021).

The core strength of gradient-based methods is their leverage of local sensitivity (i.e., the model's gradient), but this also exposes them to limitations when gradients are poorly informative or obfuscated.

2. Algorithmic Innovations and Variants

Significant research has sought to strengthen transferability, bypass local optima, or handle non-standard settings through algorithmic enhancements and theoretical analysis:

a. Replacing the Sign Operation

The $\mathrm{sign}$ function discards magnitude information, resulting in a bias between the true gradient and the applied perturbation. This has motivated a family of non-sign methods:

• Fast Gradient Non-sign Methods (FGNM): Replace $\mathrm{sign}(g)$ with a normalized $\zeta \odot g$, aligning perturbations with the gradient vector and ameliorating magnitude loss (Cheng et al., 2021).
• Sampling-based Fast Gradient Rescaling (S-FGRM): Combine depth-first sampling in the input space with a log-sigmoid rescaling transform:

$$\mathrm{rescale}(g)_i = c \cdot \mathrm{sign}(g_i) \cdot \sigma\left(\frac{\log_2|g_i| - \mu}{\sigma_z}\right)$$

where $c$ is a scale parameter (Han et al., 2023). Sampling several points in a depth chain around $x$ and averaging gradients yields more stable directions and superior transferability.

b. Adaptive and Momentum Methods

Enhancements such as MI-FGSM and adaptive step-size schemes stabilize and accelerate convergence:

• Momentum Iterative FGSM (MI-FGSM): Employ momentum buffers and $\ell_1$ normalization for the accumulated gradient, improving black-box transfer (Tao et al., 2023).
• Adaptive FGM (AdaI-FGM, AdaMI-FGM): Use per-coordinate learning rates, accumulating squared gradients in EMA/AdaGrad style, leading to improved stability and stronger black-box results (Tao et al., 2023).

c. Principled Constraint Handling

• Constrained Gradient Descent (CGD): Internalizes norm constraints in the loss explicitly, incorporating an overrun penalty and employing Adam for update steps:

$$L_{\text{CGD}}(x') = w\,L_{\text{MD}}(x',t) + (1-w)\,L_{bnd}(x')$$

where $L_{bnd}$ penalizes excursions outside the $\ell_\infty$ ball, avoiding ad-hoc clipping (Lin et al., 2021).

3. Extensions Beyond Standard Domains

While most work has focused on image-classification or tabular data, recent advances target other domains and settings:

• Constrained, Structured Data: FENCE adapts gradient-based methods to tabular data with domain and functional constraints by fusing step-wise updates with exact repair subroutines to enforce feasibility (e.g., ratio, one-hot, or statistical constraints) at each iteration (Chernikova et al., 2019).
• Evasion in Bayesian Predictive Models: Attacks on Bayesian models (incl. BNNs) optimize for both point predictions and full posterior predictives, with MC-based gradient estimation and KL-divergence objectives facilitating both mean-shifting and distributional steering (Arce et al., 11 Jun 2025).
• AI-generated Text Detector Evasion: GradEscape injects continuous, differentiable "weighted embeddings" into NLP pipelines, exploiting the victim detector's embedding layer to enable gradient-based optimization over token probability distributions (Meng et al., 9 Jun 2025).
• Malware (PE executable) Evasion: Intra-section code cave injection leverages FGSM and iterative gradient-descent within designated file regions—while preserving malware functionality via a runtime code-loader—to evade CNN-based detectors like MalConv and MalConv2 (Aryal et al., 2024).

4. Transferability and Black-Box Performance

A central challenge for gradient-based attackers is the "transferability" of adversarial examples: the likelihood that a perturbation crafted for one model induces failure in another. Empirical results consistently show:

• S-FGRM, by coupling advanced rescaling and depth-first sampling, elevates average black-box transfer rates from 44% to 82% (Inception-v3 $\rightarrow$ Inc-v4, 10-step, $\epsilon=16/255$) and up to 94% in ensemble attacks (Han et al., 2023).
• FGNM yields up to 27.5% transfer rate gains over sign-based methods in black-box image attacks (Cheng et al., 2021).
• Adaptive (per-step) step-size and momentum-based enhancement similarly increase black-box and cross-architecture effectiveness (Tao et al., 2023).

Integration with input transformation schemes (e.g., DI-FGSM, TIM, SIM) and ensemble loss averaging further boosts success in transfer and defense-hardened settings (Han et al., 2023).

5. Limitations, Defenses, and Countermeasures

Defenders continuously adapt to gradient-based threats through the following strategies:

• Gradient Masking and Obfuscation: Techniques like input transformations, non-differentiable or randomized preprocessing (e.g., JPEG, RDG, FD+Rand), or architectural tricks (convolutional front ends with skip connections) can mislead or annihilate gradients, deceiving naive white-box attacks and requiring fully adaptive (e.g., BPDA, EOT) or zeroth-order attacks to uncover true vulnerabilities (Qiu et al., 2020, Boytsov et al., 2024).
• Randomized Ensembles: Defenses that randomize preprocessing or employ stochastic ensemble selection can prevent reliable gradient estimation, impeding both standard and black-box attacks unless the attacker adapts with expectation over transformation or full model knowledge (Boytsov et al., 2024).
• Adversarial Training with Constraint-aware Examples: For structured data domains, training on feasible adversarial samples produced by constrained gradient-based attacks demonstrably increases robustness, although at some false-positive cost (Chernikova et al., 2019).
• Affine and Transformation-Invariant Gradients: Attacks that anticipate geometric transforms—via affine-invariant gradient estimators—remain effective even under affine pre/postprocessing, forcing defenders to consider more comprehensive robustness guarantees (Xiang et al., 2021).

Despite strong empirical performance, limitations of current approaches include computational overhead (e.g., multiple gradient evaluations in S-FGRM), dependence on perfect knowledge (white-box setting), and limited generalization to highly randomized or non-differentiable defenses.

6. Empirical Benchmarks and Comparative Outcomes

Empirical studies across domains consistently validate the potency of recent methods:

Method	White-box Success	Black-box Transfer	Defense Robustness	Computational Notes
FGSM/I-FGSM	$\approx$100%	16–44% (images)	Poor under input defenses	Negligible cost
S-FGRM	$\approx$100%	74–94%	Outperforms input-transf.	%%%%21$\|\delta\|_p \leq \epsilon$22%%%% MI-FGSM
FGNM	$\approx$100%	+9–27% over SOTA	Consistently higher vs. SIM	$<1\%$ cost overhead
CGD	1.2–5.1 pp higher	8.6–13.6 pp higher	Stronger vs Auto-PGD/CW	11–19% runtime gain
DiffAttack	--	--	−20 pp vs. best defense	O(1) memory, tailored
FENCE	99–100% (tabular)	40–100% (tabular)	Needs constraint-aware adv	Fast, modular repair

Experimentally, enhancements that faithfully preserve gradient directionality, adaptively scale update steps, or inject diverse input sampling consistently yield higher transferability and break new or previously robust defenses in both image and structured-data domains (Lin et al., 2021, Han et al., 2023, Chernikova et al., 2019).

7. Open Problems and Future Directions

Research continues to explore several unresolved areas:

• Theoretical guarantees for transferability, convergence, and worst-case robustness across architectures and defenses remain open (Han et al., 2023).
• Generalization beyond $\ell_\infty$ balls and to richer or perceptually-grounded metrics.
• Efficient, gradient-based attacks for non-differentiable or discrete domains, e.g., with proxy, surrogate, or embedding-based relaxations (Meng et al., 9 Jun 2025).
• Adapting attacks to preempt or defeat randomized, highly-nonlinear, or cross-modal defense pipelines.
• Integrating adversarial optimization and defense into end-to-end differentiable training for robust models under operational or regulatory constraints.

As algorithmic and theoretical sophistication increases, so does the complexity and arms race of gradient-based evasion attacks and their countermeasures, reinforcing their centrality in the study of machine learning security (Han et al., 2023, Xiang et al., 2021, Qiu et al., 2020).