Adversarial Attacks in Machine Learning

Updated 12 January 2026

Adversarial attacks are perturbation-based techniques that subtly alter inputs to cause deep neural networks to produce incorrect predictions.
They span various modalities such as vision, speech, and NLP, highlighting significant security challenges in modern AI systems.
Recent studies demonstrate high success and transferability rates of these attacks even against models employing advanced defense mechanisms.

Adversarial attacks are perturbation-based methodologies designed to manipulate machine learning models, particularly deep neural networks (DNNs), such that they produce incorrect or attacker-specified predictions. These perturbations are typically constrained to ensure imperceptibility under an appropriate metric (e.g., $L_p$ norm for signals or semantic similarity for text). While originally studied in image classification, adversarial attacks are now established across vision, speech, reinforcement learning, NLP, and power systems. They have become central to security and reliability assessments of machine learning systems, revealing intrinsic vulnerabilities, transfer behavior, and limitations of current defenses.

1. Formal Definition and Taxonomy

An adversarial example is an input $x' = x + \delta$ such that the perturbation $\delta$ is imperceptible ( $\|\delta\|_p \leq \epsilon$ ), but the induced prediction $f(x')$ differs from the correct label or is pushed toward a designated target. Formally, for a classifier $f: \mathbb{R}^d\to \{1,\dots,K\}$ and input $x$ with label $y$ , the canonical objectives are:

Non-targeted attack:

$\min_{\delta} \|\delta\|_p \;\mathrm{s.t.}\; f(x+\delta)\neq y$

Targeted attack:

$\min_{\delta} \|\delta\|_p \;\mathrm{s.t.}\; f(x+\delta) = t \neq y$

Attacks can be classified along three primary axes (Lin et al., 2021, Zhang et al., 2023):

Knowledge: white-box (full access to model parameters/gradients), black-box (query access only, either to probabilities—score-based—or just class labels—decision-based).
Objective: targeted (force $f(x+\delta)=t$ ), untargeted (force $f(x+\delta)\neq y$ ).
Setting: evasion (inference-time perturbations), poisoning (injecting malicious data during training).

Common norm constraints include $L_0$ (sparse pixel changes), $L_2$ (Euclidean), and $L_\infty$ (maximum component change), with typical $\epsilon$ calibrated for dataset granularity (e.g., $8/255$ for images) (Zhang et al., 2023).

2. Attack Algorithms: Core Methodologies

The literature has established several attack paradigms:

FGSM (Fast Gradient Sign Method): A single-step, linearized attack under $L_\infty$ norm, $\delta = \epsilon\, \mathrm{sign}(\nabla_x L(x, y))$ .
PGD (Projected Gradient Descent): Iterative $L_\infty$ -constrained attack, repeatedly stepping along the gradient and projecting back into the feasible ball (Lin et al., 2021, Zhang et al., 2023).
C&W Attack: Unconstrained optimization of $L_2$ or $L_\infty$ distance subject to classification constraints, often via the logit margin (Lin et al., 2021).
DeepFool: Iterative approach for minimal $L_2$ -norm perturbations via local linearization (Tian et al., 2022).
Sparse and Imperceivable Attacks: Black-box, score-based algorithms that optimize $L_0$ count, e.g., CornerSearch, and extensions with per-pixel adaptive bounds (Croce et al., 2019).
Entropy-Based Methods: Allocate perturbation budget to high-entropy regions to evade human perception while maintaining attack success (Göpfert et al., 2019).
GAN-Based Attacks: Use adversarial generative models ( $G$ ) to synthesize imperceptible yet highly effective perturbations optimized against both a discriminator ( $D$ ) and the target classifier ( $C$ ) (Yang, 2024).
Algebraic Attacks on Explanations: Exploit network symmetry groups to guarantee identical predictions but divergent explanations, avoiding traditional constrained optimization (Simpson et al., 16 Mar 2025).
Domain-Specific and Multimodal Attacks: Attacks on 3D rendering parameters, ISP/optics pipelines, reinforcement-learning state observations, and speech recognition (audio-visual correlation targeting) (Zeng et al., 2017, Phan et al., 2021, Pattanaik et al., 2017, Ma et al., 2019).

In NLP, attacks are constructed by transforming token sequences through synonym swaps, paraphrasing, or character-level mutations, subject to semantic and syntactic constraints, as in TextAttack's modular pipeline (Morris et al., 2020).

3. Empirical Findings and Transferability

Extensive evaluation reveals state-of-the-art DNNs are highly brittle:

FGSM on MNIST ( $\epsilon=0.3$ ) raises error from ~1% to >20%; PGD on CIFAR-10 ( $\epsilon_\infty=8/255$ ) yields nearly 100% attack success; C&W on ImageNet achieves >99% attack success with imperceptible distortions (Lin et al., 2021, Zhang et al., 2023).
Iterative or ensemble-based black-box attacks achieve high transfer rates: perturbations crafted for one model frequently mislead diverse architectures (e.g., FGSM/PGD adversarial inputs have 84–96% transfer success on black-box APIs) (Zhang et al., 2023).
Sparse attacks (e.g., CornerSearch with adaptive per-pixel bounds) can achieve high misclassification rates (e.g., median 2–7 pixels perturbed yields >95% non-targeted attack success rate) while avoiding detectability (Croce et al., 2019).

Physical attacks, such as perturbing 3D properties, camera pipelines, or embedding patterns in scenes or speech, demonstrate practical feasibility in safety-critical domains. Success rates remain high (e.g., >90% for targeted ISP/optics attacks) when the attack is tailored to actual acquisition conditions (Phan et al., 2021, Zeng et al., 2017).

Recent work on transform-dependent attacks introduces "metamorphic" perturbations that reveal vulnerabilities not just to input changes but to compositional transformation pipelines (e.g., scaling, blur, gamma). A single perturbation can control the adversarial outcome as a function of the applied transformation parameter, achieving up to 99% attack success depending on target and architecture (Tan et al., 2024).

4. Defense Mechanisms and Robustness

Defenses to adversarial attacks remain a topic of active research and challenge:

Adversarial Training: Incorporates adversarial examples (often via PGD) in the training loop, widely regarded as the most effective empirical defense, but only yields robustness near the sampled perturbation regime and often at substantial computational cost or reduction in clean accuracy (Song et al., 2017, Zhang et al., 2023).
- MAT (Multi-strength Adversarial Training) extends training to multiple perturbation strengths, improving coverage (Song et al., 2017).
Input Transformations and Denoising: JPEG compression, random resizing/padding, and learned denoisers (HGD) can partially disrupt attacks but are typically circumventable by adaptive strategies (Kurakin et al., 2018).
Certified Defenses: Interval Bound Propagation and randomized smoothing provide provable but scaling-limited robustness certificates (Lin et al., 2021).
Purification Mechanisms: In NLP, ensemble-based masked LLM purification mitigates word substitution attacks without access to the attacker's candidate set, raising after-attack accuracy by >60 percent points against strong word-substitution attacks (Li et al., 2022).
Hedge Defense: Applies a second, general $L_p$ -bounded perturbation to adversarially trained models, exploiting differences in Lipschitz continuity across class scores to reverse many adversarial errors and boost accuracy under attack by up to 7% on CIFAR-10/ImageNet (Wu et al., 2021).

A perennial theme is the trade-off between robustness and generalization, the curse of high-dimensional geometry (the existence of thin adversarial subspaces), and the intrinsic limitation of gradient-masking or obfuscated approaches.

5. Extensions and Emerging Directions

Recent advancements expand the adversarial landscape:

Multimodal and Cross-Domain Attacks: Adversarial examples targeting correlations between modalities (e.g., audio-visual sync in speech recognition) or features crossing domains (e.g., geometric, photometric, and ISP transformations) (Phan et al., 2021, Ma et al., 2019).
Universal and Signal-Agnostic Attacks: Construction of single perturbations that generalize over a large portion of the data manifold (e.g., universal perturbations with 74% fooling rates in power systems) (Tian et al., 2022).
Explainability and Interpretability Attacks: Craft attacks to diverge explanation while preserving model prediction using symmetry and group-theoretic insights (Simpson et al., 16 Mar 2025).
Adaptive, Perception-Evading Attacks: Use entropy or structural priors to minimize human detection while maximizing machine misclassification, validated by user studies (Göpfert et al., 2019).
GAN-Driven Adversarial Example Synthesis: Exploit adversarial generative models to discover realistic, imperceptible patterns evading both classifier and discriminator, consistently outperforming FGSM/BIM methods (Yang, 2024).
Textual Adversarial Attacks: Modularized approaches (TextAttack) enable transformation-constraint-search pipelines, supporting grammar/semantic-aware attacks at scale (Morris et al., 2020).

6. Representative Benchmarks and Quantitative Results

Empirical results highlight the consistently high effectiveness and transferability of adversarial attacks against SOTA models:

Dataset	Model/Method	Attack	Attack Success/Accuracy	Reference
MNIST	DNN, PGD-trained	FGSM ( $\epsilon=0.3$ )	Error >20% (from 1%)	(Lin et al., 2021)
CIFAR-10	ResNet, Adv. Training	PGD ( $\epsilon=8/255$ )	Acctest $62.7\%$ (Adv-pNML $67.2\%$ )	(Pesso et al., 2021)
ImageNet	ResNet-50, Fast AT	PGD ( $\epsilon=8/255$ )	Accbase $16.0\%$ , Adv-pNML $20.0\%$	(Pesso et al., 2021)
Power Quality	ConvNet	SAA (universal)	Misclassification up to $74\%$	(Tian et al., 2022)
Speech (LRW)	AV-SR DNN	FGSM ( $\epsilon^A{1024},\epsilon^V{16}$ )	Top1acc $10.4\%$ , detection AUC $0.99$	(Ma et al., 2019)
Text (IMDB)	BERT	TextFooler	No defense: $2.8\%$ acc; Purification: $51.0\%$	(Li et al., 2022)

Defended models, even those using adversarial training or multi-strength methods, remain vulnerable to advanced adaptive, physical, or domain-specific attacks.

7. Open Problems and Future Challenges

Despite extensive defenses, no single approach is universally effective, and research illustrates defenders must anticipate attacks that exploit transformation, perception, and group-theoretic vulnerabilities (Tan et al., 2024, Simpson et al., 16 Mar 2025). Robustness in deployed systems will require closing gaps in certified guarantees, generalizing to unseen transformations and modalities, and integrating multi-layered defense frameworks.

Transform-dependent and algebraic attacks underscore that robust models must secure not just fixed $(x, \delta)$ neighborhoods, but also compositional transformations and symmetry-induced invariants, requiring fundamentally new techniques in model design, training, and certification.