---
title: Attack Success Rates in Adversarial ML
url: https://www.emergentmind.com/topics/attack-success-rates-asrs
type: topic
---

# Attack Success Rates in Adversarial ML

Attack Success Rates (ASRs) are a foundational metric in adversarial machine learning, quantifying the proportion of adversarial examples that successfully induce a targeted error in the victim model. Across domains—including computer vision, natural language processing, and speech recognition—ASR serves as both a primary measure of attack efficacy and a diagnostic insight into model robustness, transferability characteristics, and the effectiveness of newly proposed attack methodologies. Its definition, calculation procedures, role in benchmarking, and use in defense evaluation have become increasingly sophisticated as the adversarial threat landscape evolves.

## 1. Foundational Definition and Calculation

ASR is generally defined as the percentage of adversarial inputs that, when presented to a trained model, produce the intended misclassification (or, under a targeted attack, a specific predicted label). Formally, for a set of adversarial examples $\{\tilde{x}_i\}_{i=1}^N$ and a ground truth label set $\{y^{\text{true}}_i\}_{i=1}^N$, the ASR is computed as
\[
\text{ASR} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{1}\{\text{attack succeeds on }\tilde{x}_i\}
\]
where $\mathbb{1}\{\cdot\}$ is the indicator function, returning 1 if the attack succeeds (e.g., $\arg\max_y f(\tilde{x}_i)_y \neq y^{\text{true}}_i$) and 0 otherwise. This simple rate is widely applicable across tasks ranging from image classification [2003.12760], text model evasion [2004.09984], to physical attacks on perception systems [2009.06701].

The precise operationalization of “success” depends on context: for untargeted attacks it denotes any misclassification; for targeted attacks, the output must match a specific adversarial label. In speech recognition, success may require an exact transcript match or a substantive word error [2112.01821]. In red teaming large LLMs, ASR is the sample mean over binary judgments of whether an attack was successful under each trial configuration [2507.22133].

## 2. ASR as an Attack Benchmark and Comparative Metric

ASR has become the canonical metric for comparing adversarial attacks and defenses. New attack strategies are often validated explicitly by demonstrating higher ASRs relative to state-of-the-art baselines, with fixed perturbation budgets ($L_\infty$ or perceptual equivalence constraints), and sometimes under strict black-box or physical-world conditions.

In image domain black-box settings, adversarial imitation attacks [2003.12760] improved ASRs over substitute and query-based alternatives both by optimizing transferability and minimizing sample/query requirements. In targeted language model attacks [2004.09984], ASR is complemented with semantic and fluency assessments, but remains the lead indicator of generation efficacy. In physical attacks, such as dirty road patch attacks on automotive systems, reported ASRs exceeding 97.5% (often reaching 100%) directly quantify control over system behavior, capturing both reliability and real-world risk [2009.06701].

ASR is typically reported alongside related quantities such as average perturbation size ($\|\epsilon\|$), number of queries, or attack time, to capture trade-offs among success, stealthiness, and efficiency.

| Domain                     | Typical ASR Range | Context                                            |
|----------------------------|-------------------|----------------------------------------------------|
| Image classification       | 70–99%            | Depends on attack/defense, e.g. mimic-white-box    |
| Text classification        | 80–98%            | High ASR with low perturbation rates                |
| ASR (speech recognition)   | 0–100%            | 100% possible in white-box, much lower transfer    |
| Physical/world attacks     | 50–100+%          | Strong attacks (e.g., dynamic UV mapping)          |

## 3. Methodological Factors Influencing ASR

Attack methodologies often revolve around maximizing ASR under tight operational constraints:
- **Transferability-centric frameworks** (e.g. adversarial imitation [2003.12760], semantic cropping for LVLMs [2503.10635], DropConnect-based self-ensembling [2504.18594]) are explicitly designed to improve cross-model ASRs by focusing perturbations on features or regions most likely to generalize.
- **Perceptual Distortion Control** constrains optimization via auxiliary metrics (e.g., SSIM, MAD, LPIPS) and adaptive weighting. For example, the Perceptual Distortion Reduction (PDR) approach [2105.00278] joint-optimizes attack loss and distortion to achieve high ASR with minimal visual artifacts. Adaptive scaling of perturbation steps [2111.13841] also increases ASR by preserving gradient magnitude/direction fidelity.
- **Semantic Alignment and Targeting**: In multimodal attacks on LVLMs, aligning adversarial examples to semantically meaningful features (rather than uniform perturbation) yields ASRs above 90%, even on robust commercial systems [2503.10635]. In NLP backdoor settings, label-specific triggers are optimized to induce targeted errors in ≥99% of cases [2403.13017].

Perturbation co-adaptation—where attack updates overfit the surrogate model—drives down transfer ASR. Mitigation tactics such as randomness injection in model parameters (DropConnect) generate more generic, highly transferable perturbations [2504.18594].

ASR is highly sensitive to attacker knowledge; white-box attacks tend to reach higher ASR compared to black-box or physical-world scenarios, where transferability and stealth constraints dominate.

## 4. ASR in Defense Evaluation and Robustness Assessment

ASR serves as a core metric not only for attack strength but also for defense efficacy. Defensive strategies aim to lower the ASR while preserving model utility on clean data:
- **Input Transformation Defenses** (e.g., JPEG compression, denoising, median blurring) may reduce digital perturbation ASR in controlled conditions, but often fail against adaptive or physical-world attacks without severely affecting benign accuracy [2009.06701].
- **Backdoor Defense**: Entropy-based separation of clean and poisoned data combined with CLIP-guided unlearning reduced ASRs to below 1% across 11 backdoor types, measured regardless of trigger complexity or attack style [2507.05113]. Here, maintaining low ASR is explicitly correlated with effective defense under both clean- and clean-label attack settings.
- **Latent Geometry Alignment**: In LLM safety, geometry-aware alignment (GRACE) reduces the ASR by up to 39% by explicitly separating the latent representations of safe and adversarial outputs [2506.08885]. A high ASR after defense indicates failure of the mitigation to alter the relevant internal representations.

Emerging metrics such as the Attack Successful Rate Difference (ASRD) in NLP [2201.02993] further isolate the causal effect of triggers on ASR by subtracting out OOD and data-perturbation-induced misclassifications from the baseline.

## 5. ASR as an Indicator of Transferability and Robustness Gaps

ASR also exposes cracks in robustness and generalizability:
- **Transferability Gaps**: Even successful white-box image and audio attacks often fail to transfer in black-box settings; for instance, ASRs in ASR systems may drop to near zero for transferred adversarials, a stark difference from the image domain where cross-model ASRs remain high [2007.06622].
- **Physical-World Resilience**: Physical attacks that maintain high ASRs under extensive pose, lighting, and camera variation—such as dynamic-NeRF-based attacks on person detection (92.75% ASR on FastRCNN, 49.5% on YOLOv8 [2501.05783])—are notable for their real-world impact where traditional patch attacks fall short.
- **Prompt Discoverability in LLM Red Teaming**: In prompt-based adversarial evaluation, ASR is interpreted as a distribution over repeated trials (discoverability), capturing the likelihood of attack success across random seeds, system states, or input contexts [2507.22133].

High ASR reveals practical risk: methods achieving near-perfect ASRs using only hard-label access, minimal data, or zero queries demonstrate that model security cannot be guaranteed by obscurity, query-limiting, or holding back gradients [2003.12760][2406.19311].

## 6. Limitations and Multidimensional Assessment

While ASR is essential, relying solely on ASR can mask nuances:
- In stealthy or OOD backdoor settings, high ASR may not pinpoint the effect of an explicit trigger but may instead reflect the model’s broader misclassification tendencies. The introduction of ASRD provides a more discriminative evaluation [2201.02993].
- In some domains, maximizing ASR without regard to perturbation perceptibility—in audio or image space—can yield impractical or human-detectable attacks. Recent approaches explicitly track both ASR and perceptual metrics (e.g., SNR, SSIM, semantic similarity) to ensure that high ASR is meaningful in real-world usage [2105.00278][2110.13250][2403.13017].
- ASR as a statistic is most informative when reported along with the operational definitions of “success,” the attack threat model, and corresponding baselines.

## 7. Outlook and Future Research

The measurement and maximization of ASR provide essential objective foundations for adversarial research, but the metric’s interpretation continues to grow in sophistication:
- Ongoing work seeks to link ASR to underlying geometric, semantic, or representational properties that mediate attack transferability and defense effectiveness [2506.08885].
- Red teaming frameworks increasingly leverage per-attack ASR distributions to characterize attack discoverability and system vulnerability in an operational sense [2507.22133].
- Defense evaluation relies on tracking reduction in ASR as a direct indicator of system hardening, making it critical to architect methodologies and benchmarks that test ASR under the most challenging and realistic attack scenarios.

In summary, Attack Success Rate remains a central, task-invariant metric that encapsulates both adversarial threat and model robustness. Its mathematical foundation and empirical tracking are pivotal for both attack benchmarking and principled progress in designing robust machine learning systems.

Source: https://www.emergentmind.com/topics/attack-success-rates-asrs