---
title: 'Transfer Attack Success Rate: Metrics & Mechanisms'
url: https://www.emergentmind.com/topics/transfer-attack-success-rate
type: topic
---

# Transfer Attack Success Rate: Metrics & Mechanisms

Transfer attack success rate is a central metric in adversarial machine learning, quantifying the ability of adversarial examples crafted for one ("source" or "surrogate") model to successfully induce misclassification in another ("target" or "victim") model without requiring access to its parameters or gradients. This criterion is critical for black-box attacks, threat modeling, evaluation of model robustness, and the practical security assessment of AI/ML systems.

## 1. Definition and Mathematical Formulation

Transfer attack success rate (ASR) is typically defined, for a set of adversarial examples $\{\tilde{x}_i\}$ generated on a surrogate model $f_s$, as the proportion that fool a target model $f_t$:

\[
\text{ASR} = \frac{1}{N} \sum_{i=1}^N \mathbb{I}(f_t(\tilde{x}_i) \neq y_i)
\]

for untargeted attacks, or

\[
\text{ASR}_{\text{targeted}} = \frac{1}{N} \sum_{i=1}^N \mathbb{I}(f_t(\tilde{x}_i) = y^*_i)
\]

where $y^*_i$ is the attacker-chosen target label. Transferability captures the extent to which adversarial examples, optimized to fool a source model under some perturbation constraint (e.g., $L_\infty$-norm), also succeed in fooling unrelated models.

Variants of this metric include top-$k$ attack success rate (the fraction of times the true label is not within the top-$k$ predictions) and other indices based on label rank [2204.00089]. For certain settings, the transferability gain is further normalized to account for distortion, as in:

\[
T_{s,t} = \frac{\int_0^\infty (P_{s,t}(u) - P_t^{BB}(u))\,du}{\int_0^\infty (P_t^{WB}(u) - P_t^{BB}(u))\,du}
\]

where $P_{s,t}(u)$ is the operating characteristic giving the fraction of samples with perturbation < $u$ for the transfer attack, $P_t^{BB}$ for the black-box decision-based attack, and $P_t^{WB}$ for the white-box attack [2304.02312].

## 2. Determinants and Mechanisms of Transferability

Transfer attack success is shaped by both representational and procedural factors, including intrinsic model similarity, the choice of attack/perturbation generation method, and auxiliary data transformations:

- **Feature Representations**: Transferability is high when source and target models share similar internal representations or learn comparable features and vulnerabilities. This relationship can be quantified empirically using normalized symmetric Hausdorff distances between low-dimensional manifold embeddings [2412.05351], Centered Kernel Alignment (CKA), or Diagonal Box Similarity (DBS) for per-layer analysis [2501.18629].
  
- **Attack Generation Algorithms**:
    - **Gradient Transfer and Embedding Search**: Algorithms such as TREMBA [1911.07140] construct a low-dimensional embedding of semantic adversarial perturbations via an encoder–decoder generator trained on a surrogate. Searching for adversarial updates in this space (using Natural Evolution Strategies) yields highly transferrable adversarial patterns due to shared semantic features.
    - **Transformation Ensemble**: Approaches including AITL [2111.13844] and S⁴ST [2410.13891] employ adaptive or highly optimized transformation pipelines (scaling, cropping, color adjustments) to boost the overlapping gradient directions between surrogate and target models, markedly increasing transfer ASR.
    - **Meta-Learning**: Meta-optimization across model and data augmentations, such as in LLTA [2112.06658], causes perturbations to generalize across multiple tasks, further boosting transferability.
    - **Evolution and Genetic Algorithms**: Genetic optimization strategies, especially those preserving or mimicking attribution maps (as in QuScore [2307.06496]), achieve high transferability in the context of interpretable models.
    - **Co-Adaptation Disruption**: DropConnect-based methods [2504.18594] disrupt overfitted dependency patterns in the perturbation, diversifying across model variants and thus preventing overfitting to the surrogate.
    - **Layer/Crop Selection**: Recent works for vision-language or transformer models utilize targeted perturbation of semantically rich local regions or discovery of most vulnerable layers (e.g., DTA [2408.01705], local-aggregated perturbations [2503.10635]), demonstrating high cross-architecture transfer rates.

## 3. Empirical Benchmarks and Quantitative Results

Empirical results consistently show that carefully designed transfer attacks can achieve high ASRs under black-box constraints, with substantial variance depending on the method and scenario:

| Method/Setting          | Average ASR           | Context/Notes                                        |
|------------------------|-----------------------|------------------------------------------------------|
| TREMBA [1911.07140]    | 98% (MNIST); 98.5% (ImageNet); +10% on defended models | NES-based embedding search; high sem. transferability|
| DeepPoison [2101.02562]| 91.74% (with 7% poisoning)  | GAN-based stealthy poisoning, feature-level triggers  |
| AITL [2111.13844]      | 90–96% (ImageNet); +15% over baselines | Adaptive transformation learner                     |
| LLTA [2112.06658]      | 12.85% higher than SOTA | Meta-learning across model/data augmentation         |
| SU Attack [2209.03716] | +12% improvement       | Self-universality, feature similarity loss, targeted |
| S⁴ST [2410.13891]      | 77.7–83% (targeted); +14% over H-Aug | Scaling/augmentation/blockwise local strategies      |
| DTA [2408.01705]       | >90% (ViT downstream)  | Token cosine similarity loss, per-sample attacks     |
| MCD [2504.18594]       | +13% (CNN→Transformer) | DropConnect self-ensemble against targeted models    |
| Commercial LVLMs [2503.10635] | >90% (GPT-4.5, 4o, o1, etc.) | Local-aggregated semantic perturbations, ensemble    |
| IDS/Feature Mismatch [2504.08480] | Varies (capped by TFS: α·f_align+β·A_sim+γ·D_hom) | Sensitive to feature/arch/data divergence           |

Surrogate ensemble attacks [2208.10878] and dynamic source selection (FiT) [2304.02312] can further drive ASR to near 100% for best-case selection, but random surrogate choice can underperform black-box attacks.

## 4. Model Similarity, Overfitting, and Predictability

The degree of representational and architectural similarity between source and target models is a leading determinant of transfer ASR:

- Moderate global similarity (mean CKA ≈ 0.45) with variability across architectures; DenseNet and deeper networks can exhibit lower similarity yet greater vulnerability as attack targets [2501.18629].
- Co-adaptation among features narrows the transferability; strategies that disrupt or diversify this co-adaptation (e.g., DropConnect, pseudo-victim bilevel feedback [2406.02064]) yield more transferable perturbations.
- Predictive models (e.g., DecisionTreeRegressor) trained on metrics such as CKA/DBS + layer counts can predict transfer ASR for certain black-box/C&W attacks with >90% accuracy [2501.18629], but accuracy drops for more complex scenarios.

Manifold alignment or cross-projection of feature embeddings [2412.05351] shows that lower manifold distance (Hausdorff) correlates (ρ ≈ -0.56) with increased transfer ASR, supporting a common-vulnerability hypothesis.

## 5. Impact of Transformations, Embeddings, and Augmentation

Transfer attack success rate improves with strategies that:

- Employ low-dimensional/semantic embeddings: Sampling/optimizing perturbations in the learned latent space rather than pixel space increases both ASR and query efficiency [1911.07140].
- Integrate tailored or adaptive transformation ensembles: Operations such as image resizing, cropping, color augmentation, and blockwise scaling aligned with the threat model increase gradient alignment and mitigate overfitting [2111.13844, 2410.13891].
- Optimize for universality across spatial regions instead of across images: Promoting local invariance (e.g., self-universality) produces features that are robust to network and spatial variation, improving transfer for targeted and untargeted attacks [2209.03716].
- Combine data and model augmentation: Meta-learning over a mixture of model variants (via backprop modification, dropout, or architectural stochastics) improves generalization over possible target models [2112.06658, 2504.18594].

## 6. Limitations, Realism, and Defensive Considerations

While transfer ASR can be close to perfect under ideal lab conditions (aligned features, similar data distributions, comparable architectures), real-world environments impose constraints:

- Architectural or feature-set divergence can sharply reduce transfer viability. The Transferability Feasibility Score (TFS) [2504.08480]:

  \[
  \text{TFS} = \alpha f_{\rm align} + \beta A_{\rm sim} + \gamma D_{\rm hom}
  \]
  
  where $f_{\rm align}$ (Jaccard overlap), $A_{\rm sim}$ (normalized parameter difference), and $D_{\rm hom}$ (Wasserstein data distance) are regression-calibrated predictors of ASR. Negative architectural similarity and moderate data distance often limit ASR in practical IDS scenarios.
  
- Defensive strategies leveraging architectural heterogeneity, intentional data shifts, or sophisticated detection may reduce practical transfer ASR even as laboratory methods report high values.

This creates a mismatch between theoretical and practical transferability: high ASR in controlled experiments may not always carry over to real, heterogenous environments.

## 7. Future Trends and Predictive Modeling

Research increasingly seeks to:

- Develop blind predictors of transferability based on feature correspondence and manifold alignment, allowing a priori vulnerability estimation for black-box targets [2412.05351].
- Enhance transferability by further diversifying perturbation pathways (e.g., novel transformations, task-level augmentations, bilevel and meta-optimization frameworks [2406.02064]).
- Better quantify and balance the trade-off between attack success, perturbation imperceptibility, and computational efficiency (distortion-aware metrics [2304.02312]).
- Adapt transferability frameworks for cross-modal and non-vision domains, building on insights from vision models to IDS [2504.08480] and vision-language models [2503.10635].

Transfer attack success rate remains a multi-faceted, context-dependent measure—sensitive to model similarity, architectural choices, feature alignment, and the sophistication of attack optimization and evaluation criteria. Its ongoing refinement is central to both adversarial research and the robust design of learning systems.

Source: https://www.emergentmind.com/topics/transfer-attack-success-rate