Unified Optimization View of Transfer Attacks

• The paper presents a unified optimization view that merges single-level and bilevel formulations to analyze surrogate-to-target attack transferability.
• It highlights key metrics including intrinsic vulnerability, gradient alignment, and loss surface variance that dictate transfer attack effectiveness and robustness trade-offs.
• Empirical evaluations on datasets like ImageNet and CIFAR-10 confirm that coordinated hyperparameter tuning and flatness regularization significantly improve transfer success rates.

Transfer attacks, in adversarial machine learning, refer to the phenomenon where adversarial examples crafted against a surrogate (source) model remain effective when tested against unseen target models. A unified optimization view seeks to describe and analyze the conditions and algorithmic mechanisms through which transferability emerges, encompassing both single-level and bilevel formulations, provable risk bounds, and algorithmic designs to systematically enhance or mitigate transfer.

1. Unified Optimization Frameworks for Transferability

The foundational perspective is that both test-time evasion and training-time poisoning attacks instantiate a constrained maximization of an attack loss via input or data perturbations. Formally, one optimizes

$$x^\star = \arg\max_{x' \in \Phi(x)}\;A(x', y, \kappa)$$

for sample $x$ and feasible perturbation region $\Phi(x)$, regardless of attack modality (Demontis et al., 2018). In evasion (classic adversarial attack), the surrogate model $f_{\hat{w}}$ is known, and the attacker maximizes the loss subject to norm constraints—formally,

$$\max_{x'} \; \ell(y, x'; \hat{w}) \quad \text{s.t.} \;\; \|x'-x\|_p \leq \varepsilon.$$

For transfer, one studies the effectiveness of this $x^{\star}$ on a target $f_w$, which is out-of-sample.

Bilevel optimization generalizes this framework: poisoning attacks, and advanced transfer methods, involve nested objectives—optimizing a perturbation (or initialization) for one model (LL), controlling the harm on others (UL) (Liu et al., 2024). BETAK formalizes this as

$$\min_{\delta \in \mathcal{C}} F(\phi^*(\delta)), \;\; \text{subject to} \;\; \phi^*(\delta) = \arg\min_{\phi \in \mathcal{C}} f(\phi)$$

where $f$ targets the surrogate loss and $F$ aggregates pseudo-victim or ensemble models.

Recent theory further quantifies transfer risk for black-box attacks with PAC-Bayesian bounds on adversarial success across model distributions (Zheng et al., 23 Apr 2025). The transfer risk decomposes as

$$R_{T}(\hat{x}) = R_{S}(\hat{x}) + \mathcal{E}_{\text{trans}}(\hat{x}),$$

where $R_S$ is surrogate risk and $\mathcal{E}_{\text{trans}}$ measures surrogate–target discrepancy. The bound explicitly links surrogate loss, flatness (sharpness), and distributional gap.

2. Key Metrics and Determinants of Transferability

Transferability is governed by a triad of metrics (Demontis et al., 2018, Zheng et al., 23 Apr 2025):

• Intrinsic vulnerability $S(x, y) = \|\nabla_x \ell(y, x; w)\|_q$: The norm of target loss gradients; steep boundaries amplify the effect of perturbations.
• Gradient alignment $$R(x, y) = \cos(\nabla_x \ell(y, x; \hat{w}), \nabla_x \ell(y, x; w))$$: Cosine similarity of surrogate and target gradients; high alignment guarantees effective transfer.
• Loss surface variance $V(x, y)$: Measures surrogate model instability; large variance reduces transfer.

A transfer attack is most effective when the surrogate model yields high alignment with the target, the target is intrinsically vulnerable, and the surrogate’s loss landscape is stable. These concepts generalize to the bilevel setting, where the UL objective can explicitly shape the LL starting conditions to optimize transfer success (Liu et al., 2024).

The PAC-Bayes bound (Zheng et al., 23 Apr 2025) further demonstrates that both empirical risk and "flatness" (sharpness) of the adversarial minimum over diverse surrogates critically constrains transferability:

• Flat minima (low sharpness) yield robust adversarial examples that remain effective as the target model shifts.
• The adversarial model discrepancy $D_\phi(P_\mathcal{T}\|P_\mathcal{S})$ quantifies the risk gap due to distributional shift between surrogate and target ensembles.

3. Bilevel Formulations and Algorithmic Instantiations

The bilevel perspective, as instantiated in BETAK (Liu et al., 2024), recasts transfer attacks into an explicit optimization of UL objectives over pseudo-victim models, while the LL attacker solves the standard white-box maximization for a surrogate. Initialization strategies, such as starting the LL attack from a UL-refined perturbation, bias the search toward transferable directions. Hyper Gradient Response (HGR) estimation enables principled UL-to-LL feedback by differentiating the nested optimization trajectory; Dynamic Sequence Truncation (DST) adaptively selects the best LL iteration for UL updates.

Pseudocode for BETAK demonstrates the initialization, surrogate attack, UL feedback via HGR, and computational savings via DST (see Section 6 of (Liu et al., 2024)).

4. Model Diversity, Flatness, and Transfer Bounds

The theoretical framework in (Zheng et al., 23 Apr 2025) establishes that transfer risk is minimized by jointly:

• Lowering empirical surrogate risk,
• Regularizing for flat minima via sharpness penalties,
• Reducing surrogate–target model discrepancy through diversity.

Algorithmic instantiation (DRAP) implements these principles by creating ensembles of diverse surrogate models (between-distribution and within-distribution sampling), and optimizing adversarial perturbations that promote flatness over all surrogates. The sharpness term

$$\max_{\|\epsilon\| \le \rho} R_{\hat{S}}(\hat{x} + \epsilon) - R_{\hat{S}}(\hat{x}),$$

measures flatness, and diversity is engineered to narrow $D_\phi$. DRAP achieves state-of-the-art transfer success rates, especially when composed with input transformations and is empirically validated against both standard and adversarially trained models.

5. Optimization Hyperparameters and Robustness Trade-offs

Hyperparameter selection during model training strongly influences transfer robustness (Zimmer et al., 17 Nov 2025). Learning rate ($\eta$), weight decay ($\lambda$), batch size ($B$), and momentum ($\mu$) implicitly regularize the loss landscape. Decreasing $\eta$ and increasing $B$, $\lambda$, and $\mu$ tends to yield flatter minima and less aligned gradients, thus lowering transferability. However, for query-based attacks, increasing $\eta$ improves gradient estimation convergence and boosts query-robustness.

A conflict arises—parameter regimes optimal for transfer-robustness degrade query-robustness. Joint optimization across $(\eta, \lambda, \mu, B)$ via NSGA-II enables Pareto-efficient trade-offs, as demonstrated by up to $64\%$ improvement in transfer-robustness and $28\%$ in query-robustness without adversarial training (Tables 1–2 of (Zimmer et al., 17 Nov 2025)). Distributed and ensemble setups amplify these benefits due to induced gradient misalignment.

6. Unification and Implications for Attack and Defense Strategies

Historical empirical findings that simple, weak attacks transfer better than strong, overfit attacks are now explained by the unified optimization view: strong attacks that over-optimize for the surrogate model succumb to sharp minima, poor model diversity, and low gradient alignment. Modern frameworks (Zheng et al., 23 Apr 2025, Liu et al., 2024) generalize earlier heuristics (I-FGSM, PGD, MI-FGSM, ensemble attacks) as partial optimizations of risk, flatness, or discrepancy. Only the full bilevel or flat-minima/diverse-surrogate approach systematically maximizes transferability.

For defenders, these frameworks imply that flattening the model's loss landscape, disrupting gradient alignment, and regularizing model complexity are effective countermeasures, and practical hyperparameter tuning can yield large robustness gains against transfer attacks (Zimmer et al., 17 Nov 2025), as substantiated in distributed and ensemble contexts.

7. Empirical Evaluations and Future Directions

Extensive empirical analysis across datasets (ImageNet, CIFAR-10, MNIST, DREBIN, LFW) and classifier families confirms the optimization view. Advanced bilevel methods (BETAK) yield $20$–$50\%$ gains in black-box success rates; DRAP exceeds $46.2\%$ success in targeted settings, outperforming prior techniques (Liu et al., 2024, Zheng et al., 23 Apr 2025). The alignment ($R$) and loss variance ($V$) metrics directly correlate with transfer success (Pearson $r>0.8$), and ablation studies establish the necessity of model diversity and sharpness regularization (Demontis et al., 2018, Zheng et al., 23 Apr 2025).

Open research directions include efficient bilevel optimization under resource constraints, principled construction of pseudo-victim ensembles, extensions to sequential or non-IID federated scenarios, and deeper integration of PAC-Bayes bounds to guard against adaptive transfer attacks. The unified optimization view provides a rigorous foundation for systematic algorithm design and theoretical analysis, framing ongoing work in both adversarial robustness evaluation and the design of transferable attack and defense strategies.