Proxy Adversarial Robustness

• Proxy adversarial robustness is a framework that utilizes auxiliary models or synthetic proxies to evaluate and improve adversarial resistance in deep learning systems.
• It encompasses methodologies like HPT, LAST, and robust proxy learning that transfer robustness metrics and techniques across various model types and domains.
• Empirical findings show that proxy methods can significantly boost adversarial accuracy while lowering computational costs for robust evaluations.

Proxy adversarial robustness refers to a family of methodologies, metrics, and phenomena centered on improving, transferring, and evaluating adversarial robustness in deep learning systems using auxiliary models, synthetic data, representative features, or efficient computational proxies. Proxy-based approaches span theory, algorithmic transfer, empirical evaluations, and practical efficiency in supervised, multimodal, and reinforcement learning domains. They address both the computational challenges and practical limitations of direct adversarial training or exhaustive robustness certification.

1. Formal Definitions and Theoretical Foundations

Proxy adversarial robustness can be formalized in terms of the performance of a model (“proxy” $P$) on adversarial examples crafted for another (“target” $T$). In the $\ell_\infty$-bounded threat model, given input $x\in\mathbb{R}^d$, perturbation budget $\epsilon>0$, and text prompt $t$ (for VLMs), adversarial examples are generated by maximizing task loss for $T$: $x^a = x + \delta$, with $\|\delta\|_\infty \le \epsilon$. The adversarial sample set $A_T$ is: $$A_T = \{ x^a : x^a = x + \delta, \|\delta\|_\infty\leq\epsilon, \delta \leftarrow \arg\max\ell(C_T(x+\delta), y) \}$$ Proxy adversarial robustness is then the clean-accuracy of $P$ on these adversarial samples: $$ProxyRobust(P \leftarrow T) = \mathbb{E}_{(x,y)} [ \mathbf{1}\{ \arg\max C_P(x^a, t) = y \} ], \quad x^a \in A_T$$ Empirical findings in vision-LLMs (VLMs) like CLIP show that vanilla models (without adversarial training) can retain high accuracy against adversarial examples generated by other CLIP variants (Fu et al., 19 Jan 2026).

2. Proxy Transfer Algorithms and Robustness Distillation

Heterogeneous Proxy Transfer (HPT) in VLMs

The HPT framework enables transfer of proxy robustness between CLIP variants without requiring adversarially trained proxies. A key contribution is the Generalization-Pivot Decoupling (GPD), which splits transfer into:

• Generalization-Anchored Warm-Up: low learning rate, aligning target on adversarial inputs to proxy on clean data via KL divergence.
• Generalization-Pulled HPT: high learning rate, aligning target and proxy predictions, now both on adversarial inputs.

Exponential moving average (EMA) and parameter “pulling” mitigate overfitting and loss of zero-shot generalization.

Loss functions:

$$L_{\mathrm{GA}} = \frac1N\sum_{i=1}^N \mathrm{KL}\!\left(\mathrm{softmax}[T(x^{a}_i,t_i)] \| \mathrm{softmax}[P(x_i,t_i)]\right)$$

$$L_{\mathrm{RT-CLIP}} = \frac1N\sum_{i=1}^N \mathrm{KL}\!\left(\mathrm{softmax}[T(x^{a}_i,t_i)] \| \mathrm{softmax}[P(x^{a}_i,t_i)]\right)$$

Risk bounds relate adversarial and clean risks on proxy and target—minimizing $L_{\mathrm{GA}}$ minimizes attack defense error, while EMA maintains clean maintenance error (Fu et al., 19 Jan 2026).

Proxy-Guided Training in Supervised Image Models

LAST (Learn from the Past) uses historical snapshots of the target model as the proxy, dynamically correcting target updates via the difference in gradients and employing self-distillation regularization for stability. Theoretical guarantees include boundedness and Cauchy sequence convergence of updates, yielding stable boosts to robust accuracy in both single-step and multi-step adversarial training (Liu et al., 2023).

Robust Proxy Learning in Feature Space

Robust Proxy Learning actively constructs class-wise robust feature representations by optimizing for robust perturbations (CEO) and aligning samples with hardened representative proxies. This reduces vulnerability associated with non-robust channels and filters brittle directions, leading to consistently higher adversarial accuracy gains (Lee et al., 2023).

3. Proxy Metrics for Robustness Evaluation

Proxy metrics are vital for scalable robustness estimation:

• Greedy Worst-Case Reward (GWC): In RL, evaluates the worst-case sequence of actions using fast interval bound propagation, closely approximating true worst-case reward at $O(T)$ cost (Oikarinen et al., 2020).
• Adversarial Hypervolume (AHV): Measures robustness over all perturbation budgets, integrating the minimal confidence across all attack strengths to assess the area under the adversarial frontier. This captures the global tradeoff between confidence and perturbation, providing more discriminative benchmarking than single-point adversarial accuracy (Guo et al., 2024).
• Fast Proxy Metrics for LLMs: Embedding-space attacks, direct prompting, and prefilling yield attack success rate (ASR) proxies highly correlated ($r_p = 0.87$, $r_s = 0.94$) with full attack ensembles, reducing evaluation time by three orders of magnitude (Beyer et al., 14 Feb 2025).
• ΔMAUVE for Text: Measures change in MAUVE quality score pre- and post-adversarial attack, tightly correlating with human-perceived degradation in text detection tasks (Crothers et al., 2022).

4. Proxy-Based Robustness in Architectural Search and Synthetic Data

Zero-Cost Proxies in NAS

CRoZe (Consistency across Robust Zero-cost Evaluation) utilizes feature, parameter, and gradient consistency under input and parameter perturbations, achievable with two forward/backward passes per architecture. CRoZe robustly predicts ranking and selection for architectures with respect to adversarial robustness and generalizes across datasets and perturbation types (Ha et al., 2023).

Proxy Distributions in Robust Training

Porting additional samples from generative models (proxy distributions) into adversarial training tightens the upper bound on robustness transfer via conditional Wasserstein distance. Diffusion proxies, identified via robust discrimination (ARC metric), yield superior robustness transfer compared to GANs, enabling $+7.5$ pp improvements in robust accuracy on CIFAR-10 without extra real data (Sehwag et al., 2021).

5. Practical Pathologies and Countermeasures in Proxy-Based Evaluation

Computation of adversarial robustness via first-order attacks (PGD) regularly overestimates resilience unless numerical instability, non-differentiability, and insufficient iteration are addressed. Techniques like randomized smoothing, randomized subgradients, and quasi-Newton updates yield proxy measures that are more faithful to true worst-case adversarial error (Lee et al., 2020).

6. Proxy Attacks and Gradient Diversity

For randomized neural networks, the effectiveness of EOT-style proxy-gradient attacks depends on the directional concentration of sample-wise loss gradients. GradDiv regularization penalizes gradient alignment, mitigating vulnerability to proxy-gradient attacks and reducing transferability among ensemble models (Lee et al., 2021).

7. Empirical Performance and Limitations

Across domains (vision, text, RL, LLMs), proxy-based methods yield:

Proxy Method	Domain	Robustness Gain	Evaluation Cost
HPT-GPD	VLM (CLIP)	+5.21 pp AA, minimal NG loss	Moderate
LAST	Image, supervised	+9.2–20.3 pp RA	Modest
Robust Proxy Learning	Image-class, features	+3–4% AA	Per-class, per-epoch
CRoZe	NAS	Spearman ρ ~0.47–0.53, robust selection	2 forward+backward
GWC	RL	Closely matches AWC	$O(T)$
ΔMAUVE	Text detection	Large human-degradation	Same as attack cost
Fast Proxy Attacks	LLMs	$r_p=0.87$, $r_s=0.94$	10$^3$–10$^4\times$ speedup
Diffusion Proxy (PORT)	Training w/ syn. data	+7.5 pp AA, cert.	Trivial for training

Proxy approaches are sensitive to choice (architecture similarity, diffusion vs. GAN, regularizer strength), theoretical gaps remain (e.g., proxy robustness in VLMs but not classical classifiers). Overfitting, loss of out-of-distribution generalization, and compute costs can arise if not carefully decoupled via scheduling or regularization.

8. Open Questions and Future Directions

Areas for further development include:

• Theory: Why do certain architectures (e.g., CLIP variants) admit intrinsic cross-model proxy robustness, while standard classifiers do not?
• Scheduling: Unsupervised or adaptive schedules for proxy transfer and regularization.
• Expansion: Generalization to other multimodal models, certified defenses, and broader perturbation families.
• Evaluation: Adoption of proxy metrics (AHV, ΔMAUVE, GWC) for holistic and efficient benchmarking.
• Optimization: End-to-end training of proxies, dynamic ensembles, proxy selection, and diversity maximization.

Proxy adversarial robustness establishes a versatile paradigm for efficiently transferring, measuring, and enhancing the resilience of deep learning models under adaptive threats, with key empirical and algorithmic advances across subfields of machine learning and robustness science.