---
title: 'ScoreAdv: Diffusion-Based Adversarial Framework'
url: https://www.emergentmind.com/topics/scoreadv
type: topic
---

# ScoreAdv: Diffusion-Based Adversarial Framework

ScoreAdv is a training-free framework for generating unrestricted adversarial examples with diffusion models, introduced as a method for producing natural-looking targeted attacks that are not constrained by an $\ell_p$ norm and can be generated in unlimited quantity [2507.06078]. It is formulated as score-based targeted sampling: a pretrained diffusion model maintains samples on the natural-image manifold, while gradient-based adversarial guidance steers the denoising trajectory toward a desired target label. The method is designed for both classification and image recognition settings, and its architecture combines classifier-free diffusion guidance, adversarial guidance during reverse diffusion, ScoreCAM-based reference-image injection, and noise optimization [2507.06078].

## 1. Conceptual setting and motivation

ScoreAdv was proposed against the background that conventional adversarial attacks typically rely on $\ell_p$-norm perturbation constraints, which the paper argues do not align with human perception [2507.06078]. Within that critique, unrestricted adversarial examples are defined not as minimally perturbed pixels but as plausible images that fool a model while remaining natural under a generative prior. This places the method in the unrestricted adversarial example literature rather than in small-norm perturbation analysis.

The paper positions ScoreAdv against two earlier design tendencies. GAN-based unrestricted attack methods are described as suffering from instability in training, mode collapse, lower image quality, and weaker interpretability. Diffusion-based attacks, while more stable and capable of higher-quality synthesis, are described as still relying on PGD-style perturbation injection at each step and therefore not fully exploiting diffusion’s denoising process [2507.06078]. ScoreAdv is introduced specifically to use iterative denoising itself as the mechanism that preserves realism, rather than treating diffusion as a mere perturbation engine.

The target object is a targeted unrestricted adversarial example generated from a pretrained diffusion model. The paper formalizes the adversarial sample set as
$$
A_{UAE} \triangleq \{\boldsymbol{x}_{adv} \in \mathcal{G}(\boldsymbol{z}_{adv}, y, \boldsymbol{x}^{ref}) \mid f(\boldsymbol{x}_{adv}) = y_{tar}\}.
$$
Here, $\mathcal{G}$ is the generative model, $f$ is the attacked model, $y$ is the ground-truth label, $y_{tar}$ is the target label, and $\boldsymbol{x}^{ref}$ is an optional reference image [2507.06078]. This formulation makes the attack a constrained sampling problem on the image manifold rather than a direct perturbation problem in pixel space.

A central implication is that ScoreAdv treats adversarial generation as distribution shifting under a diffusion prior. The paper’s claim is not merely that diffusion can synthesize adversarial outputs, but that the denoising trajectory can be gradually biased toward an adversarial distribution while retaining natural image structure [2507.06078].

## 2. Diffusion backbone and targeted sampling formulation

The generative backbone uses classifier-free guidance to sample natural images conditioned on the true label $y$. The guided noise estimate is
$$
\tilde{\boldsymbol{\epsilon}}_t = (1+s_c)\epsilon_\theta(\boldsymbol{x}_t, y) - s_c \epsilon_\theta(\boldsymbol{x}_t),
$$
where $s_c$ is the classifier guidance scale [2507.06078]. This combines conditional and unconditional denoising predictions.

The corresponding reverse diffusion update is
$$
\boldsymbol{\bar{x}}_{t-1} =
\frac{1}{\sqrt{\alpha_t}}
\left(
\boldsymbol{x}_t -
\frac{1-\alpha_t}{\sqrt{1-\bar{\alpha}_t}}
\tilde{\boldsymbol{\epsilon}}_t
\right)
+ \sigma_t z_t.
$$
The paper interprets $\boldsymbol{\bar{x}}_{t-1}$ as the natural denoised step, aligned with source semantics before adversarial steering is imposed [2507.06078]. This is the baseline diffusion trajectory that ScoreAdv then perturbs in a targeted direction.

The method’s central claim is that targeted unrestricted attack generation should occur inside the reverse denoising chain rather than after it. In that sense, ScoreAdv differs from diffusion attacks that append PGD-like perturbations to a generative process; it instead adjusts the sampling dynamics themselves [2507.06078]. This makes the adversarial objective endogenous to the reverse process.

The sampling procedure is also explicitly iterative at two levels. The reverse chain runs over timesteps $t=T,\dots,1$, and the full attack can be repeated for $N$ outer cycles with latent refinement [2507.06078]. The reported default setting uses a pretrained Guided Diffusion model from OpenAI with $T=1000$, $N=3$, $s_a=0.3$, and $s_n=0.8$ [2507.06078]. Those hyperparameters govern the balance between natural synthesis and adversarial steering rather than defining a norm budget.

## 3. Adversarial guidance and score-based distribution shift

The defining mechanism of ScoreAdv is the adversarial guidance term inserted after the natural denoising step. The update is
$$
\tilde{\boldsymbol{x}}_{t-1}
=
\boldsymbol{\bar{x}}_{t-1}
+
\sigma_t^2 s_a
\nabla_{\boldsymbol{\bar{x}}_{t-1}}
\log p_f(y_{tar}\mid \boldsymbol{\bar{x}}_{t-1}),
$$
where $s_a$ is the adversarial guidance scale and $p_f(y_{tar}\mid \cdot)$ is the attacked model’s target-class probability [2507.06078]. Rather than applying a one-shot attack in image space, the method adds a targeted bias at every reverse step.

The paper describes this mechanism as gradually shifting the sampling distribution toward the adversarial distribution [2507.06078]. The effect is cumulative: each step slightly increases the likelihood that the sample will be classified as $y_{tar}$, while the diffusion prior continues to regularize the image toward realism. This stepwise structure is also the basis for the paper’s claim that ScoreAdv remains effective under defenses, because the attack is not a single perturbation but a guided generative trajectory.

The supplementary derivation justifies the update by considering the target-conditional sampling distribution
$$
p_{\theta}(\tilde{\boldsymbol{x}}_{t-1} \mid \bar{\boldsymbol{x}}_{t-1}, y_{tar}),
$$
rewriting it with Bayes’ rule, and approximating $\log p(y_{tar}\mid \tilde{\boldsymbol{x}}_{t-1})$ by Taylor expansion around the mean of the sampling distribution. The resulting shifted mean has the form
$$
\tilde{\boldsymbol{x}}_{t-1} = \bar{\boldsymbol{x}}_{t-1} + s_a \boldsymbol{\sigma}_t^2 \boldsymbol{g},
$$
providing the theoretical basis for gradient-based guidance inside reverse diffusion [2507.06078].

A recurrent theme in the paper is the balance between denoising and attack pressure. This balance is controlled by $s_a$, $s_c$, $s_n$, $T$, and $N$ [2507.06078]. Larger $T^*$ or larger $s_a$ tends to improve attack success rate, but can harm visual quality; moderate $N$ is reported as sufficient, while excessive optimization can distort the image. This is not presented as a fixed optimum but as an operating trade-off inside the method’s design [2507.06078].

## 4. Reference-image conditioning, interpretable saliency, and noise optimization

A second major component is optional reference-image conditioning through ScoreCAM. The saliency map is defined as
$$
\boldsymbol{m} = \operatorname{ScoreCAM}(\boldsymbol{x}^{ref}, f, y),
$$
where the purpose is to identify semantically important regions of the reference image [2507.06078]. The paper gives the ScoreCAM construction as
$$
S(A_l^k)=f\!\left(X \odot s(\operatorname{Up}(A_l^k))\right),
$$
and
$$
\operatorname{ScoreCAM}^c = \operatorname{ReLU}\left(\sum_k \alpha_k^c A_l^k \right), \qquad
\alpha_k^c = \frac{\exp(S^c(A_l^k))}{\sum_j \exp(S^c(A_l^j))}.
$$
This saliency mechanism is presented as interpretable because it identifies which reference regions are preserved or injected during generation [2507.06078].

The reference image is first diffused to the same timestep,
$$
\boldsymbol{x}_{t-1}^{ref}\sim\mathcal{N}\left(\sqrt{\overline{\alpha}_{t-1}}\boldsymbol{x}^{ref},(1-\overline{\alpha}_{t-1})\boldsymbol{I}\right),
$$
and then fused with the adversarially guided sample by inpainting-style composition:
$$
\boldsymbol{x}_{t-1} = \tilde{\boldsymbol{x}}_{t-1}\odot(1-\boldsymbol{m}) + \boldsymbol{x}_{t-1}^{ref}\odot \boldsymbol{m}.
$$
The masking semantics are explicit in the paper: $(1-\boldsymbol{m})$ preserves adversarially sampled content, while $\boldsymbol{m}$ injects reference-image structure in salient regions [2507.06078].

The paper argues that ScoreCAM is preferable to gradient-based CAM variants such as GradCAM because it is more stable and avoids gradient issues, and it reports an ablation in which ScoreCAM outperforms GradCAM and GradCAM++ in both attack success and image quality [2507.06078]. That claim is specific to the paper’s experimental setup, but it establishes saliency choice as a substantive design variable rather than a cosmetic addition.

After synthesis of $\boldsymbol{x}_0$, ScoreAdv further refines the initial latent via noise optimization:
$$
\boldsymbol{x}_T =
\left(\sqrt{\bar{\alpha}_t}\boldsymbol{x}_0 + \sqrt{1-\bar{\alpha}_t}\boldsymbol{\epsilon}\right)
+
\bar{\sigma}_T^2 s_a
\nabla_{\boldsymbol{x}_0}\log p_f(y_{tar}\mid \boldsymbol{x}_0).
$$
The algorithmic summary also describes the latent update as
$$
\boldsymbol{x}_T \leftarrow \boldsymbol{x}_T + \bar{\sigma}_T^2 s_n \boldsymbol{g}_n.
$$
This refinement is performed for $N$ cycles and is intended to pre-shape the latent noise so that the ensuing denoising trajectory is more favorable to the target-class objective [2507.06078].

## 5. Empirical evaluation, model coverage, and defensive robustness

The evaluation spans both generic image classification and face recognition. The paper reports experiments on ImageNet, using 1,000 validation images from 1,000 classes resized to $224\times224$, and on CelebA, with LFW used for recognition experiments; for recognition, 1,000 images are randomly selected [2507.06078]. Ten target classification models are tested across three families: CNNs, Transformers, and MLPs. The listed CNNs are ResNet-50, VGG19, Inception-v3, WideResNet50-2, and ConvNeXt; the listed Transformers are ViT-B/16, Swin-B, and DeiT-B; the listed MLPs are Mixer-B/16 and Mixer-L/16. For recognition, the paper evaluates IR152, FaceNet, and MobileFace [2507.06078].

| Evaluation aspect | Reported setting |
|---|---|
| Classification datasets | ImageNet |
| Recognition datasets | CelebA, LFW |
| Classification model families | CNNs, Transformers, MLPs |
| Recognition models | IR152, FaceNet, MobileFace |
| Attack regimes | White-box and black-box |

The paper states that ScoreAdv achieves state-of-the-art attack success rates in most settings and improves transferability relative to prior baselines [2507.06078]. Concrete white-box examples reported from ImageNet include 97.6% on ResNet-50, 100.0% on VGG19, and 93.4% on Inception-v3. It also reports strong cross-model transfer to Transformers and MLPs, and particularly strong performance in black-box recognition attacks [2507.06078].

Image quality is assessed with FID, LPIPS, PSNR, and SSIM. The paper reports FID $44.932$, LPIPS $0.124$, PSNR $30.817$, and SSIM $0.8319$, and states that compared with the best prior diffusion baseline, AdvDiffuser, ScoreAdv improves all four reported image-quality indicators: lower FID, lower LPIPS, higher PSNR, and higher SSIM [2507.06078]. In the paper’s interpretation, these values indicate more natural and coherent adversarial images.

Robustness is evaluated against a diverse set of defenses: AdvProp, R&P, RandS, Bit-Red, Adv-Inc-v3, IncRes-v2 ensemble, SR, NRP, and DiffPure [2507.06078]. The paper reports that ScoreAdv remains strong under these defenses and is slightly more robust than previous methods, including AdvDiff. The stated explanation is the interaction between diffusion denoising and adversarial guidance, together with the method’s distribution-shifting behavior [2507.06078].

## 6. Methodological significance, constraints, and related directions

The principal significance claimed for ScoreAdv is that diffusion models can serve as a principled adversarial generation engine rather than as post hoc image priors [2507.06078]. The method combines score-based targeted sampling, stepwise adversarial guidance, interpretable saliency-based reference injection, and noise optimization into a single training-free framework. A plausible implication is that ScoreAdv reframes unrestricted adversarial example generation as controlled generative inference, with realism preserved by the denoising prior instead of by explicit perturbation constraints.

The paper is also explicit about constraints. ScoreAdv relies on a pretrained diffusion model and is primarily developed for targeted attacks [2507.06078]. The reference-image branch is optional, which means that content control can depend on whether a suitable reference image is available. Its practical behavior depends on balancing $T$, $N$, $s_a$, and $s_n$; too much adversarial guidance or too many optimization steps can degrade image quality. As with many gradient-based attacks, best performance in targeted settings depends on white-box access or a strong surrogate model [2507.06078].

A common misconception in unrestricted attack research is that removing an $\ell_p$ budget eliminates all structure. ScoreAdv does not support that interpretation. Its unrestrictedness refers to the absence of an explicit norm constraint, but the attack remains strongly structured by the diffusion prior, the class-conditioning mechanism, and optionally the ScoreCAM mask [2507.06078]. Another misconception is that diffusion is used only to beautify adversarial perturbations. The paper explicitly argues the opposite: denoising is the primary mechanism that keeps samples natural, while adversarial steering is injected as a controlled bias inside the reverse process [2507.06078].

In a broader score-guided generation context, related work includes “Score-Guided Generative Adversarial Networks” [2004.04396], which used an evaluator-derived score as an auxiliary signal for GAN training rather than for inference-time adversarial sampling. That earlier work is relevant because it addresses how score-based objectives can guide generation without collapsing into trivial exploitation of the scoring model [2004.04396]. ScoreAdv differs in both objective and mechanism: it is training-free, diffusion-based, targeted, and directly concerned with unrestricted adversarial image generation [2507.06078].

Taken together, ScoreAdv occupies a specific position in adversarial machine learning: it is a diffusion-native, inference-time, targeted unrestricted attack framework whose novelty lies in integrating denoising, adversarial guidance, interpretable reference injection, and latent refinement into a single generative attack pipeline [2507.06078].

Source: https://www.emergentmind.com/topics/scoreadv