Guided Diffusion Attacks

• Guided diffusion attack is a class of adversarial methods that exploit diffusion model processes by injecting malicious guidance into reverse sampling, conditioning, or training pipelines.
• These attacks enable adversarial example generation, privacy leakage, and backdoor implantation while maintaining visual or semantic plausibility across various domains.
• Practical evaluations reveal trade-offs between attack efficacy and output realism, assessed via metrics like ASR, FID, LPIPS, and other domain-specific benchmarks.

Searching arXiv for the cited papers and closely related work on guided diffusion attacks. Guided diffusion attack denotes a family of attack paradigms in which a diffusion model, its conditioning pathway, or its reverse denoising dynamics is exploited to steer generation or reconstruction toward an adversarial objective. Across recent literature, the term does not describe a single algorithm but a class of mechanisms: some methods inject gradients into reverse sampling, some optimize text or condition embeddings that guide a frozen latent diffusion backbone, some use diffusion priors for privacy leakage or constrained adversarial refinement, and some poison diffusion training so that triggers activate malicious behavior at inference (Guo et al., 2024, Lei et al., 15 Sep 2025, Dogoulis et al., 15 Jun 2025, Medghalchi et al., 2024, Li et al., 2024, Souri et al., 2024, Xia et al., 31 Jul 2025). The resulting attack surface spans targeted and untargeted misclassification, data reconstruction from intermediate representations or gradients, backdoor implantation, unsafe content generation, and adversarial control of diffusion-based policies (Ma et al., 2024, Meng et al., 2024, Chen et al., 2024).

1. Scope and conceptual boundaries

Across the cited works, “guidance” refers to an external signal that biases diffusion toward a malicious target while attempting to preserve realism, validity, or stealth. The guidance source may be semantic text, classifier-like gradients, feature-matching objectives, control maps, target subgraphs, or poisoned conditional data. The attack target may be a downstream classifier, a split-inference server, a text-to-image model, a graph generator, a human-activity recognizer, or a robotics policy (Guo et al., 2024, Lei et al., 15 Sep 2025, Ma et al., 2024, Ye et al., 23 Oct 2025, Chen et al., 2024).

A recurrent distinction is between attacks that modify reverse diffusion directly and attacks that use diffusion more indirectly. In "Diffusion Attack: Leveraging Stable Diffusion for Naturalistic Image Attacking" (Guo et al., 2024), Stable Diffusion is used only to synthesize a style exemplar; the adversarial optimization occurs afterward in pixel space. By contrast, DRAG modifies latent-space sampling through intermediate-representation guidance, and AGD for multimodal LLMs injects adversarial semantics into the reverse-diffusion noise itself (Lei et al., 15 Sep 2025, Xia et al., 31 Jul 2025). This difference is central to the modern use of the term.

Guidance locus	Mechanism	Representative papers
Reverse denoising dynamics	Gradients modify $\epsilon_\theta$, $\mu_\theta$, or the sampling noise	(Lei et al., 15 Sep 2025, Dogoulis et al., 15 Jun 2025, Xia et al., 31 Jul 2025, Chen et al., 2024)
Conditioning pathway	Text prompts, CLIP losses, learned prompts, or control maps steer a frozen or partially frozen diffusion model	(Guo et al., 2024, Medghalchi et al., 2024, Ma et al., 2024, Briston et al., 22 Jun 2026)
Training-time poisoning	Triggered data or trigger-gated objectives implant malicious conditional behavior	(Li et al., 2024, Souri et al., 2024, Lapid et al., 7 Jul 2025, Han et al., 2024, Ye et al., 23 Oct 2025)

This taxonomy suggests that guided diffusion attack is best understood as an attack principle rather than a single threat model.

2. Core mechanisms of guidance

Most papers share the standard diffusion formalism. The forward process adds Gaussian noise,

$$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$

and the reverse process uses a learned denoiser or score model to approximate recovery of $x_0$ (Guo et al., 2024, Lei et al., 15 Sep 2025, Li et al., 2024, Xia et al., 31 Jul 2025). In classic guided diffusion, one perturbs the score with an auxiliary gradient such as

$$\nabla_{x_t}\log p(x_t \mid y) \approx \nabla_{x_t}\log p(x_t) + s \cdot \nabla_{x_t}\log p(y \mid x_t),$$

thereby modifying $\mu_\theta$ or $\epsilon_\theta$ during sampling (Guo et al., 2024).

The concrete attack instantiations differ in where this auxiliary signal originates. DRAG computes a guidance gradient from a feature-matching loss between the intercepted split-inference representation and the features of the current reconstructed image, then replaces the DDIM noise with a Diffusion with Spherical Gaussian constraint update (Lei et al., 15 Sep 2025). CarDiff augments a deterministic DDIM trajectory with an explicit constraint-gradient correction after each reverse step, interpreting the update as a closed-form proximal move toward a feasible manifold (Dogoulis et al., 15 Jun 2025). AGD for multimodal LLMs computes a target-semantics guidance term from CLIP features of a target image synthesized from the target text, then injects this signal into the reverse noise via

$$\tilde{\epsilon}_t = \epsilon_\theta(x_t,t) + \gamma\,\mathrm{sign}(\epsilon_{\mathrm{tar}}),$$

with an EMA-based inner loop to stabilize the semantics (Xia et al., 31 Jul 2025).

Other methods guide diffusion without altering reverse-time equations. P2P keeps Stable Diffusion v1-4 frozen and optimizes only token embeddings in the CLIP text encoder. Its joint objective combines a classification term and a denoising alignment term,

$$-\lambda_{\mathrm{cls}}\mathrm{CE}(f(x_{\mathrm{adv}}), y) + \lambda_{\mathrm{denoise}}\|\epsilon-\epsilon_\theta(z_t,t,\mathbf C)\|_2^2,$$

and restricts optimization to early reverse steps to improve efficiency and imperceptibility (Medghalchi et al., 2024). The VR-oriented Diffusion Attack instead uses Stable Diffusion to generate a style exemplar $s$ from a prompt, then optimizes a stylized adversarial image $\mu_\theta$0 with

$\mu_\theta$1

where naturalness is enforced through style, content, and smoothness rather than explicit $\mu_\theta$2 constraints (Guo et al., 2024).

A common misconception is therefore that guided diffusion attack always means classifier guidance inside the sampler. Several papers explicitly contradict that interpretation (Guo et al., 2024, Medghalchi et al., 2024).

3. Adversarial example generation and targeted behavioral steering

A major branch of the literature uses guided diffusion to generate adversarial examples that remain visually or semantically plausible. In VR-focused naturalistic attacking, the attack pipeline first samples a prompt-conditioned style image with Stable Diffusion, then performs masked neural style transfer, and finally adversarial refinement against Inception-v3 while keeping style and content terms active (Guo et al., 2024). The reported aesthetic and quality scores are higher than those of baseline physical attacks: NIMA 4.78, Topiq_iaa 4.46, Topiq_nr 0.62, and Tres 72.32, versus notably lower scores for Woitschek et al. and SLAPs (Guo et al., 2024). Attack examples include a T-shirt misclassified as “umbrella” or “lighthouse” with $\mu_\theta$3 confidence and backpacks misclassified as “sleeping bag” or “zebra” with $\mu_\theta$4 confidence (Guo et al., 2024).

P2P extends this line to breast ultrasound imaging. Rather than retraining a domain-specific diffusion model, it updates prompt embeddings in a frozen Stable Diffusion v1-4 backbone and optimizes only early reverse steps, with $\mu_\theta$5 sampled from $\mu_\theta$6 (Medghalchi et al., 2024). On BUSI with ResNet34, P2P reports FID 43.03, LPIPS 0.12, SSIM 0.81, and Success Rate 0.97, whereas Diff-PGD reports FID 117.49, LPIPS 0.31, and Success Rate 1.00 (Medghalchi et al., 2024). The same paper emphasizes that the implemented loss is untargeted even though the broader motivation mentions targeted outcomes (Medghalchi et al., 2024).

Constraint-aware guided diffusion appears in CarDiff, where cyclic PGD steps are followed by deterministic DDIM refinement with equality-constraint corrections for tabular data (Dogoulis et al., 15 Jun 2025). On the lcld_v2_iid benchmark, CarDiff yields Robust Accuracy $\mu_\theta$7 and constraint violation $\mu_\theta$8, compared with PGD at $\mu_\theta$9 and $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$0, and CPGD at $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$1 and $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$2 (Dogoulis et al., 15 Jun 2025). This indicates that guided diffusion can be used to enforce domain rules rather than merely visual stealth.

In multimodal LLM attacks, AGD embeds target semantics into the diffusion noise channel rather than only into high-frequency image perturbations (Xia et al., 31 Jul 2025). Reported results include UniDiffuser Ensemble CLIPScore 0.721 with ASR 95.4%, BLIP-2 0.714 with ASR 89.4%, MiniGPT-4 0.648 with ASR 88.0%, and Qwen2-VL 0.387 with ASR 30.5% (Xia et al., 31 Jul 2025). The paper further reports stronger robustness than prior baselines under JPEG, SOAP, DiffPure, and MimicDiffusion (Xia et al., 31 Jul 2025).

A related but distinct setting appears in diffusion-based policies for robotics. DP-Attacker does not attack pixels via a single end-to-end policy loss; it attacks the denoiser’s noise prediction at sampled diffusion steps, thereby steering the final action trajectory while avoiding backpropagation through the full denoising chain (Chen et al., 2024). Across manipulation tasks such as Can, Square, Transport, and Toolhang, targeted and untargeted attacks reduce success rates sharply, and offline perturbations transfer across frames (Chen et al., 2024).

4. Privacy leakage and inverse reconstruction

Another major use of guided diffusion attack is reconstruction from partial internal information. DRAG targets split inference, where the client transmits an intermediate representation $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$3 to the server (Lei et al., 15 Sep 2025). The attack uses Stable Diffusion v1.5 as a latent prior, defines an IR-alignment loss over the victim’s deep features, injects guidance through the DSG noise replacement rule, and strengthens optimization by self-recurrence (Lei et al., 15 Sep 2025). On CLIP-ViT-B/16 at layer 12, DRAG reports MS-SSIM 0.6735, LPIPS 0.1857, and DINO 0.9331, compared with LM at 0.2560, 0.6024, and 0.4248, and GLASS at 0.2396, 0.5790, and 0.4553 (Lei et al., 15 Sep 2025). The same paper shows substantial recovery even under DISCO and NoPeek defenses when adaptive filtering is used (Lei et al., 15 Sep 2025).

Gradient leakage attacks use a related idea but condition on parameter gradients rather than split-layer activations. "Is Diffusion Model Safe? Severe Data Leakage via Gradient-Guided Diffusion Model" fine-tunes a DDIM denoiser so that generated images produce gradients aligned with a leaked target gradient via cosine similarity (Meng et al., 2024). It reports reconstructions up to $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$4 resolution, with ImageNet MSE 0.0030, SSIM 0.9999, PSNR 25.29, and LPIPS $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$5, whereas DLG on the same resolution reports MSE 0.0707, SSIM 0.9977, PSNR 11.46, and LPIPS $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$6 (Meng et al., 2024).

GG-CDMs address the harder case of noise-perturbed gradients. The method models the leaked gradient as a noisy measurement, defines an attack loss $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$7, and couples DDIM sampling with Gaussian Spherical Sampling and a guidance rate $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$8 (Meng et al., 13 Nov 2025). On CelebA with a CNN and original gradients, GSS-R achieves MSE $$q(x_t \mid x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I),$$9, PSNR 41.12 dB, and LPIPS $x_0$0; under Gaussian noise $x_0$1, it still reaches 14.98 dB PSNR and outperforms GGL by $x_0$2 dB (Meng et al., 13 Nov 2025). The same work formalizes Reconstruction Vulnerability as

$x_0$3

and relates higher RV to greater privacy leakage (Meng et al., 13 Nov 2025).

Taken together, these works suggest that diffusion priors convert ill-posed inversion problems into constrained sampling problems on a natural-image manifold.

5. Poisoning, backdoors, and malicious conditionality

Training-time poisoning constitutes a third major interpretation of guided diffusion attack. Guided Diffusion Poisoning synthesizes base samples from scratch with strong class guidance and weak poisoning or backdoor guidance, then hands those synthesized bases to downstream attacks such as Witches’ Brew, Poison Frogs, Bullseye, HTBA, or Sleeper Agent (Souri et al., 2024). On CIFAR-10 targeted poisoning with ResNet-18 and $x_0$4, GDP reports 30% success with 25 images and 70% with 50 images, whereas Poison Frogs, Bullseye, and Witches’ Brew report 0%, 0%, and 20% or less in the cited settings (Souri et al., 2024).

Other works poison the diffusion model itself. DIFF2 fine-tunes a security-enhancing diffusion model so that a trigger-gated guidance term is activated during reverse denoising, driving purification or certification toward an adversarial target distribution while preserving benign behavior on clean inputs (Li et al., 2024). On CIFAR-10 DDPM purification with $x_0$5, benign untargeted ASR on trigger inputs is 11.6%, whereas DIFF2 raises it to 81.7%; targeted ASR rises from 10.4% to 78.4%, while Clean ACC remains about 89% (Li et al., 2024). The same paper shows certified accuracy on triggered inputs collapsing from 59.8% to 8.7% at $x_0$6 under randomized smoothing (Li et al., 2024).

UIBDiffusion replaces conspicuous backdoor triggers with an imperceptible universal perturbation adapted from universal adversarial perturbations (Han et al., 2024). On DDPM/CIFAR-10, it reaches 100% ASR at 5% poison rate, reports SSIM 0.98 at that rate, and bypasses Elijah and TERD, which detect and neutralize prior structured-trigger attacks but not the imperceptible trigger (Han et al., 2024).

Conditional diffusion backdoors also appear in domain-specific forms. A ControlNet poisoning attack injects poisoned control-image pairs so that subtle patterns in the structural input trigger NSFW outputs without any text trigger; on ImageNet with SD v1.5, ASR is 91% at 1% poisoning and 100% at 5% poisoning (Lapid et al., 7 Jul 2025). IMU-DM-CLIP applies CLIP-guided diffusion to synthesize trigger-bearing IMU sequences for backdooring human-activity recognition, reporting attack success rate over 80% with a 10% backdoor injection rate and 10% guided data fraction (Briston et al., 22 Jun 2026). BadGraph poisons text-guided latent diffusion for molecular graph generation so that textual triggers induce a target subgraph; less than 10% poisoning yields about 33–50% ASR depending on the dataset, and 24% poisoning yields over 80% ASR on three of four benchmarks (Ye et al., 23 Oct 2025). DiffPhysBA uses similarity-guided diffusion to generate realistic physical accessories as semantic backdoor triggers for person re-identification, reporting over 90% success in digital and physical domains and a +25.6% absolute gain in physical ASR over direct paste on the reported real-world test (Sun et al., 2024).

These results show that the conditional interfaces of diffusion systems—text, control maps, sensor prompts, or structural latent spaces—are also backdoor interfaces.

6. Evaluation, defenses, and unresolved issues

The literature evaluates guided diffusion attacks with a heterogeneous metric set. Adversarial-example papers emphasize ASR, classifier confidence, LPIPS, SSIM, PSNR, FID, NIMA, or task success rate (Guo et al., 2024, Medghalchi et al., 2024, Xia et al., 31 Jul 2025, Chen et al., 2024). Reconstruction papers report MS-SSIM, LPIPS, DINO similarity, PSNR, MSE, and runtime (Lei et al., 15 Sep 2025, Meng et al., 2024, Meng et al., 13 Nov 2025). Poisoning and backdoor papers add clean accuracy, certified accuracy, validity, novelty, diversity, or benign retrieval performance (Li et al., 2024, Han et al., 2024, Ye et al., 23 Oct 2025, Sun et al., 2024). This metric diversity reflects distinct threat models rather than inconsistency.

Several recurring trade-offs are explicit. Stronger adversarial or conditional guidance can improve attack efficacy while degrading realism or task utility (Guo et al., 2024, Dogoulis et al., 15 Jun 2025, Xia et al., 31 Jul 2025). Shorter triggers are often stealthier but less reliable than longer textual triggers (Ye et al., 23 Oct 2025). Moderate guidance weights and gradient clipping are repeatedly used to balance realism and attack strength (Lei et al., 15 Sep 2025, Meng et al., 13 Nov 2025). Some methods remain compute-intensive, especially at deep split layers or high resolution, and several papers note missing runtime ablations or incomplete physical validation (Guo et al., 2024, Lei et al., 15 Sep 2025, Meng et al., 2024).

Defenses also recur across domains. For adversarial generation, the literature mentions adversarial training, input transformations, diffusion-based purification, detector models, multi-view consistency checks, and temporal consistency in VR (Guo et al., 2024). For split inference and leakage, proposed defenses include DISCO channel pruning, NoPeek, token shuffling or dropping, noise injection, learned obfuscators, secure aggregation, cryptographic protection, and differential privacy, although the gradient-leakage papers show that moderate DP-style noise may remain insufficient (Lei et al., 15 Sep 2025, Meng et al., 13 Nov 2025, Meng et al., 2024). For poisoning and backdoors, suggested mitigations include trigger-aware audits, provenance verification, control-map sanitization, prompt filtering in embedding space, dataset hygiene, and checkpoint vetting (Li et al., 2024, Ma et al., 2024, Lapid et al., 7 Jul 2025, Han et al., 2024).

A final misconception is that diffusion models used as defenses are intrinsically trustworthy. DIFF2 explicitly shows that security-enhancing diffusion models can themselves become attack vectors (Li et al., 2024). More broadly, the surveyed works indicate that any mechanism that improves controllability in diffusion—classifier guidance, classifier-free guidance, prompt conditioning, ControlNet residuals, learned prompt embeddings, or latent priors—can also enlarge the adversarial surface. This suggests that future work will likely focus less on whether diffusion can be guided and more on how its guidance channels can be audited, regularized, or certified under adversarial pressure.