SafeDiffusion-R1: Safety for Diffusion Models

• The paper presents a novel post-training framework that integrates online reinforcement learning and a CLIP-based steering reward to suppress unsafe outputs without needing supervised safety data.
• It employs LoRA adapters and Group Relative Policy Optimization on Stable Diffusion v1.4 to mix benign and unsafe prompts and effectively reduce inappropriate content.
• Empirical results demonstrate significant reductions in inappropriate rates and nudity detections, while preserving compositional utility and overall model performance.

SafeDiffusion-R1 is a post-training framework for text-to-image diffusion models that combines online reinforcement learning with a CLIP-based steering reward to suppress unsafe generations without requiring supervised paired safety data or specialized safe/unsafe reward-model tuning. Instantiated on Stable Diffusion v1.4 with LoRA adapters on the UNet attention layers, it uses Group Relative Policy Optimization (GRPO) over mixtures of benign and unsafe prompts, including explicit unsafe prompts, to reduce inappropriate generations while preserving compositional utility. The reported results include reduction of overall inappropriate content from $48.9\%$ to $18.07\%$, reduction of I2P nudity detections from $646$ to $15$ in the Unsafe Anchor variant, and improvement of GenEval accuracy from $42.08\%$ to $47.83\%$ in the GenEval+Nudity setting (Kumar et al., 18 May 2026).

1. Safety objective and problem setting

SafeDiffusion-R1 is situated in the safety post-training of public text-to-image diffusion models, particularly models such as Stable Diffusion that inherit unsafe associations from large-scale web data. The target failure modes explicitly include nudity, sexual content, harassment, violence, and other inappropriate categories, with the paper emphasizing that unsafe generations can be triggered easily by explicit prompts. The method is designed against a specific set of limitations in prior work: supervised fine-tuning and concept erasure depend on curated paired data such as unsafe text–safe image pairs or safe/unsafe image pairs; offline reinforcement learning and supervised fine-tuning on synthetic data are static and can induce distribution mismatch and catastrophic forgetting; many RL pipelines depend on specialized reward models; and inference-time filters do not change the underlying model distribution and remain bypassable under direct model access (Kumar et al., 18 May 2026).

Within that problem setting, the framework advances three linked claims. First, optimization is on-policy: the model generates on the fly and learns from immediate feedback on both positive and negative prompts. Second, the reward is constructed geometrically from CLIP text embeddings rather than from a separately trained safety classifier. Third, the safety objective is not treated as pure suppression. The training curriculum mixes benign compositional prompts with unsafe prompts so that utility preservation becomes part of the optimization problem rather than a downstream constraint.

This design choice matters because catastrophic forgetting is treated not as an incidental side effect but as a central systems problem in safety alignment. SafeDiffusion-R1 therefore frames safe post-training as a distribution-tracking problem: the current policy should be updated using samples from its own current generative behavior, while preserving useful capabilities on benign prompts through explicit mixed-prompt training.

2. Model architecture and online post-training pipeline

The implementation uses Stable Diffusion v1.4 with a UNet backbone, and applies LoRA adapters of rank $r=4$ to the attention layers; the experimental setup specifies LoRA on the $Q/K/V/O$ projections of attention. In the policy interpretation adopted by the paper, the diffusion policy $\pi_\theta$ is the denoising network that defines transition distributions over latent states along the sampling trajectory conditioned on prompt $c$. The “actions” are denoising transitions, and parameter updates are obtained through GRPO using image-level rewards (Kumar et al., 18 May 2026).

The online loop has five stages. A batch of prompts is sampled by mixing benign compositional prompts with unsafe nudity prompts; unsafe prompts are included explicitly so that the model learns suppression from the prompts it is meant to resist. For each prompt, $18.07\%$0 images are generated with DDIM, and per-timestep log-probabilities are stored for later importance sampling. Reward computation then uses a CLIP-based steering mechanism: if a prompt is detected as unsafe, its text embedding is steered toward a safety direction before alignment with the image embedding is scored; otherwise the unmodified prompt embedding is used. Advantages are normalized within each prompt group, and the model is updated with a PPO-style clipped objective using per-timestep importance ratios and a KL penalty. The system’s explicit anti-forgetting mechanisms are on-policy training, group-relative normalization, KL regularization, and prompt mixing.

The training configuration reported for the main setting uses AdamW with $18.07\%$1, $18.07\%$2, $18.07\%$3, a constant learning rate of $18.07\%$4, batch size $18.07\%$5 prompts per GPU, $18.07\%$6 generations per prompt, DDIM with $18.07\%$7 steps, classifier-free guidance scale $18.07\%$8, resolution $18.07\%$9, clipping parameter $646$0, KL coefficient $646$1, gradient clipping at $646$2, steering strength $646$3, and training for $646$4 epochs. The reported compute budget is approximately $646$5 GPU hours on $646$6 AMD MI210 (64GB) with bfloat16 mixed precision, and the timestep training fraction is $646$7, corresponding to training on the last $646$8 DDIM steps.

3. GRPO in diffusion latent space

SafeDiffusion-R1 adopts Group Relative Policy Optimization rather than standard PPO directly. The paper retains PPO’s clipped-surrogate logic but adapts it to diffusion trajectories, where the relevant probabilities are transition densities over latent states rather than discrete-action probabilities. The one-step transition log-probability is written as

$646$9

and the total trajectory log-probability is the sum over diffusion timesteps:

$15$0

For a prompt $15$1 with $15$2 generated samples and rewards $15$3, the group-relative advantage is

$15$4

with $15$5 the group standard deviation. This normalization removes dependence on absolute reward scale and makes updates comparative within each prompt group. That property is central because the training batches intentionally mix heterogeneous prompt types, including benign compositional prompts and explicit unsafe prompts.

The importance ratio is defined per timestep,

$15$6

and the clipped GRPO objective is

$15$7

An approximate KL regularization term is added, with $15$8, to constrain drift from the previous policy and avoid catastrophic forgetting (Kumar et al., 18 May 2026).

The distinction from ordinary PPO is therefore structural rather than cosmetic. Advantages are normalized within prompt-conditioned sample groups, and importance ratios are computed from Gaussian diffusion transitions across DDIM timesteps. This suggests that SafeDiffusion-R1 should be understood less as “RL on final images” than as trajectory-aware policy optimization over the denoising process.

4. CLIP-based steering reward and embedding geometry

The steering reward uses HPSv2, a CLIP-based model, for both text and image embeddings. Let $15$9 and $42.08\%$0 denote the $42.08\%$1-normalized image and text embeddings. From a small set of safe anchor texts $42.08\%$2 and unsafe anchor texts $42.08\%$3, the safety direction is constructed as

$42.08\%$4

$42.08\%$5

Prompt safety is detected by projection onto that direction:

$42.08\%$6

If $42.08\%$7, the prompt is treated as unsafe, and the text embedding is steered toward the safety direction:

$42.08\%$8

Otherwise the reward is the ordinary image–text alignment,

$42.08\%$9

The key methodological point is that unsafe prompts are not censored or excluded from training. Instead, the alignment target is modified geometrically. The paper characterizes this as exploiting an inherent property of CLIP embeddings: steering text representations toward positive safety directions and away from negative ones in embedding space. Because the safety direction is computed from anchor phrases once, no specialized safe/unsafe reward model is trained or tuned (Kumar et al., 18 May 2026).

The ablations clarify why this matters. The “Unsafe Anchor Only” variant performs worse overall on inappropriate content rate, at $47.83\%$0, than the default steering method’s $47.83\%$1. In a separate reward-design ablation, the neg-only $47.83\%$2 condition attains MeanUnsafe $47.83\%$3 but collapses utility, with CLIP-T $47.83\%$4 and FID $47.83\%$5, whereas the steering reward yields CLIP-T $47.83\%$6, FID $47.83\%$7, and MeanUnsafe $47.83\%$8. A plausible implication is that anchor-based positive–negative geometry functions as a utility-preserving safety target, whereas penalty-only designs overfit suppression.

5. Experimental regime and empirical performance

Training uses approximately $47.83\%$9 unsafe nudity prompts generated via Grok and more than $r=4$0 compositional prompts from FlowGRPO; one scaling experiment additionally uses more than $r=4$1 safety-focused prompts from SafetyDPO. Evaluation covers I2P for nudity detection and inappropriate content proportions, GenEval for compositional utility, and a COCO-3k split for CLIP-T and FID. Harm categories are Hate, Harassment, Violence, Self-harm, Sexual, Shocking, and Illegal Activity, with “out-of-domain” referring to categories such as Hate and Violence that were not directly targeted during primarily nudity-focused training. NudeNet with threshold $r=4$2 is used for nudity detection, the Q16 classifier for inappropriate content rates, and HPSv2 (ViT-H-14) for reward embeddings (Kumar et al., 18 May 2026).

Metric	SD v1.4	SafeDiffusion-R1
Overall inappropriate rate	48.9%	18.07%
I2P nudity detections	646	31 (default), 15 (Unsafe Anchor)
GenEval overall accuracy	42.08%	47.83% (GenEval+Nudity)

The inappropriate-content results are reported across seven categories, with SafeDiffusion-R1 achieving Hate $r=4$3, Harassment $r=4$4, Violence $r=4$5, Self-harm $r=4$6, Sexual $r=4$7, Shocking $r=4$8, and Illegal activity $r=4$9. The paper describes this as strong out-of-domain generalization because training is primarily nudity-focused while evaluation spans all seven harm categories. On GenEval, the best reported configuration is SafeDiffusion-R1 trained on GenEval+Nudity prompts, with overall accuracy $Q/K/V/O$0; task-wise values are two_object $Q/K/V/O$1, colors $Q/K/V/O$2, position $Q/K/V/O$3, color_attr $Q/K/V/O$4, single_object $Q/K/V/O$5, and counting $Q/K/V/O$6. The Nudity-only variant still exceeds the SD v1.4 baseline at $Q/K/V/O$7.

The CLIP-T/FID analysis is more mixed. On I2P nudity prompts and COCO-3K, SD v1.4 has CLIP-T $Q/K/V/O$8 and FID $Q/K/V/O$9, while SafeDiffusion-R1 default has CLIP-T $\pi_\theta$0 and FID $\pi_\theta$1. The Negative Anchor variant has CLIP-T $\pi_\theta$2 and FID $\pi_\theta$3. The paper’s interpretation is that FID increases because training occurs on a synthetic post-training distribution, while CLIP-T remains near baseline and qualitative results preserve realism and structure. Scheduler ablations further report that, with steering, unsafe scores converge to near-zero across nine schedulers by epoch $\pi_\theta$4, indicating robustness to sampler choice.

6. Relation to adjacent lines of safety research

SafeDiffusion-R1 addresses safety post-training for text-to-image generation, which distinguishes it from other diffusion-safety paradigms. “SafeDiffuser” addresses safe planning with diffusion probabilistic models by embedding control barrier functions and finite-time diffusion invariance into trajectory denoising for tasks such as maze planning, locomotion, and manipulation, rather than post-training safety alignment of text-to-image models (Xiao et al., 2023). The conceptual overlap is the use of the denoising process as the site of safety control; the technical objects, however, are different: SafeDiffusion-R1 operates on prompt-conditioned image synthesis with RL and embedding-space rewards, whereas SafeDiffuser operates on trajectory generation with barrier-constrained QP corrections.

A second neighboring line concerns data sanitization rather than generative alignment. “Certified Robustness to Clean-Label Poisoning Using Diffusion Denoising” uses off-the-shelf diffusion denoising to sanitize training data and derive a randomized-smoothing-style certificate against $\pi_\theta$5 clean-label poisoning, which is a training-data defense rather than a post-training behavioral alignment method (Hong et al., 2024). Another distinct safety layer is infrastructure hardening: “DiffusionHijack” studies supply-chain PRNG backdoors that deterministically control diffusion outputs without modifying model weights, and proposes QRNG-based mitigation (You et al., 13 May 2026). These works address different threat models from unsafe semantic generation.

The name may also invite confusion with non-diffusion alignment systems. “RealSafe-R1” is a safety-aligned DeepSeek-R1 family for reasoning models trained on $\pi_\theta$6 safety-aware reasoning trajectories, targeting refusal behavior under harmful and jailbreak prompts rather than image-generation safety (Zhang et al., 14 Apr 2025). SafeDiffusion-R1 is therefore best located within diffusion-model post-training and not within safety alignment of large reasoning models or certified control in diffusion planning.

7. Limitations, ethics, and reproducibility

The paper identifies several limits. Because the method depends on CLIP/HPSv2 embeddings, it inherits biases in that embedding geometry and may not align perfectly with human norms across cultures. NudeNet and Q16 are only proxy detectors and can produce false positives or false negatives. Over-sanitization remains a risk: the paper notes that aggressive penalties or anchor-only designs can harm generalization or creativity, and recommends careful monitoring. The framework is parameter-efficient, but on-policy generation plus GRPO updates still imposes substantial compute cost. Generalization beyond nudity is strong in the reported experiments, but further categories may require tailored anchors (Kumar et al., 18 May 2026).

The ethical discussion follows directly from the training recipe. Unsafe prompts are deliberately included during online learning, because the method learns suppression by training on explicit unsafe inputs rather than by blocking them. The paper therefore recommends responsible handling of such datasets, avoiding redistribution of explicit content, and adherence to institutional and legal guidelines for sensitive data.

Reproducibility is comparatively explicit. A public repository is provided at https://github.com/MAXNORM8650/SafeDiffusion-R1. The documented setup specifies Stable Diffusion v1.4, HPSv2 embeddings, nudity prompts of approximately $\pi_\theta$7, more than $\pi_\theta$8 FlowGRPO compositional prompts, optional SafetyDPO prompts for scaling, LoRA rank $\pi_\theta$9, AdamW with learning rate $c$0, batch size $c$1, $c$2 generations per prompt, DDIM $c$3 steps, CFG $c$4, resolution $c$5, GRPO clipping $c$6, KL coefficient $c$7, gradient clip $c$8, and steering strength $c$9. Checkpoints are saved every $18.07\%$00 epochs and evaluated on I2P and GenEval. In that sense, SafeDiffusion-R1 presents a relatively complete recipe for post-training alignment of public diffusion backbones when retraining from scratch, curated paired datasets, and reward-model tuning are all undesirable.