Adaptive Windowed Negative Prompting

• Adaptive Windowed Negative Prompting is a method that dynamically introduces windowed negative prompts to suppress unwanted content in diffusion-based image generation.
• It leverages a Vision-Language Model to assess intermediate image states and update negative guidance, ensuring targeted content control and preserving image integrity.
• Empirical evaluations demonstrate significant improvements in safety metrics and image quality, outperforming traditional static negative prompting approaches.

Adaptive Windowed Negative Prompting is a methodology for enhancing the safety and controllability of diffusion-based image generation by dynamically introducing context-specific negative guidance at selected denoising steps. By leveraging a Vision-LLM (VLM) to generate negative prompts in response to the state of intermediate reconstruction, this approach enables targeted suppression of emergent unwanted content while avoiding excessive suppression after an undesired concept has been removed. This paradigm offers a significant advance over traditional, static negative prompting methods by adaptively responding to the temporal evolution of generated content and optimizing the trade-off between content safety and text-image alignment fidelity (Chang et al., 30 Oct 2025).

1. Key Principles and Rationale

Adaptive Windowed Negative Prompting is motivated by two limitations in conventional negative prompting: the use of fixed negative prompts throughout the generation process, and the inability to adapt to the changing content of partially denoised samples. In this approach, negative prompts are not static, but are periodically updated at a predefined set of denoising timesteps ("windows"). At each window, the current intermediate prediction of the clean image is analyzed by a VLM, which outputs concise negative concepts reflective of the present content. The negative prompt is held fixed in between these windows.

This adaptive process enables the model to suppress only the specific unwanted content present at each stage, preventing both under- and over-suppression. Once a problematic concept is eliminated, subsequent negative prompts omit it, thereby minimizing collateral impact on image fidelity.

2. Algorithmic Workflow

The method, exemplified in Dynamic VLM-Guided Negative Prompting (VL-DNP), can be outlined as follows:

1. Denoising with Positive Prompt: The diffusion process begins with standard classifier-free guidance (CFG) using a positive text prompt $c^+$.
2. Windowed Clean Prediction: At each window timestep $t_i$, the model computes an estimate $\hat x_0^{(i)}$ of the clean image via

$$\hat x_0^{(i)} = \frac{x_{t_i} - \sqrt{1-\bar\alpha_{t_i}}\, s_{\theta,\rm cfg}(x_{t_i}, t_i \mid c^+)}{\sqrt{\bar\alpha_{t_i}}}, \quad \bar\alpha_{t_i} = \prod_{s=1}^{t_i}(1-\beta_s).$$

where $s_{\theta,\rm cfg}$ is the CFG score.

1. VLM Query: The intermediate image $\hat x_0^{(i)}$ and a set of demonstration pairs $\mathcal{D}$ are provided to the VLM $\Theta$, which outputs a dynamic negative prompt $c^-_{t_i}$:

$$c^-_{t_i} = \Theta\left(\hat x_0^{(i)},\,\mathcal{D}\right).$$

2. Adaptive Guidance Update: For the following denoising steps $t \in (t_i, t_{i+1})$, classifier-free guidance utilizes $c^-_{t_i}$. The overall guided score function is:

$$\tilde s_\theta(x_t, t \mid c^+, c^-_{t_i}) = \nabla_{x_t}\log p(x_t \mid c^+) + \omega_{\rm pos} \left( \nabla_{x_t} \log p(x_t \mid c^+) - \nabla_{x_t} \log p(x_t) \right) - \omega_{\rm neg} \left( \nabla_{x_t} \log p(x_t \mid c^-_{t_i}) - \nabla_{x_t} \log p(x_t) \right).$$

3. Negative Embedding Integration: The VLM’s generated textual descriptors are embedded using the diffusion model’s CLIP text encoder before being incorporated into the classifier-free guidance.

The process is repeated at each selected window until the final output image is produced.

3. Model Architecture and Implementation

The VLM backbone, used to extract negative concepts, is based on Qwen2.5-VL-7B-Instruct. Prompts to the VLM are formulated as: “Identify any potentially inappropriate or unwanted elements in the following image and list them as concise negative concepts.” Demonstration sets $\mathcal{D}$ comprise 5–10 image/text pairs typifying "unsafe" content (e.g., “blurry person” $\rightarrow$ “exposed genitals”), to encourage precision and context-specificity in VLM outputs.

The adaptive guidance is implemented within Stable Diffusion v1.4, with 50 denoising steps and DPM-Solver++ as the numerical integration method. Window timesteps are selected as $$\mathcal{T} = \{45, 44, 43, 41, 38, 34, 29, 23, 16, 8\}$$. Generation settings for the VLM include temperature $T=0.3$, a maximum of 16 tokens per output, and top-p=0.9 sampling.

Pseudocode for the inference routine demonstrates core steps: prediction, VLM query, guidance, and stepwise image update.

4. Experimental Evaluation and Metrics

Empirical assessment uses established metrics including Attack Success Rate (ASR), Toxic Rate (TR), CLIP score, and Fréchet Inception Distance (FID). Comparative experiments examine static negative prompting (with a fixed negative prompt), no negative prompting, and VL-DNP at varying negative guidance strengths ($\omega_{\rm neg}$).

Method	Ring-ASR ↓	Ring-TR ↓	COCO-CLIP ↑	COCO-FID ↓
No neg	0.958	0.961	0.312	–
Static $\omega_{\rm neg}=15$	0.000	0.028	0.296	136.1
VL-DNP $\omega_{\rm neg}=15$	0.084	0.147	0.311	12.9
VL-DNP $\omega_{\rm neg}=20$	0.011	0.081	0.311	15.3

VL-DNP consistently attains lower ASR and TR at comparable levels of CLIP alignment, compared to static negative prompting. Notably, VL-DNP reduces FID from 136.1 in the static setting to 12.9–15.3, indicating substantial preservation of image quality even at high negative guidance strengths. Pareto frontiers for various datasets demonstrate superior safety–fidelity trade-offs.

Ablation on $\omega_{\rm neg}$ indicates increasing negative guidance strength enhances safety metrics, with only modest degradation in FID when adaptivity is used.

5. Guidance Scale, Window Selection, and Best Practices

Best practices for configuration emerge from systematic ablation:

• Window Frequency and Placement: 8–12 windows, uniformly distributed over the denoising process, balance computational overhead and safety coverage. Early windows (high $t$) target coarse artifact removal, while late windows suppress persistent details.
• Update Frequency: Querying the VLM every 2–5 steps is sufficient for most applications, with decreased interval yielding marginal increases in safety. Unchanged negative concepts can be cached; the VLM can indicate when no unwanted content remains.
• Guidance Strengths: A positive guidance scale $\omega_{\rm pos}=7.5$ is typical, while negative guidance $\omega_{\rm neg}\in[15, 25]$ yields robust content suppression. For maximum fidelity, lower values may be preferable at a minor safety cost.
• Prompt Specificity: Concept-specific negatives (e.g., “male breast”, “bare buttocks”) are preferred over generic terms for precise suppression.
• Overshooting Prevention: The dynamic mechanism prevents unnecessary and accumulative suppression, as unwanted descriptors are dropped from negative prompts immediately upon removal.

6. Context, Implications, and Related Directions

Adaptive Windowed Negative Prompting represents a training-free, plug-and-play technique for context-aware safety enhancement in diffusion models. By operationalizing negative prompt adaptation through VLMs, it permits granular and targeted suppression across the denoising trajectory, mitigating the fidelity degradation associated with static high-magnitude negative conditioning.

This method integrates with existing CFG pipelines and relies on low-overhead inference adjustments, rather than model retraining or architecture modification. Its modular framework suggests extensibility to diverse base models, VLMs, and use cases where compliance, safety, or content control is critical.

A plausible implication is that further refinements in VLM capability, window scheduling, or guidance adaptivity may enable fine-grained alignment for safety-critical generative tasks with minimal intervention costs. Such adaptive techniques may inform broader applications in controllable generation, concept erasure, and real-time interactive prompting within diffusion frameworks (Chang et al., 30 Oct 2025).