---
title: In-Generation Detection (IGD) in Diffusion Models
url: https://www.emergentmind.com/topics/in-generation-detection-igd
type: topic
---

# In-Generation Detection (IGD) in Diffusion Models

Searching arXiv for recent papers on in-generation detection in generative models.
I’ll look up recent arXiv papers related to in-generation detection, especially diffusion-model safety detection.
In-Generation Detection (IGD) denotes a safety-monitoring paradigm for generative models in which detection is performed during the generation process rather than before generation from prompts or after generation from completed outputs. In the recent literature represented here, IGD is formulated for diffusion-based text-to-image models, where the detector inspects model-internal states at intermediate denoising steps to identify Not-Safe-for-Work (NSFW) intent or content early enough to permit intervention, including aborting the generation before completion [2508.03006][2604.07879].

## 1. Conceptual scope and placement in the safety pipeline

IGD was introduced as a third operating point in the moderation pipeline alongside pre-detection and post-detection. Pre-detection filters prompts before image generation; post-detection analyzes output images after generation; IGD instead monitors generation-time internals during denoising itself [2508.03006]. This distinction is central because the in-generation phase of diffusion models was described as largely unexplored for NSFW detection, despite diffusion models’ iterative structure providing internal signals before a final image is synthesized [2508.03006].

The motivation is partly operational and partly epistemic. Operationally, early detection can save computation by terminating unsafe generations before the full denoising trajectory finishes. Epistemically, prompt-only moderation faces a gap between prompt safety and image safety, while final-image moderation cannot act until the unsafe image already exists [2604.07879]. IGD therefore treats the denoising trajectory itself as the object of analysis.

A common misconception is to equate IGD with prompt classification. The cited work explicitly separates the two. In these papers, the critical observation is that internal denoising signals may retain the semantic intent of unsafe generation even when prompts are adversarially rewritten to evade text-based filters [2508.03006]. This suggests that safety-relevant evidence can be more stable in model-internal trajectories than in surface text alone.

## 2. Diffusion-model basis and internal observables

The mathematical basis for IGD in diffusion models follows the standard forward and reverse processes. In the 2025 IGD formulation, if $x^0$ is the original image latent, $x^t$ the noisy latent at timestep $t$, and $c$ the prompt encoding, then the forward process is
$$
x^t = \sqrt{\bar{\alpha}^t} x^0 + \sqrt{1 - \bar{\alpha}^t} \, \epsilon, \quad \epsilon \sim \mathcal{N}(0, I),
$$
and the reverse process predicts noise via
$$
\epsilon_t = \epsilon_\theta(x^{t+1}, t, c).
$$
IGD leverages the predicted noise $\epsilon_t$ at a particular early timestep as a semantic feature for NSFW/SFW discrimination [2508.03006].

The core empirical intuition is that predicted noise is conditioned on the text prompt and reflects the intended semantics of the prompt even before the actual image is synthesized. The paper reports that predicted noise vectors for NSFW prompts, including adversarial prompts, form coherent, separable clusters in feature space, whereas adversarial prompts can fool prompt embeddings but not the predicted noise patterns, which remain consistent with their unsafe intent [2508.03006].

FlowGuard addresses a different internal observable. Rather than classifying predicted noise directly, it inspects intermediate denoising steps in latent diffusion models by approximately decoding intermediate latents into image-like reconstructions [2604.07879]. This is motivated by a specific difficulty: in latent diffusion, early-stage noise obscures visual signals, and raw latents are heterogeneous across models, making universal raw-latent classifiers difficult [2604.07879].

## 3. Predicted-noise IGD

The IGD method introduced in “Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Models” uses a binary classifier $f_\phi$ trained on predicted noise:
$$
y = f_\phi(\epsilon_t), \qquad y \in \{0, 1\},
$$
where $y=1$ denotes NSFW [2508.03006]. The classifier is a 5-layer MLP. Training uses pairs of prompt and label, with $\epsilon_t$ extracted from naive NSFW and SFW prompts; at inference, the detector monitors predicted noise at early timesteps and triggers early intervention if NSFW intent is detected [2508.03006].

Its pipeline is described in four stages: prompt encoding, diffusion generation, classification, and intervention. A text encoder produces the prompt embedding $c$; during denoising, the model extracts the predicted noise vector at a preselected timestep, such as step 5; the MLP classifies the noise as NSFW or SFW; and generation is aborted or allowed based on the classifier output [2508.03006]. The detector is described as lightweight and as introducing negligible overhead because it uses early, partial denoising steps.

The same study reports that IGD can be extended from binary NSFW/SFW detection to multi-class detection across seven NSFW types plus SFW, and that it maintains similar high performance across Stable Diffusion v1.4, v1.5, and v2.1 [2508.03006]. It therefore frames IGD not merely as a binary safety gate but as a general internal-state classification mechanism over diffusion trajectories.

## 4. FlowGuard and linear latent decoding

FlowGuard generalizes the in-generation safety-detection idea to a cross-model setting by separating model-specific latent decoding from model-agnostic safety classification [2604.07879]. Its first component is a linear latent decoder:
$$
\hat{x}_t = D_{\mathrm{lin}}(z_t) = W z_t + b,
$$
learned from latent-image pairs by minimizing
$$
\min_{W, b} \ \frac{1}{N} \sum_{i=1}^N \Vert W z_i + b - D_{\mathrm{VAE}}(z_i) \Vert_2^2.
$$
The linear decoder is trained independently for each backbone [2604.07879].

To suppress noise-dominated detail in these reconstructions, FlowGuard applies a 2D Fourier low-pass filter:
$$
\begin{align*}
\mathscr{F}_t &= \mathscr{F}(\hat{x}_t) \\
\tilde{\mathscr{F}}_t &= M_r \odot \mathscr{F}_t \\
\tilde{x}_t &= \mathscr{F}^{-1}(\tilde{\mathscr{F}}_t).
\end{align*}
$$
The paper reports that low-pass filtering, especially with $r = 0.2$, improves detection accuracy and F1 on noisy steps [2604.07879].

The third component is curriculum learning. Rather than training immediately on early noisy states, FlowGuard starts from late-step, nearly clean images and gradually moves to earlier timesteps. It adds a temporal consistency loss
$$
\mathcal{L}_\mathrm{consis} = \mathbb{E}_{t, t'}\left[ \| g(\tilde{x}_t) - g(\tilde{x}_{t'}) \|^2_2 \right]
$$
and optimizes
$$
\mathcal{L} = \mathcal{L}_\mathrm{cls} + \lambda \mathcal{L}_\mathrm{consis}.
$$
At inference, it evaluates a small set of early steps, such as steps 20–30, and exits early if any predicted NSFW probability exceeds a threshold $\delta$ [2604.07879].

This architecture is explicitly cross-model. The same NSFW classifier is reused across many text-to-image backbones, while only the lightweight decoder is retrained for a new model. The implementation summary specifies a linear decoder per model, a ViT-B/16 backbone for classification, and a custom cross-model dataset with approximately 4000 diverse prompts and thousands of prompt-latent-image-label tuples across nine backbones [2604.07879].

## 5. Empirical performance, adversarial robustness, and efficiency

The two principal IGD-style systems differ in signal choice but converge on the same claim: intermediate denoising states support effective NSFW detection substantially earlier than final-image moderation.

| System | Internal signal | Reported results |
|---|---|---|
| IGD | Predicted noise $\epsilon_t$ | 91.32% average detection accuracy in the abstract; 92.45% accuracy, 96.78% AUROC, 16.78% FPR@TPR95 for overall binary detection |
| FlowGuard | Linearly decoded intermediate latents with LPF | Over 30% F1 improvement over existing methods; peak GPU memory reduced by over 97%; projection time reduced from 8.1 s to 0.2 s |

For the predicted-noise IGD method, experiments were conducted on seven NSFW categories—sexual, violence, self-harm, harassment, hate, shocking, and illegal activity—using I2P for NSFW prompts and MSCOCO for clean prompts [2508.03006]. The paper reports 89.5% accuracy for sexual, 89.0% for violence, 89.5% for self-harm, 90.5% for harassment, 95.7% for hate, 90.0% for shocking, and 89.0% for illegal activity, with a naive-prompt average of 90.46%. For adversarial prompts generated by I2P-sexual, MMA-Diffusion, Ring-A-Bell, SneakyPrompt, P4D, and DiffZOO, it reports 93.77% accuracy and 98% AUROC. The same study states that IGD outperforms seven baseline methods, including NSFW-text-classifier, Detoxify, OpenAI Moderation API, Aliyun/Azure APIs, Latent Guard, and GuardT2I, and reports attack success rates of 9.8% on naive prompts and 3.9% on adversarial prompts when compared with concept-erasing defenses [2508.03006].

FlowGuard reports in-generation detection performance at step 20 of 50 with F1 in the range 0.86–0.90 for in-distribution models and 0.73–0.88 for out-of-distribution models, whereas the best listed baselines—LlavaGuard, Falconsai, and Qwen3-VL—are reported at 0.66–0.73 in-distribution and much lower out-of-distribution [2604.07879]. Its efficiency claims are equally prominent: compared with standard VAE decoding, it reports slashing peak GPU memory demand by over 97% and reducing projection time from 8.1 seconds to 0.2 seconds [2604.07879]. These measurements support the practical premise of IGD: safety screening can be inserted into denoising with lower marginal cost than full post-generation moderation.

## 6. Limitations, unresolved questions, and acronym ambiguity

The current IGD literature also states clear limitations. The predicted-noise method notes that discrimination among semantically close NSFW subtypes may require improvement, that the classifier must be trained on predicted noise from the actual generation model or a related family, and that it is unknown whether early IGD will always suffice if an adversary manipulates the denoising trajectory at later steps [2508.03006]. FlowGuard identifies strong Gaussian noise in intermediate latents, latent space heterogeneity across models, and limited cross-model latent datasets as core obstacles for in-generation detection in diffusion systems [2604.07879].

A second source of confusion is terminological rather than methodological: the acronym “IGD” is heavily overloaded in the broader technical literature. In recommendation, “IGD” denotes “Information Gain-based Decisiveness-aware Token handling” for LLMs [2506.13229]. In one-class classification, it denotes the “Interpolated Gaussian Descriptor” [2101.10043]. In graphene transport, “IGD” denotes “individual group delay” [1304.5589]. None of these is synonymous with In-Generation Detection. In current generative-model safety work, the term refers specifically to generation-time moderation of intermediate states, especially in diffusion-based text-to-image models [2508.03006][2604.07879].

Taken together, these studies define IGD as a model-internal, temporally localized detection regime. One line of work uses predicted noise directly; another reconstructs intermediate latents into filtered image-like representations. Both reject the assumption that safety must be inferred only from prompts or final outputs. The immediate research direction suggested by these results is not a replacement of pre- or post-detection, but a reallocation of moderation effort toward the denoising trajectory itself, where semantic intent may already be measurable before image synthesis completes.

Source: https://www.emergentmind.com/topics/in-generation-detection-igd