SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Images

Abstract: Diffusion models trained on noisy datasets often reproduce high-frequency training artifacts, significantly degrading generation quality. To address this, we propose SCoRe (Spectral Cutoff Regeneration), a training-free, generation-time spectral regeneration method for clean image generation from diffusion models trained on noisy images. Leveraging the spectral bias of diffusion models, which infer high-frequency details from low-frequency cues, SCoRe suppresses corrupted high-frequency components of a generated image via a frequency cutoff and regenerates them via SDEdit. Crucially, we derive a theoretical mapping between the cutoff frequency and the SDEdit initialization timestep based on Radially Averaged Power Spectral Density (RAPSD), which prevents excessive noise injection during regeneration. Experiments on synthetic (CIFAR-10) and real-world (SIDD) noisy datasets demonstrate that SCoRe substantially outperforms post-processing and noise-robust baselines, restoring samples closer to clean image distributions without any retraining or fine-tuning.

• The paper presents SCoRe as a training-free method that uses spectral cutoff and SDEdit to regenerate clean images from noisy-trained diffusion models.
• It leverages the inherent spectral bias of diffusion models to suppress high-frequency noise while preserving low-frequency, semantic image structures.
• Experimental evaluations on CIFAR-10 and SIDD demonstrate that SCoRe significantly outperforms baseline denoising methods with lower FID scores and improved visual quality.

SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Images

Motivation and Problem Statement

Diffusion models have become a foundational approach in generative modeling for high-fidelity image synthesis. However, a major practical challenge arises when the training dataset is contaminated with noisy images, which is common in real-world data pipelines aggregating from heterogeneous sources. In such settings, diffusion models tend to reproduce high-frequency artifacts learned from the noisy data distribution, severely degrading generation quality. Standard post-sampling denoising techniques and noise-robust training frameworks fail to address this challenge effectively—either sacrificing fine structure or requiring prohibitive retraining. This paper introduces SCoRe (Spectral Cutoff Regeneration), a training-free, generation-time technique for recovering clean image generations from diffusion models trained on noisy datasets (2604.09436).

Figure 1: Impact of a dataset containing noisy training images on the generated results. Images in red boxes are clean examples; all others are noisy.

Methodology: Spectral Cutoff Regeneration (SCoRe)

Spectral Perspective on Diffusion Models

Diffusion models exhibit a pronounced spectral bias: semantic, low-frequency image structure is prioritized, while high-frequency content, including noise, is inferred conditionally later in the generative process. Empirically and theoretically, most real-world noise manifests in high-frequency bands.

Figure 2: (a) Frequency dynamics through the diffusion/reverse process. (b) SCoRe—suppression of high-frequency content via spectral cutoff, followed by SDEdit-based regeneration.

SCoRe Algorithm

SCoRe leverages this spectral property via a two-stage process, entirely at sample generation time:

1. Spectral Cutoff: The generated image undergoes a frequency-domain operation where all high-frequency components above a cutoff $f_\text{cutoff}$ are zeroed out, producing a low-pass filtered image that suppresses noise and artifacts.
2. Regeneration via SDEdit: SDEdit, a diffusion-based editing process, is initialized at a specific diffusion timestep $t'$, injecting controlled stochasticity to regenerate the high-frequency details. The choice of $t'$ is analytically derived to correspond with $f_\text{cutoff}$, using Radially Averaged Power Spectral Density (RAPSD) to avoid excessive noise reinjection and to tightly couple the amount of regeneration with the frequency boundary. This mapping ensures only frequencies above $f_\text{cutoff}$ are synthesized under the model prior.

   Figure 3: (a) Standard diffusion sampling produces noisy images. (b) SDEdit alone fails to suppress inherited noise. (c) SCoRe eradicates high-frequency noise through spectral cutoff + regeneration, yielding cleaner images.

Unlike classical or learning-based post-processing, SCoRe is agnostic to the exact form of corruption and does not require retraining, explicit noise modeling, or knowledge of the noise distribution.

Experimental Evaluation

Synthetic Noise: CIFAR-10 Benchmark

The primary evaluation uses CIFAR-10 with artificially structured corruption: 90% of the training set is contaminated via Gaussian, Poisson, or mixed noise, with only 10% clean samples retained.

Quantitatively, SCoRe achieves substantially lower FID scores than all baselines, including bilateral filtering, Noise2Void, SDEdit, and NR-GAN. Notably, FID improvements are robust across a range of cutoff frequencies, outperforming both learning-based robust models and handcrafted denoisers.

Figure 4: Comparison of generated images (under Gaussian noise): (a) training set, (b) standard diffusion output, (c) bilateral filter, (d) Noise2Void, (e) NR-GAN, (f) SCoRe.

Qualitative analysis reveals that SCoRe restores object fidelity and texture without introducing smoothing artifacts typical of filtering approaches. Spectral analysis via RAPSD demonstrates that SCoRe reconstructions closely match ground truth power spectra in mid-to-high frequencies, verifying the effectiveness of frequency-selective regeneration.

Figure 5: RAPSD comparison—SCoRe results maintain high-frequency power close to ground-truth while suppressing noise-laden artifacts.

The model demonstrates exceptional resilience as the ratio of noisy images is increased from 10% to 90%, maintaining stable, low FID in contrast to the steady degradation observed in standard diffusion models.

Figure 6: SCoRe maintains low FID regardless of noisy training data proportion, in stark contrast with the standard approach.

Real-World Noise: SIDD Dataset

On the SIDD dataset, encompassing realistic sensor and environmental noise, SCoRe again yields superior FID scores and visual quality over all baselines, including denoisers and robust GAN alternatives. Neither bilateral filtering nor NR-GAN demonstrates adequate generalization to complex, unknown noise distributions, confirming the need for SCoRe's frequency-domain approach.

Figure 7: Real-world results—SCoRe removes complex imaging noise and accurately restores structure and fine details.

Theoretical and Practical Implications

SCoRe's success affirms the potential of spectral manipulation and reverse-diffusion regeneration for sample-time correction—without retraining, explicit noise modeling, or privileged information about data corruption. The analytical RAPSD-based mapping between cutoff frequency and regeneration timestep solves the hitherto open problem of reconciling frequency-domain interventions with the discrete stochastic process of diffusion sampling.

Practically, SCoRe is deployable with any pretrained diffusion model and is robust to variation in corruption types and levels, as well as to cutoff hyperparameters within a reasonable range. This positions SCoRe as a highly pragmatic approach for real-world generative applications where large-scale clean data is not feasible.

On the theoretical front, this work strengthens the spectral perspective on deep generative modeling, inviting future avenues that more directly integrate spectral priors, adaptive cutoff scheduling, or spatially localized frequency interventions. Extending SCoRe to video, multimodal, or non-Cartesian data domains, as well as integrating with prompt-guided or conditional generative processes, are promising directions.

Conclusion

SCoRe introduces a principled, training-free framework for clean image generation from diffusion models compromised by noisy data. By algorithmically mapping a frequency cutoff to a precise SDEdit regeneration schedule, SCoRe outperforms traditional post-processing, noise-robust adversarial methods, and naive diffusion editing across synthetic and real-world benchmarks. The paper evidences both high quantitative gains (e.g., FID) and strong perceptual improvements, underscoring the efficacy and generality of spectral regeneration for generative image restoration (2604.09436).