Blind-Spot Guided Diffusion

• Blind-spot guided diffusion is a self-supervised image restoration framework that fuses BSN structural priors with diffusion-based denoising without relying on paired clean data.
• It employs a dual-branch architecture where the BSN branch enforces spatial independence to suppress artifacts while the UNet branch preserves fine local details.
• State-of-the-art performance on benchmarks like SIDD and DND highlights its practical impact for real-world applications such as low-light and smartphone photography.

Blind-spot guided diffusion is a framework that integrates blind-spot neural network principles with denoising diffusion probabilistic models to address the challenges of real-world image restoration, particularly in the absence of clean training data. The approach leverages the context-aware yet spatially independent filtering of blind-spot networks to deliver guidance during the generative denoising process of a diffusion model, aiming to combine the benefits of structural regularization and fine-detail reconstruction.

1. Definition and Fundamental Principles

Blind-spot guided diffusion refers to a self-supervised restoration paradigm in which a diffusion model is guided by the output of a blind-spot network (BSN) during its sampling process, without the necessity of paired clean/noisy training data. The core principle is to generate a candidate restoration (the "semi-clean" image) via a BSN, which predicts each pixel's value exclusively from its neighbors, deliberately omitting the central (and potentially corrupted) input pixel. This BSN output, which avoids learning a degenerate identity mapping, is used to guide the denoising diffusion process, which otherwise operates in an unconstrained generative fashion.

This dual-branch system is motivated by two complementary factors:

• The blind-spot branch enforces structural priors and suppresses trivial copying of noise.
• The conventional diffusion branch, often realized as a UNet, preserves local detail and models the full, potentially correlated, noise distribution in the observed image (Cheng et al., 19 Sep 2025).

2. Architectural Design: Dual-Branch Diffusion Framework

The architecture comprises two distinct but jointly trained diffusion models:

1. Blind-Spot Diffusion Branch:
   • Implements a BSN that reconstructs the image using only neighborhood pixels for each estimation, never accessing the center pixel in its receptive field.
   • This branch is typically trained in a self-supervised manner, minimizing a loss of the form:

   $\min_\theta \sum_t \| f_b(x_t, x_0, t) - x_0 \|_1$

   where $f_b$ is the BSN denoiser, $x_t$ is the noisy input at diffusion timestep $t$, and $x_0$ is the clean latent.

2. Conventional Diffusion Branch:
   • Uses a standard UNet architecture as the backbone, trained to predict either the clean signal or noise residuals in the noisy image.
   • The loss is similarly:

   $\min_\theta \sum_t \| f_u(x_t, t) - x_0 \|_1$

   with $f_u$ as the UNet denoiser.

During inference, the BSN output provides soft guidance to the reverse diffusion process. The score estimate for each denoising step is a linear combination: $$\epsilon(x_t) = w \cdot \epsilon_{fb}(x_t, x_0) + (1 - w) \cdot \epsilon_{fu}(x_t)$$ where $\epsilon_{fb}$ is from the BSN and $\epsilon_{fu}$ from the conventional branch; $w \in [0,1]$ regulates the influence of the blind spot prior.

3. Mitigating BSN and Diffusion Model Limitations

Standard BSN architectures, while excellent at avoiding identity mapping pitfalls, often diminish local detail and introduce grid-like artifacts due to the imposition of spatial independence. Using the BSN as guidance—not as the sole generator—alleviates these weaknesses. Meanwhile, conventional diffusion models require explicit supervision or paired data to reliably learn nontrivial inverse mappings.

Blind-spot guided diffusion simultaneously:

• Leverages the BSN to regularize the diffusion trajectory (reducing spatial discontinuities and unnatural artifacts).
• Uses the UNet branch to restore fine local details and correct pixel-wise anomalies otherwise missed by the spatial structure-agnostic BSN (Cheng et al., 19 Sep 2025).

Furthermore, a sampling strategy based on random pixel replacement reinforces the independence between the estimate and the noisy input, helping to prevent overfitting to the noise realization.

4. Training and Sampling Strategies

The method operates exclusively on noisy data. Both branches are trained to predict the underlying clean signal from partial or corrupted observations, employing:

• Blind-spot masking in the BSN branch.
• Standard denoising objectives in the UNet branch.

During sampling, the following strategy is used:

• At each reverse diffusion step, a "complementary replacement" is performed by stochastically replacing predicted clean pixels with corresponding pixels from the noisy input, in two rounds: one primary replacement and one complementary round. This augments the diversity and robustness of sampling, serving as an implicit regularizer (Cheng et al., 19 Sep 2025).

The process can be summarized as follows:

Branch	Training Loss	Sampling Role
BSN branch	$\|f_b(x_t, x_0, t) - x_0\|_1$	Provides structural guidance (semi-clean)
UNet branch	$\|f_u(x_t, t) - x_0\|_1$	Preserves local detail in output

5. Quantitative and Qualitative Performance

Blind-spot guided diffusion achieves state-of-the-art results on real-world denoising benchmarks such as SIDD and DND, approaching PSNR values of 38 dB and exhibiting competitive SSIM. Ablation studies confirm that both the mixing weight $w$ and replacement strategies are essential for optimal performance, substantially outperforming traditional BSN-only or non-blind-spot diffusion models in both quantitative and qualitative assessments. Visual inspection reveals that fine textural detail loss and grid artifacts common in vanilla BSN outputs are notably reduced.

Multiple rounds of random/complementary replacement during sampling further refine output consistency and stability, making the system practically valuable for deployment in real-world image processing tasks where clean labels are unavailable (Cheng et al., 19 Sep 2025).

6. Practical Implications and Broader Impact

This dual-branch approach is applicable to a variety of real-world image restoration settings—most notably smartphone photography, low-light imaging, and other situations marked by spatially correlated noise and a lack of ground truth. The capacity to generate high-fidelity, artifact-free images under self-supervision is particularly significant for consumer devices and downstream vision pipelines. By moving beyond the limitations of both traditional BSNs and standard diffusion models, blind-spot guided diffusion provides a template adaptable to further advances in self-supervised learning and inverse imaging.

7. Future Directions and Extensions

A plausible implication is that blind-spot guided diffusion may be extended to more complex inverse problems, such as those involving spatially varying noise models or temporally coherent video restoration. The explicit design of guidance schedules (i.e., adaptive $w$), more sophisticated pixel replacement policies, and broader integration with attention-based blind-spot modules are likely areas for future research. The methodology sets a foundation for self-supervised generative restoration where local detail, global structure, and noise diversity must be reconciled in the absence of labeled data.

---

In summary, blind-spot guided diffusion combines the structure-aware priors of blind-spot networks with the expressive, unsupervised sampling power of diffusion models, yielding a powerful self-supervised denoising method that overcomes the limitations of each constituent approach—validated by state-of-the-art empirical results in real-world image restoration (Cheng et al., 19 Sep 2025).