UNSURE: self-supervised learning with Unknown Noise level and Stein's Unbiased Risk Estimate

Abstract: Recently, many self-supervised learning methods for image reconstruction have been proposed that can learn from noisy data alone, bypassing the need for ground-truth references. Most existing methods cluster around two classes: i) Stein's Unbiased Risk Estimate (SURE) and similar approaches that assume full knowledge of the noise distribution, and ii) Noise2Self and similar cross-validation methods that require very mild knowledge about the noise distribution. The first class of methods tends to be impractical, as the noise level is often unknown in real-world applications, and the second class is often suboptimal compared to supervised learning. In this paper, we provide a theoretical framework that characterizes this expressivity-robustness trade-off and propose a new approach based on SURE, but unlike the standard SURE, does not require knowledge about the noise level. Throughout a series of experiments, we show that the proposed estimator outperforms other existing self-supervised methods on various imaging inverse problems.

• The paper introduces UNSURE, a novel self-supervised approach that reconstructs images without prior noise level knowledge by relaxing the strict divergence constraint.
• The paper derives closed-form solutions under various noise distributions, achieving near-MMSE performance and outperforming traditional cross-validation methods.
• The paper validates UNSURE on tasks like MRI reconstruction, demonstrating its practical ability to balance robustness and expressivity in real-world imaging scenarios.

UNSURE: Unknown Noise Level Stein's Unbiased Risk Estimator

The paper “UNSURE: Unknown Noise Level Stein's Unbiased Risk Estimator,” authored by Juli an Tachella, Mike Davies, and Laurent Jacques, proposes a novel approach to self-supervised learning for image reconstruction when the noise distribution is unknown. This method, termed UNSURE, stands as a significant contribution to addressing the inherent trade-offs between robustness and expressivity in image denoising algorithms.

Introduction and Context

Image reconstruction methods have achieved notable success in recent applications, most prominently in medical imaging and computational photography. Traditional supervised learning paradigms demand ground-truth data for training, which is often impractical in these domains due to the high cost or inaccessibility of clean, noise-free images. This limitation has given rise to self-supervised approaches that only necessitate noisy data for training.

Existing self-supervised methods can be broadly categorized into two classes:

1. Methods such as Noise2Self and other cross-validation techniques that operate under minimal assumptions regarding the noise distribution.
2. Approaches based on Stein's Unbiased Risk Estimator (SURE), assuming comprehensive knowledge of the noise distribution.

The former category, while robust, often yields suboptimal performance compared to supervised techniques. The latter, although potentially more accurate, is impractical due to the common absence of precise knowledge about the noise level in real-world applications.

The UNSURE Framework

The authors introduce a theoretical framework to elucidate the robustness-expressivity trade-off inherent in these methods. They pinpoint that methods relying on full noise distribution information (like SURE) are highly expressive but less robust against noise misestimation. In contrast, cross-validation methods, which impose stricter constraints on estimator derivatives, are robust but less expressive.

UNSURE is proposed as a compromise, aiming to strike a balance by not requiring noise level knowledge yet maintaining a higher level of expressivity than cross-validation approaches. The key insight is constraining the divergence of the estimator to zero in expectation rather than strictly zero. Mathematically, this translates to optimizing the following objective:

$$f^{\text{DF}} = \argmin_{f \in \mathcal{S}_{\text{DF}}} \mathbb{E}_{y} \|f(y) - y\|^2$$

where

$$\mathcal{S}_{\text{DF}} = \{ f \in L : \mathbb{E}_{y} \,\text{div} f(y) = 0 \}$$

This approach effectively relaxes the zero-derivative constraint of cross-validation methods, thereby enhancing expressivity.

Analytical and Experimental Validation

The theoretical underpinnings of UNSURE are detailed rigorously. The authors derive closed-form solutions under various noise settings, including Gaussian, Poisson-Gaussian, and spatially correlated noise, emphasizing the generality and adaptability of their framework.

Key results derived include:

• The expected mean squared error (MSE) under optimal divergence-free denoisers, demonstrating the close-to-MMSE performance

$$\mathbb{E}_{x,y} \| {f(y) - x }^2 = \sigma^2 \left(\frac{1}{1 - \frac{\text{MMSE}}{\sigma^2}} - 1\right)$$

• New insights into noise estimation where optimal divergence-free estimators offer a conservative yet effective measure of noise level.

The empirical validation comprises various image denoising and reconstruction tasks. Key findings include:

• UNSURE consistently outperforms traditional cross-validation methods and is on par with SURE where noise distribution details are known.
• In practical applications like MRI reconstruction and computed tomography, UNSURE exhibits robust performance under unknown noise levels, validating its theoretical claims.

Implications and Future Directions

The implications of UNSURE are manifold:

• In practice, it alleviates the need for noise level estimation, a common hurdle in applying SURE-based methods.
• Theoretically, it provides a structured way to balance robustness and expressivity, contributing to the broader understanding of self-supervised learning methodologies.
• The adaptability to various noise distributions widens its applicability across different imaging scenarios beyond the conventional Gaussian noise settings.

Future research can build on this framework to explore adaptive techniques that could self-tune the divergence constraint parameters in more complex noise scenarios. Additionally, extending UNSURE to handle more perceptually relevant losses could bridge the gap between mathematical optimality and perceptual quality in image reconstruction tasks.

Conclusion

The UNSURE methodology innovatively extends the utility of Stein’s Unbiased Risk Estimator to scenarios with unknown noise levels, maintaining robustness while enhancing expressivity over existing cross-validation methods. This work marks a significant step forward in self-supervised learning for image reconstruction, promising improved practical performance across a spectrum of real-world applications.