Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising (2010.11971v1)

Abstract: Self-supervised frameworks that learn denoising models with merely individual noisy images have shown strong capability and promising performance in various image denoising tasks. Existing self-supervised denoising frameworks are mostly built upon the same theoretical foundation, where the denoising models are required to be J-invariant. However, our analyses indicate that the current theory and the J-invariance may lead to denoising models with reduced performance. In this work, we introduce Noise2Same, a novel self-supervised denoising framework. In Noise2Same, a new self-supervised loss is proposed by deriving a self-supervised upper bound of the typical supervised loss. In particular, Noise2Same requires neither J-invariance nor extra information about the noise model and can be used in a wider range of denoising applications. We analyze our proposed Noise2Same both theoretically and experimentally. The experimental results show that our Noise2Same remarkably outperforms previous self-supervised denoising methods in terms of denoising performance and training efficiency. Our code is available at https://github.com/divelab/Noise2Same.

• The paper introduces a novel self-supervised loss that bypasses the strict J-invariance requirement to enhance image denoising performance.
• It combines a reconstruction MSE and an invariance MSE term, enabling effective denoising without relying on paired clean images.
• Experimental results demonstrate that Noise2Same outperforms traditional methods and previous self-supervised approaches under various noise types.

Overview of Noise2Same: Optimizing a Self-Supervised Bound for Image Denoising

The paper "Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising" presents a novel framework, Noise2Same, designed to enhance self-supervised image denoising techniques. Traditional supervised denoising methods, which require paired noisy and clean images, struggle in situations where clean images are unavailable due to practical limitations. Earlier alternatives, such as Noise2Noise, use pairs of noisy images instead, but these scenarios are constrained due to the need for such pairs and potential registration issues.

Noise2Same advances self-supervised denoising methods, which are typically based on the theoretical foundation of $\mathcal{J}$-invariant functions. Previous studies have relied on this concept to ensure that denoising models do not learn the identity function when no clean data is available. However, the researchers point out that the $\mathcal{J}$-invariance assumption may lead to suboptimal performance, as demonstrated by their insightful analyses.

Novel Contributions

The core contribution of Noise2Same lies in its approach to remove the strict $\mathcal{J}$-invariance constraint, proposing an alternative self-supervised loss function. The derivation within the paper formulates an upper bound of the traditional supervised loss in a self-supervised context. This reframes the task by minimizing a self-supervised loss that consists of both a reconstruction mean squared error (MSE) term and an invariance MSE term, where the latter governs how strictly the denoising function should adhere to being $\mathcal{J}$-invariant.

Methodology

The Noise2Same framework includes the following conceptual elements:

1. Absence of $\mathcal{J}$-Invariance Need: By pivoting away from the conventional necessity for $\mathcal{J}$-invariance, Noise2Same allows the denoising model to leverage full image information without any need for post-processing or noise model knowledge.
2. Self-Supervised Loss Design: The authors introduce a unique self-supervised loss consisting of a reconstruction MSE term and a square-root of variance invariance MSE. This enables the model to avoid learning the identity function while optimizing denoising performance.
3. Experimental Evaluation: Empirical experiments demonstrate that Noise2Same consistently excels past its predecessors across several datasets with varied noise levels and types. This includes improvements over methods such as Noise2Self and Noise2Void, especially in scenarios without known noise models.

Key Results

The experimental results showcased that the Noise2Same framework achieves superior denoising performance compared to traditional methods like BM3D and self-supervised baselines on datasets with diverse noise characteristics. Particularly noteworthy is the robustness of Noise2Same to combined and unknown noise types, which are challenging for other methods that depend on noise model information.

Implications and Future Directions

The removal of the $\mathcal{J}$-invariance constraint marks a significant theoretical advancement, suggesting practical applications across a wider range of denoising scenarios than previously achievable. In applying Noise2Same, tasks requiring image pre-processing, such as object detection and academic microscopy, will benefit by obtaining cleaner input data without the necessity of clean counterparts or prior noise modeling.

Future work can explore integrating known noise models with Noise2Same, potentially augmenting its capability and bridging the performance gap between self-supervised and supervised methods. Moreover, expanding Noise2Same to other domains like audio noise reduction could be another fruitful avenue of research.

In summary, the Noise2Same framework represents a vital step toward more efficient, widely applicable self-supervised image denoising strategies, advancing both theoretical understanding and practical capability.