CVF-SID: Cyclic multi-Variate Function for Self-Supervised Image Denoising by Disentangling Noise from Image (2203.13009v3)

Abstract: Recently, significant progress has been made on image denoising with strong supervision from large-scale datasets. However, obtaining well-aligned noisy-clean training image pairs for each specific scenario is complicated and costly in practice. Consequently, applying a conventional supervised denoising network on in-the-wild noisy inputs is not straightforward. Although several studies have challenged this problem without strong supervision, they rely on less practical assumptions and cannot be applied to practical situations directly. To address the aforementioned challenges, we propose a novel and powerful self-supervised denoising method called CVF-SID based on a Cyclic multi-Variate Function (CVF) module and a self-supervised image disentangling (SID) framework. The CVF module can output multiple decomposed variables of the input and take a combination of the outputs back as an input in a cyclic manner. Our CVF-SID can disentangle a clean image and noise maps from the input by leveraging various self-supervised loss terms. Unlike several methods that only consider the signal-independent noise models, we also deal with signal-dependent noise components for real-world applications. Furthermore, we do not rely on any prior assumptions about the underlying noise distribution, making CVF-SID more generalizable toward realistic noise. Extensive experiments on real-world datasets show that CVF-SID achieves state-of-the-art self-supervised image denoising performance and is comparable to other existing approaches. The code is publicly available from https://github.com/Reyhanehne/CVF-SID_PyTorch .

• The paper introduces CVF-SID, a self-supervised image denoising method that uses a Cyclic multi-Variate Function to disentangle noise from images without requiring paired clean data.
• This approach avoids typical noise assumptions by decomposing images into clean, signal-dependent, and signal-independent noise components through cyclic learning and various self-supervised loss functions.
• CVF-SID demonstrates comparable or superior denoising performance on real-world datasets like SIDD and DND using only noisy data, highlighting its practical effectiveness and potential for other tasks.

Analyzing the CVF-SID Approach for Self-Supervised Image Denoising

The paper introduces a self-supervised image denoising method named CVF-SID, leveraging a Cyclic multi-Variate Function (CVF) coupled with a self-supervised image disentangling (SID) framework. The research tackles the issue of noise removal from images without depending on paired noisy-clean datasets, aiming for more practical applications in real-world scenarios.

Core Contributions

The CVF-SID methodology is built around a cyclic function capable of decomposing an input image into constituent elements: a clean image and signal-dependent and signal-independent noise components. This cyclical decomposition approach allows the method to learn denoising without requiring ground truth or paired datasets, usually crucial in supervised methods. The CVF module forms the core of this mechanism by enabling recursive learning through cyclic inputs and outputs.

Technical Insights

The paper's approach to image denoising is innovative in its use of the CVF model to separate noisy inputs into clean and noise components based on different signal dependencies. It circumvents conventional reliance on prior assumptions about noise, notably the Gaussian noise assumption, making CVF-SID suitable for a broader range of real-world noise conditions.

The algorithm uses multiple self-supervised loss terms to ensure that each disentangled part remains statistically independent and requires that the noise maps maintain a zero-mean property, enhancing its applicability in practice. The loss functions employed for self-supervised learning include consistency, identity, zero constraints, regularization, and augmentation-related terms to provide a comprehensive framework capable of effectively training with only noisy data.

Results and Implications

When evaluated against existing methods, CVF-SID shows comparable or superior denoising performance on real-world datasets like SIDD and DND, reinforcing its effectiveness in practical applications. The numerical results demonstrate significant improvements with PSNR and SSIM metrics, validating the proposed self-supervised training strategy even when applied directly to sRGB inputs without the necessity for large-scale datasets or paired noisy-clean images.

Future Directions

The implications of CVF-SID extend to broader applications in image processing where labeled data is scarce. The cyclic decomposition model offers an opportunity for future exploration of similar self-supervised techniques in other computer vision tasks beyond just denoising.

A noted limitation concerns the fixed correlation parameter employed in the signal-dependent noise modeling. Future work could aim to dynamically learn this parameter, potentially improving the decomposition accuracy and adaptability across various image contexts. Additionally, exploring the integration of CVF principles with other unsupervised learning frameworks could broaden its utility in more complex image processing scenarios.

Overall, the paper contributes a robust, versatile approach to denoising with minimal reliance on assumptions, setting a precedent for future research in self-supervised and noise modeling methodologies.