Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation (2007.05946v1)

Abstract: Real-world image noise removal is a long-standing yet very challenging task in computer vision. The success of deep neural network in denoising stimulates the research of noise generation, aiming at synthesizing more clean-noisy image pairs to facilitate the training of deep denoisers. In this work, we propose a novel unified framework to simultaneously deal with the noise removal and noise generation tasks. Instead of only inferring the posteriori distribution of the latent clean image conditioned on the observed noisy image in traditional MAP framework, our proposed method learns the joint distribution of the clean-noisy image pairs. Specifically, we approximate the joint distribution with two different factorized forms, which can be formulated as a denoiser mapping the noisy image to the clean one and a generator mapping the clean image to the noisy one. The learned joint distribution implicitly contains all the information between the noisy and clean images, avoiding the necessity of manually designing the image priors and noise assumptions as traditional. Besides, the performance of our denoiser can be further improved by augmenting the original training dataset with the learned generator. Moreover, we propose two metrics to assess the quality of the generated noisy image, for which, to the best of our knowledge, such metrics are firstly proposed along this research line. Extensive experiments have been conducted to demonstrate the superiority of our method over the state-of-the-arts both in the real noise removal and generation tasks. The training and testing code is available at https://github.com/zsyOAOA/DANet.

Dual Adversarial Network for Real-world Noise Removal and Generation

The paper by Yue et al. introduces a novel unified framework termed as Dual Adversarial Network (DANet) which concurrently tackles the tasks of real-world noise removal and noise generation. This work signifies an advancement in image processing by leveraging deep learning methodologies to handle the inherent noise in real-world images effectively.

Framework Overview

The proposed DANet framework diverges from traditional Bayesian approaches that typically focus on Maximum A Posteriori (MAP) estimates conditioned on noisy observations. Instead, the framework centers on learning the joint distribution of clean and noisy image pairs. The motivation behind this dual approach is to circumvent the necessity of manually crafting image priors or making assumptions about noise characteristics, which can lead to deviations when applied to real images.

DANet simultaneously models noise removal and generation tasks through two distinct mappings: a denoiser which infers clean images from noisy counterparts, and a generator which synthesizes noisy images from clean ones. This dual approach ensures the joint distribution contains symbiotic information between noisy and clean images.

Technical Detailing

The proposed method introduces a dual adversarial architecture akin to Generative Adversarial Networks (GANs), where:

• Denoiser $R$: Learns the conditional distribution $p(\bm{x}|\bm{y})$ to recover the clean image from its noisy observation.
• Generator $G$: Synthesizes noisy images using a distribution $p(\bm{y}|\bm{x})$, driven by a latent variable reflecting noise factors.

The framework integrates adversarial learning with traditional constraint-based learning, optimizing through adversarial loss augmented with $L_1$ regularization for the denoising task and a novel metric involving Gaussian filtering to stabilize noise synthesis.

Results and Implications

The conducted experiments highlight DANet's prowess over state-of-the-art methods in both noise removal and generation tasks across real-world benchmarks such as the SIDD and DND datasets. The framework is capable of producing high-fidelity denoised images, surpassing traditional and recent deep learning-driven approaches.

Numerically, DANet achieves commendable PSNR and SSIM improvements, indicating its robustness in preserving structural details and visual content integrity. Additionally, innovative metrics like PSNR Gap (PGap) and Average KL Divergence (AKLD) were introduced to objectively assess noise generation quality, further validating the efficiency of the proposed generator model in replicating real noise distributions.

Future Directions

The implications of this research are notably significant in the field of low-level vision tasks. By effectively modeling the joint distribution of noise and clarity, the DANet framework sets precedence for forthcoming enhancements in image restoration applications like super-resolution and deblurring. The ability to operate without dependencies on manual priors makes this framework adaptable to diverse datasets, a critical requirement for practical, real-world application.

Further investigation into improving the stability and scalability of the proposed dual adversarial learning, as well as extending similar methodologies to multi-modal and multi-task learning frameworks, could pave the way for even more effective noise management solutions in digital imaging and beyond.