Toward Convolutional Blind Denoising of Real Photographs (1807.04686v2)

Abstract: While deep convolutional neural networks (CNNs) have achieved impressive success in image denoising with additive white Gaussian noise (AWGN), their performance remains limited on real-world noisy photographs. The main reason is that their learned models are easy to overfit on the simplified AWGN model which deviates severely from the complicated real-world noise model. In order to improve the generalization ability of deep CNN denoisers, we suggest training a convolutional blind denoising network (CBDNet) with more realistic noise model and real-world noisy-clean image pairs. On the one hand, both signal-dependent noise and in-camera signal processing pipeline is considered to synthesize realistic noisy images. On the other hand, real-world noisy photographs and their nearly noise-free counterparts are also included to train our CBDNet. To further provide an interactive strategy to rectify denoising result conveniently, a noise estimation subnetwork with asymmetric learning to suppress under-estimation of noise level is embedded into CBDNet. Extensive experimental results on three datasets of real-world noisy photographs clearly demonstrate the superior performance of CBDNet over state-of-the-arts in terms of quantitative metrics and visual quality. The code has been made available at https://github.com/GuoShi28/CBDNet.

Toward Convolutional Blind Denoising of Real Photographs

Overview

The paper, titled "Toward Convolutional Blind Denoising of Real Photographs," addresses the critical challenge of image denoising under realistic scenarios. Traditional denoising methods, which are often tailored to additive white Gaussian noise (AWGN), fail to generalize to the complex noise present in real-world photographs. The authors of this paper propose a novel Convolutional Blind Denoising Network (CBDNet) designed to bridge this gap by incorporating a more sophisticated noise model and leveraging both synthetic and real noisy-clean image pairs for training.

Methodology

CBDNet comprises two subnetworks:

1. Noise Estimation Subnetwork (CNN_E): This subnetwork predicts a noise level map from the input noisy image.
2. Non-Blind Denoising Subnetwork (CNN_D): This subnetwork uses the noisy image and the predicted noise level map to produce a cleaned image.

To mimic real-world noise, the authors employ a noise model that includes:

• Poisson-Gaussian Noise to account for the signal-dependent noise characteristics.
• In-Camera Signal Processing (ISP) Pipeline such as demosaicing, gamma correction, and JPEG compression to generate realistic noisy images.

Moreover, the paper incorporates synthetic and real noisy images during training to enhance the generalization capability of the model. Real noisy images are obtained from datasets with nearly noise-free counterparts, synthesized by averaging multiple images of the same scene.

An asymmetric loss function is introduced in the noise estimation subnetwork to penalize under-estimation errors more heavily than over-estimation errors. This significantly improves the robustness of the network when the actual noise does not perfectly match the synthetic model.

Results

The authors benchmark CBDNet against state-of-the-art methods using three datasets: NC12, DND, and Nam. Quantitative results demonstrate that CBDNet achieves superior performance in terms of PSNR and SSIM metrics. Specifically:

• On the DND dataset, CBDNet outperforms other methods such as BM3D, WNNM, and newer methods like TWSC.
• On the Nam dataset, CBDNet trained with JPEG compression (CBDNet(JPEG)) provides the best results, highlighting the importance of including JPEG compression in the ISP pipeline for practical noise scenarios.

Qualitative evaluations also show that CBDNet effectively removes noise while preserving image details, outperforming both traditional denoising approaches and recent CNN-based methods.

Implications and Future Work

The practical implications of this research are profound. By using a realistic noise model and incorporating real-world noisy images in training, CBDNet significantly improves the performance of blind denoising in real-world scenarios. This advancement can be directly applied to various fields such as digital photography, medical imaging, and surveillance.

Theoretically, the introduction of asymmetric loss functions opens new avenues for noise estimation in neural networks. Future work can explore other forms of asymmetric losses and their impacts on different image restoration tasks.

Moreover, interactive image denoising facilitated by the noise estimation subnetwork offers a user-friendly approach, allowing users to adjust the denoising levels dynamically. This adaptability can be crucial for applications requiring fine-tuned denoising thresholds, such as professional photo editing and scientific imaging.

In conclusion, while CBDNet represents a significant step forward in practical image denoising, continued research into more comprehensive noise models and advanced training methodologies will further enhance the capabilities and applications of convolutional denoising networks in real-world scenarios.