FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising (1710.04026v2)

Abstract: Due to the fast inference and good performance, discriminative learning methods have been widely studied in image denoising. However, these methods mostly learn a specific model for each noise level, and require multiple models for denoising images with different noise levels. They also lack flexibility to deal with spatially variant noise, limiting their applications in practical denoising. To address these issues, we present a fast and flexible denoising convolutional neural network, namely FFDNet, with a tunable noise level map as the input. The proposed FFDNet works on downsampled sub-images, achieving a good trade-off between inference speed and denoising performance. In contrast to the existing discriminative denoisers, FFDNet enjoys several desirable properties, including (i) the ability to handle a wide range of noise levels (i.e., [0, 75]) effectively with a single network, (ii) the ability to remove spatially variant noise by specifying a non-uniform noise level map, and (iii) faster speed than benchmark BM3D even on CPU without sacrificing denoising performance. Extensive experiments on synthetic and real noisy images are conducted to evaluate FFDNet in comparison with state-of-the-art denoisers. The results show that FFDNet is effective and efficient, making it highly attractive for practical denoising applications.

• The paper introduces FFDNet’s integration of a noise level map, enabling a single CNN to handle diverse noise levels.
• It employs sub-image processing to balance computational efficiency and robust performance, achieving faster processing times.
• Orthogonal initialization enhances network robustness, reducing artifacts from noise level mismatches and improving denoising quality.

An Analysis of FFDNet: A Fast and Flexible Solution for CNN Based Image Denoising

The paper "FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising" by Kai Zhang, Wangmeng Zuo, and Lei Zhang introduces a convolutional neural network (CNN) framework tailored for image denoising tasks. Unlike traditional discriminative learning methods, which often require separate models for various noise levels, FFDNet incorporates a noise level map directly into the network input, achieving both speed and flexibility.

Key Contributions

FFDNet distinguishes itself with the following primary contributions:

1. Integration of Noise Level Map: The model introduces a noise level map as part of the input, allowing FFDNet to handle a range of noise levels (0-75) with a single network. This feature also permits the model to deal with spatially variant noise, a capability that was absent in prior discriminative denoisers.
2. Sub-image Processing: By working on downsampled sub-images rather than full-resolution images, FFDNet strikes an optimal balance between computational efficiency and denoising performance. This design choice accelerates the inference process without sacrificing image quality.
3. Orthogonal Initialization: The incorporation of orthogonal initialization of convolutional filters enhances the network's robustness, especially when the noise level map mismatches the ground-truth noise level. This addresses potential visual artifacts, ensuring reliable denoising across varied input conditions.

Experimental Results

Extensive experiments affirm FFDNet's efficacy and efficiency. For synthetic noisy images, the model shows improved performance and faster processing times compared to benchmark methods, including BM3D, WNNM, MLP, and DnCNN. On real noisy images, FFDNet demonstrates robust practical application, effectively dealing with complex noise types such as signal-dependent, non-Gaussian, and spatially variant noise.

Numerical Performance

• Set12 Dataset: FFDNet consistently outperforms WNNM and other classic denoising methods for higher noise levels, with incremental gains in PSNR, particularly for noise levels above $\sigma$ 25.
• BSD68 Dataset: Comparing with DnCNN, FFDNet achieves competitive PSNR values, particularly at noise levels above $\sigma$ 25, highlighting its strength in handling strong noise scenarios.

Practical Implications

The integration of a noise level map enables end-users to tune FFDNet according to different noise levels without retraining the model, addressing a common limitation in existing discriminative methods. The model's capability to process downsampled sub-images means significant improvements in speed, confirmed by empirical tests showing it can be up to three times faster than BM3D on CPU and markedly faster on GPU.

Theoretical Implications

FFDNet's design shifts the paradigm in CNN-based denoising by moving away from the conventional need to train specialized models for fixed noise levels. This innovation not only extends the flexibility and generalization of image denoising models but also sets a precedent for future work in noise adaptation and handling complex noise structures within a unified framework.

Future Developments

FFDNet paves the way for further exploration in several key areas:

• Adaptive Noise Model Integration:

Improving noise level map estimation methods or incorporating adaptive learning mechanisms within FFDNet to dynamically adjust the noise level map based on input data characteristics.

• Real-Time Applications:

Leveraging the computational efficiency of FFDNet for real-time image processing applications, particularly in areas requiring immediate noise suppression, such as medical imaging and live video enhancement.

• Generalization to Other Noise Types:

Extension of the noise model beyond Gaussian assumptions to include more complex noise distributions, thereby broadening the versatility of FFDNet in diverse real-world conditions.

In conclusion, FFDNet offers a sophisticated and practical solution for image denoising, combining speed, flexibility, and robust performance under varying noise conditions. Its novel use of a noise level map and sub-image processing technique marks a significant contribution to the field, laying foundational work for future advancements in adaptive and real-time image denoising technologies.