A Model-driven Deep Neural Network for Single Image Rain Removal (2005.01333v1)

Abstract: Deep learning (DL) methods have achieved state-of-the-art performance in the task of single image rain removal. Most of current DL architectures, however, are still lack of sufficient interpretability and not fully integrated with physical structures inside general rain streaks. To this issue, in this paper, we propose a model-driven deep neural network for the task, with fully interpretable network structures. Specifically, based on the convolutional dictionary learning mechanism for representing rain, we propose a novel single image deraining model and utilize the proximal gradient descent technique to design an iterative algorithm only containing simple operators for solving the model. Such a simple implementation scheme facilitates us to unfold it into a new deep network architecture, called rain convolutional dictionary network (RCDNet), with almost every network module one-to-one corresponding to each operation involved in the algorithm. By end-to-end training the proposed RCDNet, all the rain kernels and proximal operators can be automatically extracted, faithfully characterizing the features of both rain and clean background layers, and thus naturally lead to its better deraining performance, especially in real scenarios. Comprehensive experiments substantiate the superiority of the proposed network, especially its well generality to diverse testing scenarios and good interpretability for all its modules, as compared with state-of-the-arts both visually and quantitatively. The source codes are available at \url{https://github.com/hongwang01/RCDNet}.

• The paper introduces RCDNet, a model-driven architecture that integrates convolutional dictionary learning to enhance single image rain removal.
• It formulates rain removal as an optimization problem solved via proximal gradient descent, using an unrolled network structure with dedicated sub-networks for rain maps and background layers.
• Experiments on benchmark datasets show significant improvements in PSNR and SSIM, demonstrating both enhanced performance and interpretability over traditional CNN approaches.

A Model-driven Deep Neural Network for Single Image Rain Removal

The task of single image rain removal is a crucial preprocessing step for numerous computer vision applications, especially in outdoor scenarios where rain severely hinders the visibility of images. This paper introduces a model-driven deep neural network named Rain Convolutional Dictionary Network (RCDNet) geared towards enhancing the interpretability of neural networks while improving their performance in rain removal from images. The key innovation lies in integrating convolutional dictionary learning within a deep learning framework to address the inadequacies of existing deep learning methods, which often operate as black-box models without leveraging the intrinsic properties of rain streaks.

Methodology

The core foundation of this work is the novel integration of the convolutional dictionary learning mechanism with neural network design. The paper formulates the task of single image deraining as an optimization problem, where the rain layer is modeled using rain kernels and rain maps. This formulation leads to a problem-solving approach where proximal gradient descent is employed to derive an iterative algorithm.

The iterative algorithm forms the basis of an "unrolled" deep network structure, where each iteration corresponds to a specific layer in the proposed RCDNet. This network architecture consists of stages, each comprising two sub-networks: the M-net for updating rain maps and the B-net for updating the background layer. The learned network incorporates the extraction of rain kernels and proximal operators, automatically tailored from training data, allowing for enhanced rain removal performance across diverse and complex scenarios.

Results and Analysis

Comprehensive experiments substantiate RCDNet's superiority over prior state-of-the-art techniques both qualitatively and quantitatively. On benchmark datasets such as Rain100L, Rain100H, Rain1400, and Rain12, the proposed method not only achieves remarkable peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) metrics but also demonstrates robustness in handling varying densities and patterns of rain streaks. Particularly, the RCDNet exhibits improved generalization to unseen rain conditions in real-world data, significantly advancing the capability of rain removal across different image contexts.

Implications and Future Directions

The implications of this research are significant both practically and theoretically. By introducing a model-driven approach within a deep learning framework, the RCDNet demonstrates enhanced interpretability, which is a notable improvement over traditional end-to-end CNNs that often act as opaque black boxes. This interpretability allows researchers to have clearer insights into network functioning and the learning process, which might lead to further optimizations and applications beyond rain streak removal.

The introduction of convolutional dictionary learning into neural network design opens avenues for further exploration of dictionary-based approaches for other image restoration tasks. It raises the prospect of utilizing similar strategies for diverse applications such as dehazing, denoising, and deblurring, where model-driven deep learning could potentially offer significant benefits.

In conclusion, while RCDNet makes a notable contribution to the domain of rain removal, the principles laid out establish a foundation that may drive future innovation in model-driven deep neural networks, incrementally bridging the gap between model interpretability and the versatility of deep learning.