Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset (1904.01538v2)

Abstract: Removing rain streaks from a single image has been drawing considerable attention as rain streaks can severely degrade the image quality and affect the performance of existing outdoor vision tasks. While recent CNN-based derainers have reported promising performances, deraining remains an open problem for two reasons. First, existing synthesized rain datasets have only limited realism, in terms of modeling real rain characteristics such as rain shape, direction and intensity. Second, there are no public benchmarks for quantitative comparisons on real rain images, which makes the current evaluation less objective. The core challenge is that real world rain/clean image pairs cannot be captured at the same time. In this paper, we address the single image rain removal problem in two ways. First, we propose a semi-automatic method that incorporates temporal priors and human supervision to generate a high-quality clean image from each input sequence of real rain images. Using this method, we construct a large-scale dataset of $\sim$$29.5K$ rain/rain-free image pairs that covers a wide range of natural rain scenes. Second, to better cover the stochastic distribution of real rain streaks, we propose a novel SPatial Attentive Network (SPANet) to remove rain streaks in a local-to-global manner. Extensive experiments demonstrate that our network performs favorably against the state-of-the-art deraining methods.

• The paper proposes a novel SPANet leveraging a two-round four-directional IRNN to capture local and global rain streak patterns.
• It constructs a high-quality real rain dataset of around 29.5K paired images using temporal priors and human supervision.
• Experimental results demonstrate that SPANet outperforms existing methods in PSNR and SSIM, enhancing real-world image clarity.

Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset

This paper addresses the problem of removing rain streaks from single images, a significant challenge in outdoor computer vision tasks. The authors identify two primary issues limiting progress in this domain: the limited realism of synthetic datasets and the absence of robust public benchmarks for evaluating deraining methods on real-world images.

Dataset Construction

The authors propose a semi-automatic method utilizing temporal priors and human supervision to construct a large-scale dataset comprising approximately 29.5K paired rain/clean images. The procedure incorporates selecting rain-free pixels from sequences of rainy images, leveraging the observation that rain rarely covers the same pixel in consecutive frames. This constructed dataset offers a substantial advantage over existing synthetic datasets by accurately capturing the stochastic variations in real rain streaks, such as different shapes and densities.

Spatial Attentive Network (SPANet)

To enhance the accuracy of rain removal, the authors introduce the Spatial Attentive Network (SPANet). This network is designed to operate in a local-to-global manner, initially leveraging neighborhood information to model rain streak characteristics and then removing them using non-local context. SPANet's architecture utilizes a Spatial Attentive Module (SAM) comprising a two-round four-directional IRNN, which projects rain streaks effectively while capturing spatial contextual information. This helps in explicitly identifying rain-affected regions.

Experimental Evaluation

The SPANet demonstrates superior performance over existing methods, as evidenced by quantitative metrics, outperforming several state-of-the-art derainers. Extensive experiments confirm the efficacy of SPANet, especially when trained on the proposed real dataset, showing significant improvements in PSNR and SSIM scores. The explicit attention mechanism in SPANet effectively identifies and removes rain streaks, even those with significant appearance variations.

Implications and Future Directions

The introduction of a high-quality real-world dataset paves the way for more effective training of single-image derainers, addressing a gap in the field where synthetic datasets have been inadequate. This research expands the understanding of rain streak modeling and removal, offering a practical approach that may influence future developments in outdoor computer vision tasks affected by adverse weather conditions.

Looking ahead, this approach can be further extended to tackle complex environments with heavy rain and mist, an area where SPANet shows limitations. Additionally, automating the dataset generation to reduce reliance on human supervision could refine this method, allowing for more extensive and efficient datasets. The insights from this work suggest that spatial attention mechanisms may see broader application in other image restoration tasks, enhancing model interpretability and accuracy.

In summary, this paper makes significant contributions to the image deraining field by addressing fundamental dataset limitations and introducing a novel spatial attentive framework that effectively identifies and removes rain streaks. This advancement holds promise for improving the robustness and applicability of computer vision systems in real-world scenarios.