Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining (1807.05698v2)

Abstract: Rain streaks can severely degrade the visibility, which causes many current computer vision algorithms fail to work. So it is necessary to remove the rain from images. We propose a novel deep network architecture based on deep convolutional and recurrent neural networks for single image deraining. As contextual information is very important for rain removal, we first adopt the dilated convolutional neural network to acquire large receptive field. To better fit the rain removal task, we also modify the network. In heavy rain, rain streaks have various directions and shapes, which can be regarded as the accumulation of multiple rain streak layers. We assign different alpha-values to various rain streak layers according to the intensity and transparency by incorporating the squeeze-and-excitation block. Since rain streak layers overlap with each other, it is not easy to remove the rain in one stage. So we further decompose the rain removal into multiple stages. Recurrent neural network is incorporated to preserve the useful information in previous stages and benefit the rain removal in later stages. We conduct extensive experiments on both synthetic and real-world datasets. Our proposed method outperforms the state-of-the-art approaches under all evaluation metrics. Codes and supplementary material are available at our project webpage: https://xialipku.github.io/RESCAN .

• The paper introduces RESCAN, a recurrent network that iteratively refines image quality by removing rain streaks using context aggregation and squeeze-and-excitation mechanisms.
• The method leverages dilated convolutions and omits batch normalization to improve PSNR and SSIM scores on both synthetic and real-world datasets.
• The study demonstrates RESCAN’s practical potential for applications like autonomous driving and surveillance, setting a new benchmark in image deraining performance.

Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining

The paper presents a sophisticated neural architecture designed for single image deraining, addressing critical challenges posed by rain streaks that impair visibility and performance in computer vision tasks. The proposed framework, termed RESCAN (Recurrent Squeeze-and-Excitation Context Aggregation Net), draws upon a blend of deep convolutional, recurrent neural networks, and squeeze-and-excitation blocks.

Core Methodology

RESCAN is structured to leverage spatial contextual information, essential for effective rain removal. The network utilizes dilated convolutional layers, facilitating an expansive receptive field necessary for context aggregation. Central to the design is the decomposition of rain removal into iterative stages, employing recurrent neural networks (RNNs) to ensure that preceding stage outputs inform later stages, enhancing the overall deraining capability.

Squeeze-and-Excitation Mechanism

A notable inclusion is the squeeze-and-excitation (SE) block, enabling channel-wise weighting and adjustment through learned alpha-values. This mechanism allows the network to discern and apply differential intensity and transparency treatments across multiple rain streak layers, thus refining the deraining process.

Comparative Analysis

The authors conduct extensive empirical evaluations against state-of-the-art benchmarks across synthetic datasets (Rain800, Rain100H) and real-world scenarios. Quantitatively, RESCAN achieves superior PSNR and SSIM scores, marking a consistent improvement over existing methodologies, such as DetailsNet and JORDER. Qualitative assessments affirm the model’s prowess in not only reducing rain streaks but also retaining background detail.

Architectural Insights

An intrinsic advantage of the RESCAN architecture is its elimination of batch normalization (BN) layers, which traditionally normalize feature maps, potentially disrupting the independence of rain layer representations. By removing BN, the model conserves GPU memory and maintains the integrity of diverse rain streak characterizations.

Additionally, a critical evaluation of different recurrent structures (ConvRNN, ConvGRU, ConvLSTM) and prediction frameworks (Additive vs. Full Prediction) highlights the architecture's adaptability and the benefits of recurrent connections between stages.

Practical and Theoretical Implications

Practically, the implementation of RESCAN addresses a significant problem in computer vision applications deployed in adverse weather conditions, such as autonomous driving and surveillance. Theoretically, the careful integration of SE blocks and the innovative use of RNNs in a sequential task framework present new directions for multi-stage image processing methodologies.

Future Prospects

Looking forward, advancement in this domain could involve the exploration of more sophisticated attention mechanisms or leveraging unsupervised learning paradigms to reduce dependency on synthetic datasets. Further, scaling the approach to handle diverse environmental artifacts beyond rain, such as fog or snow, offers promising research avenues.

In conclusion, RESCAN represents an essential step forward in image deraining technology, demonstrating significant advancements in both methodological sophistication and empirical performance. As the landscape of computer vision continues to evolve, approaches such as RESCAN will be pivotal in enhancing the robustness and adaptability of visual systems in real-world environments.