GT-Rain Single Image Deraining Challenge Report (2403.12327v1)

Abstract: This report reviews the results of the GT-Rain challenge on single image deraining at the UG2+ workshop at CVPR 2023. The aim of this competition is to study the rainy weather phenomenon in real world scenarios, provide a novel real world rainy image dataset, and to spark innovative ideas that will further the development of single image deraining methods on real images. Submissions were trained on the GT-Rain dataset and evaluated on an extension of the dataset consisting of 15 additional scenes. Scenes in GT-Rain are comprised of real rainy image and ground truth image captured moments after the rain had stopped. 275 participants were registered in the challenge and 55 competed in the final testing phase.

• The paper highlights the GT-Rain challenge, introducing a novel dataset and evaluation benchmarks for real-world single image deraining.
• It showcases diverse transformer-based methodologies, with teams using models like Restormer and Uformer to enhance performance based on PSNR and SSIM metrics.
• The study reveals innovative training strategies, including pseudo ground truth generation and multi-module pipelines, to tackle rain effects effectively.

GT-Rain Single Image Deraining Challenge Report: Insights and Analysis

Introduction

The challenge of dealing with adverse weather effects like rain in computer vision applications is well acknowledged within the research community. The GT-Rain Single Image Deraining Challenge, as part of the UG2+ workshop at CVPR 2023, aimed at addressing these concerns directly. It provided a platform for exploring innovative solutions to single image deraining on real-world images. The challenge was grounded on the GT-Rain dataset, offering an extended dataset for evaluation to stir advancement in the development and assessment of deraining methods.

The GT-RAIN Dataset and Challenge

The GT-RAIN Dataset emerges as a novel collection of high-quality, time-multiplexed pairs of real rainy and corresponding ground-truth images, captured moments apart. This dataset includes a diverse range of rain conditions, urban and natural settings, varying illumination, and camera specifications.

Challenge Structure

The challenge constituted three phases: Training, Validation, and Testing. Participants were encouraged to use the GT-RAIN dataset, optionally supplemented with other datasets. A total of 55 teams participated in the final phase, aiming to showcase their deraining techniques optimized towards realism and effectiveness, as quantified by PSNR and SSIM metrics.

Observations from Participants' Submissions

Submissions demonstrated a generalized trend towards employing transformer-based models, indicating their growing influence across computer vision tasks. Popular base models included Restormer and Uformer, highlighting a preference for building upon existing successful architectures. An interesting diversity in training approaches was observed, with teams either exclusively utilizing the GT-RAIN dataset or incorporating additional synthetic rain datasets to enhance their models' performance.

Highlighted Approaches

Two teams, HUST_VIE and FDL@ZLab, exemplified standout methodologies, with both opting for transformer-based models and real datasets in their training strategies.

Team HUST_VIE

This team showcased a two-stage technique focusing on generating pseudo ground truth images for initial model training, followed by fine-tuning on these pseudo GT pairs. Their approach emphasized tailoring model adjustments based on detailed analysis of the dataset characteristics, particularly the prevalence of rain-veiling effects.

Team FDL@ZLab

Employing a Restormer-Plus model, this team introduced a comprehensive four-module approach. This strategy included initial deraining, median filtering for temporal stability, weighted averaging to prevent overfitting, and post-processing adjustments for brightness correction, encapsulating a multifaceted approach to single image deraining.

Concluding Remarks

The GT-Rain Single Image Deraining Challenge represents a significant step forward in the pursuit of effective deraining techniques applicable to real-world scenarios. The diversity of approaches and the emphasis on real datasets indicate a maturing understanding of the complexities involved in weather effect removal from images. This initiative not only showcases the potential of current methodologies but also sets the stage for future innovations in the field of computer vision under challenging weather conditions.