Raindrop-Aware 3DGS Evaluation

• The paper introduces a comprehensive evaluation framework that quantifies raindrop-induced degradations in 3DGS, highlighting significant PSNR drops and feature loss.
• It employs a unique triplet dataset—with raindrop-focused, background-focused, and rain-free images—to enable precise analysis of camera pose estimation and point cloud initialization.
• The evaluation compares multiple 3DGS variants and deraining techniques, underscoring limitations of single-image restoration and the potential of adaptive multi-view processing.

Raindrop-aware 3D Gaussian Splatting (3DGS) evaluation addresses the fundamental challenges posed by raindrop contamination in multi-view image-based 3D scene reconstruction. The disruptive nature of raindrops—including severe occlusions, refractive distortions, intensity attenuation, and loss of background detail—results in substantial degradation of both camera pose estimation and point cloud initialization, thereby limiting the fidelity of 3DGS outputs. Contemporary benchmarks and evaluation pipelines move beyond idealized synthetic scenarios and tackle unconstrained, in-the-wild raindrop conditions by integrating physically plausible dataset generation, robust preprocessing, and comparative analyses of pipeline components.

1. Benchmark Structure and Dataset Acquisition

The RaindropGS benchmark (Teng et al., 20 Oct 2025) exemplifies the comprehensive evaluation pipeline for raindrop-aware 3DGS, consisting of: (a) data preparation, (b) data processing, and (c) 3DGS evaluation.

Data Preparation

A real-world dataset is constructed using a fixed camera and a calibrated glass plate. Each scene comprises three perfectly aligned image sets:

• Raindrop-focused (camera focus on sharp raindrops, background blurred)
• Background-focused (raindrops out-of-focus, background sharp)
• Rain-free ground truth (clean reference)

This design ensures the full spectrum of raindrop-induced image artifacts—from strong optical distortions due to refraction and lensing in focused captures, to subtler attenuation and blurring in defocused backgrounds. The dataset spans multiple viewpoints and raindrop distributions to facilitate robust, generalizable evaluation.

Significance

Such explicit separation enables fine-grained analysis of how lens focus and raindrop appearance influence downstream processing, including feature detection, pose estimation, and geometric reconstruction. The aligned triplets support paired and cross-modal comparisons.

2. Raindrop Interference Typology

Raindrop interference manifests in two principal forms, each differently affecting pipeline components:

Interference Type	Description	Impact on Pipeline
Raindrop-focused	Raindrops are sharp, background blurred	Severe occlusion, refraction; poor feature matching
Background-focused	Raindrops blurred, background sharp	Mild occlusion, local distortions; partial retention of scene geometry

In raindrop-focused cases, surface droplets function as convex lenses, producing complex refractive and attenuation effects governed by Snell’s law: $n_1 \sin \theta_1 = n_2 \sin \theta_2$. This exacerbates feature loss and geometric errors. Background-focused settings allow recovery of scene geometry but retain multi-view inconsistencies due to residual distortions.

3. Camera Pose Estimation and Point Cloud Initialization

Successful 3DGS relies on accurate camera pose recovery and robust point cloud seeding.

• Classical SfM/MVS (e.g., COLMAP): Performance is directly impaired by raindrop occlusions; feature matching degrades, leading to missing or inaccurate camera poses and sparse point clouds.
• Transformer-based Methods (e.g., VGGT): Utilizing DINO-like features and transformer encoders improves robustness, but is still challenged by severe interference.
• Fallback Techniques: For extreme cases (no successful initialization from features), random point cloud seeding (e.g., 100,000 points) is used as a last resort.

AUC@30 (area under the curve at 30° threshold) and the number of reconstructed 3D points quantify the success of pose/point cloud estimation, showing marked declines under severe raindrop impact.

4. Single Image Raindrop Removal Strategies

To mitigate raindrop artifacts prior to 3DGS, several state-of-the-art deraining methods are benchmarked:

Model	Description and Mechanism	Noted Performance
Uformer	Window-based self-attention; multi-scale modulator	Best PSNR/SSIM on multiple setups
Restormer	Multi-Dconv attention; gated-Dconv FFN	Comparable to Uformer, slightly lower
IDT	Dual Transformer, distinct encoder/decoder	Noticeably lower restoration quality

Quantitative metrics (PSNR, SSIM, LPIPS) indicate that even advanced deraining networks introduce artifacts and fail to enforce multi-view consistency, which can propagate errors into reconstruction.

5. 3D Gaussian Splatting Method Comparison

Multiple 3DGS variants are evaluated for robustness in raindrop scenarios:

Method	Optimization Policy	Observed Performance
3DGS	Baseline, no raindrop-aware adaptation	Sensitive to input errors
WeatherGS	Single-image-based deraining in the pipeline	Marginal improvement over baseline
GS-W	Adaptive to unconstrained artifacts, longer training	Highest PSNR for background-focused images
3DGS-MCMC	Robust to point cloud/pose errors via MCMC strategies	Best for raindrop-focused, degraded initializations

GS-W achieves approximately 19.123 dB PSNR on background-focused images, but suffers from initialization sensitivity. 3DGS-MCMC exhibits enhanced resilience when conventional initialization fails due to occlusions.

6. Experimental Findings and Critical Insights

Comprehensive analyses reveal dominant limitations:

• Raindrop-focused images cause a substantial decrease in PSNR (∼4 dB drop for baseline) and dramatic feature loss.
• Even transformer-based pose estimation falters when feature loss is extreme.
• The incremental quality gains from deraining are limited by lack of multi-view consistency.
• Robust scene reconstruction under raindrop conditions is cumulatively sensitive to all pipeline stages.

Notably, single-image restoration and adaptive point cloud seeding partially offset but cannot wholly eliminate the errors introduced by lens contamination. Reference-aligned datasets and thorough metric tables (PSNR, SSIM, LPIPS) substantiate the findings.

7. Future Research Directions

Several consequential avenues are established:

• Robust Multi-view Raindrop Removal: Enforcing cross-view consistency could address multi-view geometric artifacts.
• Physical Modeling Integration: Directly modeling raindrop optics (e.g., via Snell’s law or light scattering) during pose and reconstruction could improve accuracy.
• Adaptive Training/Initialization: Dynamic point cloud seeding and iterative optimization attuned to varying occlusion levels can enhance robustness.
• Cross-modal Fusion: Techniques such as RGB-to-polarization inference may help disentangle scene from artifact.
• Generalization to Broader Weather Phenomena: Extensions to simultaneous rain, fog, and other artifacts will require holistic pipeline adaptations.

References and Context

The RaindropGS benchmark (Teng et al., 20 Oct 2025) is foundational for evaluating 3DGS reliability under real-world adverse weather. Its pipeline bridges the gap between synthetic, constrained tests and authentic, unconstrained scenarios. Comparisons to methods such as WeatherGS, GS-W, 3DGS-MCMC, and advanced deraining architectures, along with auxiliary physical and algorithmic models, provide a comprehensive view of current capabilities and limitations in raindrop-aware 3DGS evaluation.