---
title: 'Raw-Rain: Benchmark for Raw-Domain Deraining'
url: https://www.emergentmind.com/topics/raw-rain
type: topic
---

# Raw-Rain: Benchmark for Raw-Domain Deraining

Searching arXiv for the benchmark and closely related raw-domain and rain-restoration papers.
Raw-Rain is a benchmark for rainy-scene image reconstruction and deraining that was introduced by “$\mathbf{R}^3$: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain” as the **first public benchmark of real rainy scenes captured in both 12-bit Bayer and bit-depth-matched sRGB** [2509.24022]. Its central purpose is to compare restoration **before** and **after** the image signal processor (ISP), and thereby test the claim that deraining directly on raw Bayer sensor measurements preserves recoverable scene information that post-ISP sRGB processing irreversibly alters. In the literature on rain removal, Raw-Rain occupies a specific position: it is neither a purely synthetic rain generator nor a generic real-rain RGB dataset, but a paired raw/sRGB empirical basis for evaluating an **ISP-last** restoration pipeline under real rainy capture conditions [2509.24022].

## 1. Historical placement within rain-image research

Raw-Rain emerged from a line of work in which most rain synthesis and deraining methods operated in processed image space rather than in sensor space. Early exemplar-based synthesis transferred real rain streak appearance by extracting rainy patches from an exemplar, computing residual rain patches as \(P^o = P^r - \operatorname{mean}(P^r)\), and adding them to clean targets as \(P^s = P^o + P^t\); the method explicitly preserved real streak appearance but did **not** model true sensor-RAW rain formation [1610.00427]. Model-based RGB deraining then treated rainy images through additive decompositions such as \(I = I_R + I_{NR}\), with hand-crafted rain detection and gradient-domain quasi-sparsity priors, again in post-ISP image space rather than raw sensor space [1812.08348]. Real-rain supervision was later approached by data distillation, where rain estimated from real rainy RGB images was transferred onto unrelated clean RGB images to form realistic training pairs without conventional software rain rendering [1904.04605].

A more direct precursor to Raw-Rain in the strict raw-domain sense is “End-to-end Rain Streak Removal with RAW Images” [2312.13304]. That work argued that rain removal should occur **before** demosaicing, white balance, exposure scaling, tone mapping, gamma, and other ISP stages, and proposed a joint solution for rain removal and RAW processing that maps rainy RAW input to clean RGB output. However, its supervision depended on **synthetic rainy RAW generation** obtained by unprocessing color rain streaks into RAW space, rather than on a public benchmark of **real rainy scenes captured in both raw and matched sRGB** [2312.13304]. Raw-Rain closes precisely that gap.

This historical trajectory suggests a shift in problem formulation. Earlier work largely asked how to synthesize or remove rain in RGB images; Raw-Rain asks whether the **domain of restoration itself** should move upstream to linear Bayer measurements, where ISP-induced clipping, channel mixing, tone compression, and interpolation have not yet occurred [2509.24022].

## 2. Acquisition protocol and dataset composition

The Raw-Rain dataset is also referred to as **Raw-Rain-Stereo (RRS)** and is acquired with a **dual-camera stereo rig** using **two synchronized FLIR Blackfly S 2.8MP cameras**, each capturing **12-bit Bayer (RGGB)** frames at **\(1080 \times 1920\)** resolution [2509.24022]. All frames were recorded with **fixed analog gain = 1**, while **exposure times varied from 300 ms to 10,000 ms** to cover illumination conditions including daylight and low-light or evening scenes. The paper does not provide sensor model numbers, focal lengths, stereo baseline, frame rate, shutter synchronization details, or explicit geometric calibration details [2509.24022].

The acquisition protocol is designed to create realistic rain corruption rather than synthetic overlays. A **rain system** scatters water-rain-like droplets **on the windshield and in the depth of the scene**. Clean ground truth is captured **before triggering the rain system**, and rainy observations are captured **immediately after**; a rig is used to ensure spatial stability throughout the process [2509.24022]. The paper further states that some scenes were recorded **through a stationary glass window**, others **from within a moving vehicle**, and some included **windshield wipers operating at different speeds**, broadening the benchmark beyond static outdoor imagery.

In composition, the dataset contains **89 diverse training scenes**, **30 identity training scenes**, **15 validation scenes**, and **10 held-out test scenes** [2509.24022]. Each scene is a **300-frame video sequence for each camera**, producing **48,000 training frames**, **18,000 identity frames**, **9,000 validation frames**, and **6,000 test frames**. The validation split is explicitly marked **“no GT”** and used for **visual testing only**. Rain intensity is categorized as **light, medium, or heavy**; the training set contains **47 light**, **31 medium**, and **11 heavy** rain scenes, while the test set contains **2 light**, **6 medium**, and **2 heavy** scenes [2509.24022].

The benchmark also records scene categories and operating conditions. Training locations are summarized as **17 forest**, **29 city**, **30 playground**, and **13 parking lot** scenes. The train scene settings are reported as **30 behind-glass window**, **42 within a vehicle**, and **14 in a vehicle with wipers**; these counts do **not** exactly sum to 89, and the paper does not clarify whether this reflects overlap or reporting inconsistency [2509.24022]. The validation set contains **5 driving scenes**, **8 scenes with local motion**, and **2 static scenes**.

## 3. Paired modalities and controlled ISP comparison

The defining feature of Raw-Rain is that it publishes **paired 12-bit Bayer and bit-depth-matched sRGB rain scenes** derived from the **same sensor measurements** [2509.24022]. The raw side is the directly captured Bayer mosaic. The sRGB side is **not** obtained from a separate camera mode; instead, it is produced by a **software ISP pipeline** applied to those raw captures. This ISP performs, in order, **black level subtraction, demosaicing, lens shading, white balancing, color correction, global tone mapping, local tone mapping, and gamma**, and converts Bayer data to **12-bit sRGB** while **omitting quantization for fairer comparison** [2509.24022]. The comparison is therefore controlled: both raw-domain and RGB-domain pipelines begin from the same rainy Bayer measurements and differ mainly in whether deraining is applied **before** or **after** ISP.

The benchmark protocol supports **single-image deraining in raw Bayer**, **single-image deraining in sRGB**, and **stereo versus monocular** comparisons within each domain [2509.24022]. Inputs are either **sRGB tensors of shape \((B,3,H,W)\)** or **Bayer tensors of shape \((B,1,H,W)\)**. The paper’s only explicit output formula is the residual reconstruction
\[
\hat I = \alpha_1 X + \alpha_2 \Delta,
\]
where \(X\) is the corrupted input, \(\Delta\) is the learned residual or rain component, and \(\alpha_1,\alpha_2\) are per-pixel coefficients estimated by the model [2509.24022].

To isolate the effect of domain rather than architecture, the study uses closely matched networks in Bayer and RGB settings: **UNET**, **MONOCULAR UNET**, and a **CBAM**-augmented U-Net [2509.24022]. The shared backbone is a **four-level encoder-decoder U-Net** with **stride-2 convolutions** for downsampling, **bilinear upsampling** in the decoder, and **two Conv-BN-ReLU blocks per level**. The only domain-specific architectural differences are the first and last layers: the RGB stem is **\(3 \to 32\)** channels, the Bayer stem is **\(1 \to 32\)**, and the output head restores either 3 RGB channels or 1 Bayer channel. In the CBAM variant, **CBAM blocks** are inserted at the bottleneck and after each decoder up-convolution, with a reported overhead of **0.06M parameters**, less than 4% of the total [2509.24022].

Training uses **AdamW**, initial learning rate **\(1 \times 10^{-4}\)**, **600k iterations**, **batch size 1**, **\(512 \times 512\)** crops, and **\(\mathcal{L}_1\)** loss, with runtime reported as about **8 hours on an RTX3090** [2509.24022]. The paper states that there is no special Bayer packing, no Bayer-preserving augmentation scheme, no demosaicing-aware loss, and no reported test-time augmentation.

## 4. ISP-last formulation and Information Conservation Score

Raw-Rain is designed to test an **ISP-last** hypothesis: the ISP irreversibly changes signal statistics in ways that make restoration more ill-posed than restoration directly in Bayer space [2509.24022]. The paper argues that demosaicing introduces interpolation artifacts and mixes neighboring measurements; white balance and color correction alter physically measured intensities; tone mapping and gamma compress dynamic range; and the overall post-ISP representation blurs fine detail and distorts color statistics. A specific qualitative claim is that rain can skew auto white-balance and color-correction statistics, so post-ISP reconstruction tends to preserve rainy-image color bias, whereas Bayer-domain deraining followed by ISP yields white-balance statistics closer to clean ground truth [2509.24022].

Because conventional full-reference metrics are confounded by ISP-induced color and tone changes, the paper introduces the **Information Conservation Score (ICS)** [2509.24022]. ICS is described as combining a spatial structural term, **MS-SSIM**, with a frequency-domain similarity term based on **KL divergence between normalized power spectra**. The normalized power spectrum is defined as
\[
\tilde{P}_X(u, v) = \frac{\left| \mathcal{F}\{I(x, y)\}(u, v) \right|^2}
{\sum\limits_{u,v} \left| \mathcal{F}\{I(x, y)\}(u, v) \right|^2 }.
\]
The paper invokes Parseval’s theorem,
\[
\sum_{x,y} I(x,y)^2 = \sum_{u,v} \left|\mathcal{F}\{I\}(u,v)\right|^2 = E_I,
\]
and interprets the normalized power spectrum as a probability mass function. The KL term is intended as a proxy for information loss, with a second-order approximation reported as
\[
D_{\text{KL}} \approx \frac12 \sum_{u,v} \frac{\bigl(p_{\hat I} - p_I\bigr)^2}{p_I}.
\]
In experiments the balancing coefficient is \(\lambda=\tfrac12\). The paper also notes a formatting inconsistency in the printed ICS equation: as typeset, the sign convention is not fully recoverable, even though the benchmark reports **ICS↑** and interprets larger values as better [2509.24022].

This metric design reflects a broader conceptual point. Raw-Rain is not only a dataset but also a benchmark argument that restoration quality should be judged by **information preservation under domain change**, not only by post-ISP pixelwise fidelity.

## 5. Empirical findings and human evaluation

The central empirical result on Raw-Rain is that Bayer-domain deraining generally outperforms matched sRGB-domain deraining on the held-out test split [2509.24022]. The paper’s abstract and conclusion summarize the headline gain as **up to +0.99 dB PSNR** and **+1.2% ICS**, while running faster with **half of the GFLOPs**. The stated reason for the FLOP reduction is that the raw model operates on a **single-channel Bayer input** rather than a **three-channel RGB input**, although the paper does not provide a separate FLOP table [2509.24022].

The average quantitative entries show that the strongest gains are reported for the **CBAM** configuration. In RGB, CBAM averages **46.884 / 0.970 / 0.908**; in Bayer, it averages **47.870 / 0.977 / 0.920**, corresponding almost exactly to the advertised **+0.99 dB PSNR** and **+1.2% ICS** [2509.24022]. For **UNET**, the average changes from **47.702 / 0.977 / 0.920** in RGB to **47.804 / 0.977 / 0.922** in Bayer. For **MONO**, it changes from **48.307 / 0.978 / 0.923** to **48.318 / 0.979 / 0.925**. These numbers indicate that the raw-domain advantage is consistent but architecture-dependent, with the most visible improvement appearing in the CBAM case [2509.24022].

The paper also reports a **double-blind two-alternative forced choice (2AFC) pairwise comparison with reference** [2509.24022]. Across **12 subjects**, the Bayer-pipeline reconstruction was selected as closer to the reference **76.4%** of the time, versus **23.6%** for the RGB pipeline. Among the benchmarked metrics, **ICS** aligned best with human selections at **77.2%**, compared with **73.4% for SSIM** and **71.7% for PSNR** [2509.24022]. A notable observation is **Scene 3**, where PSNR and SSIM suggest little or no improvement while visual inspection indicates reduced corruption; the paper argues that **ICS captures this improvement better** [2509.24022].

A common misconception is that Raw-Rain merely reaffirms that “better deraining architecture” wins. The benchmark is explicitly structured to argue the opposite: the same general network family is used in Bayer and RGB settings so that the measured advantage is attributed to **domain placement**, namely deraining **before** rather than **after** ISP [2509.24022].

## 6. Relation to adjacent benchmarks and open issues

Raw-Rain should be distinguished from several neighboring resources that are often grouped together under “real rain” research. The differences are primarily in **domain**, **pairing**, and **task definition**.

| Resource | Domain and data | Primary role |
|---|---|---|
| **Raw-Rain** [2509.24022] | Real rainy scenes, paired **12-bit Bayer** and matched **12-bit sRGB** | Controlled benchmark for pre-ISP vs post-ISP deraining |
| **“End-to-end Rain Streak Removal with RAW Images”** [2312.13304] | **Synthetic rainy RAW** with clean RGB targets | Joint RAW deraining and RAW processing |
| **RaidaR** [2104.04606] | Real rainy **RGB** street scenes with dense labels | Rainy-scene perception, not RAW or paired deraining |
| **“Rainy screens”** [2003.04742] | Indoor re-photographed rainy **RGB** pairs through wet glass | Scalable proxy paired data, not native outdoor RAW rain |
| **HydroViews / DeRainGS** [2408.11540] | Synthesized and real rainy **RGB** multi-view scenes | 3D reconstruction in rainy environments |

This comparison clarifies Raw-Rain’s niche. It is not a large-scale rainy driving annotation resource like RaidaR, which contains **58,542 rainy images** and dense semantic or instance annotations but no RAW data or clean paired supervision [2104.04606]. It is not an indoor proxy collection like “Rainy screens,” which produces paired rainy/clear RGB data by re-photographing a display through a wet pane, but not native raw outdoor rainy capture [2003.04742]. It is also not a multi-view rainy-scene reconstruction dataset such as HydroViews, which targets **3D Reconstruction in Rainy Environments (3DRRE)** in RGB space rather than raw-domain restoration [2408.11540].

Several caveats remain explicit in the Raw-Rain paper itself. The benchmark is real and paired, but the text does **not** specify a full registration pipeline beyond saying that a rig ensured spatial stability [2509.24022]. The validation split lacks ground truth and is used only for visual testing. Rain is produced with a controlled **rain system**, so the weather is realistic but not purely naturally occurring. The benchmark is also stereo and **window-/vehicle-centric**, which may bias it toward through-glass imaging and automotive scenarios. Finally, the paper omits several technical details that would matter for exhaustive reproducibility, including exact sensor calibration, lens details, stereo baseline, frame rate, full ISP equations, and the finalized ICS sign convention [2509.24022].

The broader significance of Raw-Rain lies in the scope of its claim. The paper explicitly extends the **ISP-last** argument beyond deraining to other low-level restoration tasks such as **fog, low-light, motion blur**, and to **end-to-end learnable camera pipelines** [2509.24022]. This suggests that Raw-Rain is not merely a benchmark for one weather artifact; it is a concrete empirical testbed for a more general proposition in computational imaging and restoration: if the ISP destroys information useful for low-level vision, then restoration should move upstream into the sensor domain whenever raw measurements are available.

Source: https://www.emergentmind.com/topics/raw-rain