Raw-Rain: Benchmark for Raw-Domain Deraining

• Raw-Rain is a benchmark that provides paired 12-bit Bayer and sRGB images for deraining evaluation before and after ISP processing.
• It uses a dual-camera stereo rig to capture diverse scenes under varying rain intensities and environmental conditions for realistic benchmarking.
• The dataset supports controlled comparisons with matched architectures, demonstrating that raw-domain deraining improves PSNR, ICS scores, and computational efficiency.

Searching arXiv for the benchmark and closely related raw-domain and rain-restoration papers. Raw-Rain is a benchmark for rainy-scene image reconstruction and deraining that was introduced by “$\mathbf{R}^3$: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain” as the first public benchmark of real rainy scenes captured in both 12-bit Bayer and bit-depth-matched sRGB (Rothschild et al., 28 Sep 2025). Its central purpose is to compare restoration before and after the image signal processor (ISP), and thereby test the claim that deraining directly on raw Bayer sensor measurements preserves recoverable scene information that post-ISP sRGB processing irreversibly alters. In the literature on rain removal, Raw-Rain occupies a specific position: it is neither a purely synthetic rain generator nor a generic real-rain RGB dataset, but a paired raw/sRGB empirical basis for evaluating an ISP-last restoration pipeline under real rainy capture conditions (Rothschild et al., 28 Sep 2025).

1. Historical placement within rain-image research

Raw-Rain emerged from a line of work in which most rain synthesis and deraining methods operated in processed image space rather than in sensor space. Early exemplar-based synthesis transferred real rain streak appearance by extracting rainy patches from an exemplar, computing residual rain patches as $P^o = P^r - \operatorname{mean}(P^r)$, and adding them to clean targets as $P^s = P^o + P^t$; the method explicitly preserved real streak appearance but did not model true sensor-RAW rain formation (Son et al., 2016). Model-based RGB deraining then treated rainy images through additive decompositions such as $I = I_R + I_{NR}$, with hand-crafted rain detection and gradient-domain quasi-sparsity priors, again in post-ISP image space rather than raw sensor space (Wang et al., 2018). Real-rain supervision was later approached by data distillation, where rain estimated from real rainy RGB images was transferred onto unrelated clean RGB images to form realistic training pairs without conventional software rain rendering (Lin et al., 2019).

A more direct precursor to Raw-Rain in the strict raw-domain sense is “End-to-end Rain Streak Removal with RAW Images” (Du et al., 2023). That work argued that rain removal should occur before demosaicing, white balance, exposure scaling, tone mapping, gamma, and other ISP stages, and proposed a joint solution for rain removal and RAW processing that maps rainy RAW input to clean RGB output. However, its supervision depended on synthetic rainy RAW generation obtained by unprocessing color rain streaks into RAW space, rather than on a public benchmark of real rainy scenes captured in both raw and matched sRGB (Du et al., 2023). Raw-Rain closes precisely that gap.

This historical trajectory suggests a shift in problem formulation. Earlier work largely asked how to synthesize or remove rain in RGB images; Raw-Rain asks whether the domain of restoration itself should move upstream to linear Bayer measurements, where ISP-induced clipping, channel mixing, tone compression, and interpolation have not yet occurred (Rothschild et al., 28 Sep 2025).

2. Acquisition protocol and dataset composition

The Raw-Rain dataset is also referred to as Raw-Rain-Stereo (RRS) and is acquired with a dual-camera stereo rig using two synchronized FLIR Blackfly S 2.8MP cameras, each capturing 12-bit Bayer (RGGB) frames at $1080 \times 1920$ resolution (Rothschild et al., 28 Sep 2025). All frames were recorded with fixed analog gain = 1, while exposure times varied from 300 ms to 10,000 ms to cover illumination conditions including daylight and low-light or evening scenes. The paper does not provide sensor model numbers, focal lengths, stereo baseline, frame rate, shutter synchronization details, or explicit geometric calibration details (Rothschild et al., 28 Sep 2025).

The acquisition protocol is designed to create realistic rain corruption rather than synthetic overlays. A rain system scatters water-rain-like droplets on the windshield and in the depth of the scene. Clean ground truth is captured before triggering the rain system, and rainy observations are captured immediately after; a rig is used to ensure spatial stability throughout the process (Rothschild et al., 28 Sep 2025). The paper further states that some scenes were recorded through a stationary glass window, others from within a moving vehicle, and some included windshield wipers operating at different speeds, broadening the benchmark beyond static outdoor imagery.

In composition, the dataset contains 89 diverse training scenes, 30 identity training scenes, 15 validation scenes, and 10 held-out test scenes (Rothschild et al., 28 Sep 2025). Each scene is a 300-frame video sequence for each camera, producing 48,000 training frames, 18,000 identity frames, 9,000 validation frames, and 6,000 test frames. The validation split is explicitly marked “no GT” and used for visual testing only. Rain intensity is categorized as light, medium, or heavy; the training set contains 47 light, 31 medium, and 11 heavy rain scenes, while the test set contains 2 light, 6 medium, and 2 heavy scenes (Rothschild et al., 28 Sep 2025).

The benchmark also records scene categories and operating conditions. Training locations are summarized as 17 forest, 29 city, 30 playground, and 13 parking lot scenes. The train scene settings are reported as 30 behind-glass window, 42 within a vehicle, and 14 in a vehicle with wipers; these counts do not exactly sum to 89, and the paper does not clarify whether this reflects overlap or reporting inconsistency (Rothschild et al., 28 Sep 2025). The validation set contains 5 driving scenes, 8 scenes with local motion, and 2 static scenes.

3. Paired modalities and controlled ISP comparison

The defining feature of Raw-Rain is that it publishes paired 12-bit Bayer and bit-depth-matched sRGB rain scenes derived from the same sensor measurements (Rothschild et al., 28 Sep 2025). The raw side is the directly captured Bayer mosaic. The sRGB side is not obtained from a separate camera mode; instead, it is produced by a software ISP pipeline applied to those raw captures. This ISP performs, in order, black level subtraction, demosaicing, lens shading, white balancing, color correction, global tone mapping, local tone mapping, and gamma, and converts Bayer data to 12-bit sRGB while omitting quantization for fairer comparison (Rothschild et al., 28 Sep 2025). The comparison is therefore controlled: both raw-domain and RGB-domain pipelines begin from the same rainy Bayer measurements and differ mainly in whether deraining is applied before or after ISP.

The benchmark protocol supports single-image deraining in raw Bayer, single-image deraining in sRGB, and stereo versus monocular comparisons within each domain (Rothschild et al., 28 Sep 2025). Inputs are either sRGB tensors of shape $(B,3,H,W)$ or Bayer tensors of shape $(B,1,H,W)$. The paper’s only explicit output formula is the residual reconstruction

$\hat I = \alpha_1 X + \alpha_2 \Delta,$

where $X$ is the corrupted input, $\Delta$ is the learned residual or rain component, and $P^o = P^r - \operatorname{mean}(P^r)$0 are per-pixel coefficients estimated by the model (Rothschild et al., 28 Sep 2025).

To isolate the effect of domain rather than architecture, the study uses closely matched networks in Bayer and RGB settings: UNET, MONOCULAR UNET, and a CBAM-augmented U-Net (Rothschild et al., 28 Sep 2025). The shared backbone is a four-level encoder-decoder U-Net with stride-2 convolutions for downsampling, bilinear upsampling in the decoder, and two Conv-BN-ReLU blocks per level. The only domain-specific architectural differences are the first and last layers: the RGB stem is $P^o = P^r - \operatorname{mean}(P^r)$1 channels, the Bayer stem is $P^o = P^r - \operatorname{mean}(P^r)$2, and the output head restores either 3 RGB channels or 1 Bayer channel. In the CBAM variant, CBAM blocks are inserted at the bottleneck and after each decoder up-convolution, with a reported overhead of 0.06M parameters, less than 4% of the total (Rothschild et al., 28 Sep 2025).

Training uses AdamW, initial learning rate $P^o = P^r - \operatorname{mean}(P^r)$3, 600k iterations, batch size 1, $P^o = P^r - \operatorname{mean}(P^r)$4 crops, and $P^o = P^r - \operatorname{mean}(P^r)$5 loss, with runtime reported as about 8 hours on an RTX3090 (Rothschild et al., 28 Sep 2025). The paper states that there is no special Bayer packing, no Bayer-preserving augmentation scheme, no demosaicing-aware loss, and no reported test-time augmentation.

4. ISP-last formulation and Information Conservation Score

Raw-Rain is designed to test an ISP-last hypothesis: the ISP irreversibly changes signal statistics in ways that make restoration more ill-posed than restoration directly in Bayer space (Rothschild et al., 28 Sep 2025). The paper argues that demosaicing introduces interpolation artifacts and mixes neighboring measurements; white balance and color correction alter physically measured intensities; tone mapping and gamma compress dynamic range; and the overall post-ISP representation blurs fine detail and distorts color statistics. A specific qualitative claim is that rain can skew auto white-balance and color-correction statistics, so post-ISP reconstruction tends to preserve rainy-image color bias, whereas Bayer-domain deraining followed by ISP yields white-balance statistics closer to clean ground truth (Rothschild et al., 28 Sep 2025).

Because conventional full-reference metrics are confounded by ISP-induced color and tone changes, the paper introduces the Information Conservation Score (ICS) (Rothschild et al., 28 Sep 2025). ICS is described as combining a spatial structural term, MS-SSIM, with a frequency-domain similarity term based on KL divergence between normalized power spectra. The normalized power spectrum is defined as

$P^o = P^r - \operatorname{mean}(P^r)$6

The paper invokes Parseval’s theorem,

$P^o = P^r - \operatorname{mean}(P^r)$7

and interprets the normalized power spectrum as a probability mass function. The KL term is intended as a proxy for information loss, with a second-order approximation reported as

$P^o = P^r - \operatorname{mean}(P^r)$8

In experiments the balancing coefficient is $P^o = P^r - \operatorname{mean}(P^r)$9. The paper also notes a formatting inconsistency in the printed ICS equation: as typeset, the sign convention is not fully recoverable, even though the benchmark reports ICS↑ and interprets larger values as better (Rothschild et al., 28 Sep 2025).

This metric design reflects a broader conceptual point. Raw-Rain is not only a dataset but also a benchmark argument that restoration quality should be judged by information preservation under domain change, not only by post-ISP pixelwise fidelity.

5. Empirical findings and human evaluation

The central empirical result on Raw-Rain is that Bayer-domain deraining generally outperforms matched sRGB-domain deraining on the held-out test split (Rothschild et al., 28 Sep 2025). The paper’s abstract and conclusion summarize the headline gain as up to +0.99 dB PSNR and +1.2% ICS, while running faster with half of the GFLOPs. The stated reason for the FLOP reduction is that the raw model operates on a single-channel Bayer input rather than a three-channel RGB input, although the paper does not provide a separate FLOP table (Rothschild et al., 28 Sep 2025).

The average quantitative entries show that the strongest gains are reported for the CBAM configuration. In RGB, CBAM averages 46.884 / 0.970 / 0.908; in Bayer, it averages 47.870 / 0.977 / 0.920, corresponding almost exactly to the advertised +0.99 dB PSNR and +1.2% ICS (Rothschild et al., 28 Sep 2025). For UNET, the average changes from 47.702 / 0.977 / 0.920 in RGB to 47.804 / 0.977 / 0.922 in Bayer. For MONO, it changes from 48.307 / 0.978 / 0.923 to 48.318 / 0.979 / 0.925. These numbers indicate that the raw-domain advantage is consistent but architecture-dependent, with the most visible improvement appearing in the CBAM case (Rothschild et al., 28 Sep 2025).

The paper also reports a double-blind two-alternative forced choice (2AFC) pairwise comparison with reference (Rothschild et al., 28 Sep 2025). Across 12 subjects, the Bayer-pipeline reconstruction was selected as closer to the reference 76.4% of the time, versus 23.6% for the RGB pipeline. Among the benchmarked metrics, ICS aligned best with human selections at 77.2%, compared with 73.4% for SSIM and 71.7% for PSNR (Rothschild et al., 28 Sep 2025). A notable observation is Scene 3, where PSNR and SSIM suggest little or no improvement while visual inspection indicates reduced corruption; the paper argues that ICS captures this improvement better (Rothschild et al., 28 Sep 2025).

A common misconception is that Raw-Rain merely reaffirms that “better deraining architecture” wins. The benchmark is explicitly structured to argue the opposite: the same general network family is used in Bayer and RGB settings so that the measured advantage is attributed to domain placement, namely deraining before rather than after ISP (Rothschild et al., 28 Sep 2025).

6. Relation to adjacent benchmarks and open issues

Raw-Rain should be distinguished from several neighboring resources that are often grouped together under “real rain” research. The differences are primarily in domain, pairing, and task definition.

Resource	Domain and data	Primary role
Raw-Rain (Rothschild et al., 28 Sep 2025)	Real rainy scenes, paired 12-bit Bayer and matched 12-bit sRGB	Controlled benchmark for pre-ISP vs post-ISP deraining
“End-to-end Rain Streak Removal with RAW Images” (Du et al., 2023)	Synthetic rainy RAW with clean RGB targets	Joint RAW deraining and RAW processing
RaidaR (Jin et al., 2021)	Real rainy RGB street scenes with dense labels	Rainy-scene perception, not RAW or paired deraining
“Rainy screens” (Porav et al., 2020)	Indoor re-photographed rainy RGB pairs through wet glass	Scalable proxy paired data, not native outdoor RAW rain
HydroViews / DeRainGS (Liu et al., 2024)	Synthesized and real rainy RGB multi-view scenes	3D reconstruction in rainy environments

This comparison clarifies Raw-Rain’s niche. It is not a large-scale rainy driving annotation resource like RaidaR, which contains 58,542 rainy images and dense semantic or instance annotations but no RAW data or clean paired supervision (Jin et al., 2021). It is not an indoor proxy collection like “Rainy screens,” which produces paired rainy/clear RGB data by re-photographing a display through a wet pane, but not native raw outdoor rainy capture (Porav et al., 2020). It is also not a multi-view rainy-scene reconstruction dataset such as HydroViews, which targets 3D Reconstruction in Rainy Environments (3DRRE) in RGB space rather than raw-domain restoration (Liu et al., 2024).

Several caveats remain explicit in the Raw-Rain paper itself. The benchmark is real and paired, but the text does not specify a full registration pipeline beyond saying that a rig ensured spatial stability (Rothschild et al., 28 Sep 2025). The validation split lacks ground truth and is used only for visual testing. Rain is produced with a controlled rain system, so the weather is realistic but not purely naturally occurring. The benchmark is also stereo and window-/vehicle-centric, which may bias it toward through-glass imaging and automotive scenarios. Finally, the paper omits several technical details that would matter for exhaustive reproducibility, including exact sensor calibration, lens details, stereo baseline, frame rate, full ISP equations, and the finalized ICS sign convention (Rothschild et al., 28 Sep 2025).

The broader significance of Raw-Rain lies in the scope of its claim. The paper explicitly extends the ISP-last argument beyond deraining to other low-level restoration tasks such as fog, low-light, motion blur, and to end-to-end learnable camera pipelines (Rothschild et al., 28 Sep 2025). This suggests that Raw-Rain is not merely a benchmark for one weather artifact; it is a concrete empirical testbed for a more general proposition in computational imaging and restoration: if the ISP destroys information useful for low-level vision, then restoration should move upstream into the sensor domain whenever raw measurements are available.