RainShift: Precipitation Downscaling Benchmark

• RainShift is a benchmark for precipitation downscaling that measures cross-geography generalization using probabilistic metrics such as CRPS, PSNR, and SSIM.
• It integrates low-resolution ERA5 inputs, high-resolution NASA GPM IMERG targets, and invariant geographic features to simulate realistic climate variability.
• The benchmark also evaluates domain alignment methods like quantile mapping to mitigate performance degradation from Global North to Global South transitions.

Searching arXiv for RainShift benchmark and closely related precipitation-shift work. RainShift is a dataset and benchmark for evaluating precipitation downscaling under geographic distribution shifts. It is framed around the problem that Earth System Models are the main tool for projecting climate-change impacts, yet cannot feasibly be run at resolutions sufficient for local-scale risk assessments, while deep learning-based super-resolution models typically require retraining for each geographical area because climatic processes vary regionally and high-resolution observational data are unevenly available across the globe (Harder et al., 7 Jul 2025). RainShift therefore targets cross-geography generalization: models are trained on data-rich regions in the Global North and evaluated zero-shot on data-sparse regions in the Global South, with the aim of quantifying how downscaling performance degrades under out-of-distribution geographic shift and which modeling or alignment strategies partially mitigate that degradation (Harder et al., 7 Jul 2025).

1. Definition and scope

RainShift is explicitly introduced as “a benchmark for precipitation downscaling across geographies” (Harder et al., 7 Jul 2025). Its central task is probabilistic downscaling: given low-resolution meteorological inputs and invariant geographic covariates, a model is required to approximate the conditional distribution of higher-resolution precipitation fields,

$$G(x)\sim p_G(\cdot\mid x), \qquad p_G(\cdot\mid x)\approx p(\,\cdot\mid x),$$

where $y\in\mathbb{R}^{200\times200}$, $x_\ell\in\mathbb{R}^{9\times80\times80}$, $x_h\in\mathbb{R}^{2\times200\times200}$, and the upsampling factor is $N=200/80=2.5$ (Harder et al., 7 Jul 2025).

The benchmark is motivated by a particular failure mode of learned downscaling systems: they can perform competitively in-distribution yet degrade when transferred to climatically distinct regions. RainShift formalizes this issue through train/test splits in which the training and evaluation regions are disjoint and geographically differentiated, specifically contrasting dense observational coverage in the Global North with data-sparse areas in the Global South (Harder et al., 7 Jul 2025). The benchmark therefore addresses not only super-resolution fidelity, but also the broader question of equitable access to high-resolution climate information across unevenly observed regions (Harder et al., 7 Jul 2025).

A broader usage of the term “RainShift” also appears in adjacent literature as shorthand for rain-related distribution shift problems in other domains, including rainy street-scene perception and precipitation nowcasting under non-stationary meteorological distributions (Jin et al., 2021, Di et al., 4 Jan 2026, Sun et al., 28 Jan 2026). In RainShift proper, however, the task is specifically geographic generalization in climate-scale precipitation downscaling (Harder et al., 7 Jul 2025).

2. Dataset construction and geographical design

RainShift combines low-resolution atmospheric inputs, high-resolution precipitation targets, and static geographic fields over a common spatial footprint. The low-resolution inputs $x_{\ell}$ are ERA5 reanalysis from ECMWF on a regular $0.25^\circ\times0.25^\circ$ grid, approximately $25$ km per pixel in mid-latitudes, sampled hourly over 2001–2022. These inputs are organized as $c=9$ channels on $80\times80$ patches spanning $y\in\mathbb{R}^{200\times200}$0 (Harder et al., 7 Jul 2025). The high-resolution targets $y\in\mathbb{R}^{200\times200}$1 are NASA GPM IMERG Final Run V07 precipitation at $y\in\mathbb{R}^{200\times200}$2, approximately $y\in\mathbb{R}^{200\times200}$3 km per pixel, originally half-hourly and aggregated to hourly over the same 2001–2022 period. These targets are single-channel $y\in\mathbb{R}^{200\times200}$4 patches over the same $y\in\mathbb{R}^{200\times200}$5 area (Harder et al., 7 Jul 2025). The invariant geographic features $y\in\mathbb{R}^{200\times200}$6 are land-sea mask and orography on the $y\in\mathbb{R}^{200\times200}$7 target grid (Harder et al., 7 Jul 2025).

The geographical split is intrinsic to the benchmark design. Training regions $y\in\mathbb{R}^{200\times200}$8 consist of twelve $y\in\mathbb{R}^{200\times200}$9 patches in the Global North selected for dense observational coverage, whereas evaluation regions $x_\ell\in\mathbb{R}^{9\times80\times80}$0 are six $x_\ell\in\mathbb{R}^{9\times80\times80}$1 patches in data-sparse areas of the Global South: Cape Horn, Amazon Basin, West Africa, Horn of Africa, Tibetan Plateau, and Melanesia (Harder et al., 7 Jul 2025). Four nested training scenarios $x_\ell\in\mathbb{R}^{9\times80\times80}$2 are defined by unions of the $x_\ell\in\mathbb{R}^{9\times80\times80}$3, allowing systematic study of how expansion of the source training domain affects zero-shot transfer (Harder et al., 7 Jul 2025).

RainShift also introduces explicit distributional notation. If $x_\ell\in\mathbb{R}^{9\times80\times80}$4 denotes samples from region $x_\ell\in\mathbb{R}^{9\times80\times80}$5, then the aggregated train and test input distributions are written as

$x_\ell\in\mathbb{R}^{9\times80\times80}$6

and region-wise mean precipitation is summarized by

$x_\ell\in\mathbb{R}^{9\times80\times80}$7

These definitions make the benchmark’s shift setting mathematically explicit: training and evaluation differ through geographically indexed underlying distributions rather than only through random holdout sampling (Harder et al., 7 Jul 2025).

3. Preprocessing and evaluation protocol

RainShift applies a four-stage preprocessing pipeline. First, ERA5 precipitation is clipped to $x_\ell\in\mathbb{R}^{9\times80\times80}$8 to mitigate spurious light rain and extreme outliers; values below IMERG’s minimum are set to zero and values above its maximum are clipped (Harder et al., 7 Jul 2025). Second, each variable is standardized via a global Z-score,

$x_\ell\in\mathbb{R}^{9\times80\times80}$9

where $x_h\in\mathbb{R}^{2\times200\times200}$0 are region-wise mean and variance and $x_h\in\mathbb{R}^{2\times200\times200}$1 (Harder et al., 7 Jul 2025). Third, a log-transform is applied to non-negative variables $x_h\in\mathbb{R}^{2\times200\times200}$2,

$x_h\in\mathbb{R}^{2\times200\times200}$3

Fourth, the resulting arrays are stored in Zarr format with 200-timestep chunks, approximately $x_h\in\mathbb{R}^{2\times200\times200}$4 MB for ERA5 and $x_h\in\mathbb{R}^{2\times200\times200}$5 MB for IMERG (Harder et al., 7 Jul 2025).

The benchmark uses both temporal and geographical separation. Temporally, models are trained on 2001–2020 and evaluated on 2021–2022, in a zero-shot setting with no target labels in evaluation (Harder et al., 7 Jul 2025). Geographically, the sub-tasks are $x_h\in\mathbb{R}^{2\times200\times200}$6 with $x_h\in\mathbb{R}^{2\times200\times200}$7, ensuring that generalization is genuinely cross-region rather than interpolative within a shared climate regime (Harder et al., 7 Jul 2025).

Performance is assessed with deterministic and probabilistic metrics. The Continuous Ranked Probability Score is defined as

$x_h\in\mathbb{R}^{2\times200\times200}$8

and reduces to mean absolute error for deterministic outputs (Harder et al., 7 Jul 2025). Peak Signal-to-Noise Ratio is defined by

$x_h\in\mathbb{R}^{2\times200\times200}$9

Structural Similarity Index is

$N=200/80=2.5$0

and Fractional Skill Score is

$N=200/80=2.5$1

with FSS ranging from $N=200/80=2.5$2 to $N=200/80=2.5$3 (Harder et al., 7 Jul 2025). The metric suite reflects both pointwise fidelity and spatial organization, which is particularly relevant for precipitation fields.

4. Baseline models and domain-alignment strategy

RainShift evaluates several baseline families spanning deterministic, adversarial, and diffusion-based formulations. The simplest baseline is bilinear interpolation of ERA5 precipitation to the $N=200/80=2.5$4 grid,

$N=200/80=2.5$5

used as a lower bound (Harder et al., 7 Jul 2025). The deterministic neural baseline is a fully convolutional residual network $N=200/80=2.5$6 trained by minimizing $N=200/80=2.5$7 loss,

$N=200/80=2.5$8

Adversarial modeling is represented by Wasserstein GAN with gradient penalty, with critic and generator losses

$N=200/80=2.5$9

$x_{\ell}$0

The diffusion baseline uses a score network $x_{\ell}$1 trained by denoising score matching,

$x_{\ell}$2

with a forward SDE $x_{\ell}$3 and a learned reverse SDE for sampling (Harder et al., 7 Jul 2025).

A central methodological component is domain alignment through quantile mapping applied to the low-resolution precipitation channel. To mitigate shifts where $x_{\ell}$4, RainShift uses

$x_{\ell}$5

where $x_{\ell}$6 are historical CDFs and $x_{\ell}$7 indexes the historical period; the multiplicative variant is used for non-negative variables (Harder et al., 7 Jul 2025). This is not presented as a complete solution to geographic shift, but as a concrete mechanism for pre-aligning marginal input distributions across regions.

The benchmark’s methodological framing yields two immediate comparisons. First, it distinguishes deterministic super-resolution from probabilistic downscaling, with the latter better matched to uncertainty in precipitation fields (Harder et al., 7 Jul 2025). Second, it separates raw model capacity from cross-domain alignment, showing that performance under shift depends not only on the generative architecture but also on how source and target distributions are brought into correspondence (Harder et al., 7 Jul 2025).

5. Empirical findings under geographic distribution shift

RainShift reports substantial out-of-distribution degradation across geographies. In absolute terms, ERA5 interpolation yields CRPS values ranging from $x_{\ell}$8 mm/h in Cape Horn to $x_{\ell}$9 mm/h in Melanesia (Harder et al., 7 Jul 2025). Under the smallest training scenario $0.25^\circ\times0.25^\circ$0, GAN achieves CRPS approximately $0.25^\circ\times0.25^\circ$1–$0.25^\circ\times0.25^\circ$2, diffusion model approximately $0.25^\circ\times0.25^\circ$3–$0.25^\circ\times0.25^\circ$4, while ResNet is unstable, including a case where CRPS tends to $0.25^\circ\times0.25^\circ$5 in the Tibetan Plateau (Harder et al., 7 Jul 2025). Under the largest training scenario $0.25^\circ\times0.25^\circ$6, GAN and diffusion model reach CRPS approximately $0.25^\circ\times0.25^\circ$7–$0.25^\circ\times0.25^\circ$8, substantially outperforming ResNet, which is around $0.25^\circ\times0.25^\circ$9–$25$0 (Harder et al., 7 Jul 2025).

Relative to interpolation, RainShift defines

$25$1

Under this measure, GAN and diffusion model provide consistent gains of $25$2–$25$3 across regions, except in two outlier cases, whereas ResNet provides only modest improvements and exhibits instabilities under strong shifts (Harder et al., 7 Jul 2025). To quantify transfer loss against in-region training, the benchmark defines

$25$4

where $25$5 is the score when training directly on $25$6. For $25$7, this out-of-distribution drop reaches up to $25$8; even with the broadest source domain $25$9, residual drops up to $c=9$0 remain, with model- and region-dependent variation (Harder et al., 7 Jul 2025).

The effect of enlarging the source domain is heterogeneous. Expanding $c=9$1 generally reduces CRPS, especially for GANs in high-rainfall regions such as the Amazon and Melanesia (Harder et al., 7 Jul 2025). However, some regions, specifically the Horn of Africa and the Tibetan Plateau, show diminishing or no gains from adding more North-American and East-Asian data (Harder et al., 7 Jul 2025). This suggests that source-domain diversity is beneficial but not sufficient when climatic regimes remain mismatched.

Quantile mapping improves CRPS in five of six regions for all models under the $c=9$2 setting (Harder et al., 7 Jul 2025). One reported example is the diffusion model in the Tibetan Plateau, improving from $c=9$3 to $c=9$4 mm/h, a $c=9$5 relative reduction (Harder et al., 7 Jul 2025). The benchmark therefore supports a nuanced conclusion: geographic shift is persistent, but some part of it is addressable through relatively simple input-space alignment.

6. Relation to adjacent “RainShift” problems

The RainShift benchmark belongs to a wider family of distribution-shift problems involving rain, precipitation, and transfer across environments, but its formulation is distinct. In autonomous driving, RaidaR addresses rainy street-scene perception by providing a real-rain annotated dataset containing $c=9$6 rainy images, $c=9$7 rainy semantic annotations, and $c=9$8 rainy instance annotations, together with unpaired rain–sun translation methods such as Masked CycleGAN and Masked GANHopper (Jin et al., 2021). That work states that incorporating RaidaR and masked translation-based synthetic images improves semantic and instance segmentation under rainy conditions and explicitly describes this as addressing the RainShift challenge (Jin et al., 2021). Here, “RainShift” refers to weather-induced visual domain shift in perception.

In precipitation nowcasting, REE-TTT studies cross-regional extreme precipitation scenarios and uses an adaptive Test-Time Training mechanism to handle non-stationary meteorological distributions (Di et al., 4 Jan 2026). Its ST-TTT block combines a motion-enhanced query projection, a linear training view, and temporal attention in a self-supervised inner loop, with reported gains in Beijing single-region testing and Hangzhou zero-shot generalization (Di et al., 4 Jan 2026). The same details text explicitly frames this as handling distribution shifts in precipitation, again referring to such tasks as “RainShift” (Di et al., 4 Jan 2026).

StormDiT extends the shift discussion to the “2–6 hour gray zone” in precipitation nowcasting, where extrapolation errors compound and numerical weather prediction remains too slow for storm-scale operational deployment (Sun et al., 28 Jan 2026). Its report links this limitation directly to RainShift as an effort to extend high-resolution precipitation forecasts beyond the traditional nowcasting horizon and into the $c=9$9–$80\times80$0 h regime (Sun et al., 28 Jan 2026). It further proposes several transferable ideas for RainShift, including unified latent-space generative modeling, causal spatiotemporal compression, rectified flow ODEs, global self-attention, probabilistic calibration via spread-skill analysis, and multi-modal extension (Sun et al., 28 Jan 2026).

These adjacent usages indicate that “RainShift” functions as a broader organizing concept for rain- or precipitation-related distribution shifts across multiple ML subfields. A plausible implication is that the benchmark definition in geographic downscaling (Harder et al., 7 Jul 2025) and the broader rain-shift usage in perception and nowcasting (Jin et al., 2021, Di et al., 4 Jan 2026, Sun et al., 28 Jan 2026) are converging on a common concern: whether models trained under one meteorological or geographical regime remain reliable under another.

7. Significance, limitations, and methodological implications

RainShift is described as the first large-scale benchmark explicitly designed to measure how well deep downscaling models generalize across geographic distribution shifts, particularly from data-rich Global North regions to data-sparse Global South regions (Harder et al., 7 Jul 2025). Its reported findings emphasize that geographic and climatic variability is the principal bottleneck to global generalization, more so than pure model architecture or data volume, and that probabilistic models such as GANs and diffusion models systematically outperform deterministic CNNs and interpolation in out-of-distribution settings (Harder et al., 7 Jul 2025).

The benchmark also motivates a set of best practices. It recommends probabilistic generative frameworks to capture precipitation uncertainty; domain-alignment methods such as quantile mapping or optimal-transport-based distribution matching; region-aware or meta-learning strategies such as location embeddings and task-informed meta-learning; physical constraints or hybrid physics-ML approaches; and selective augmentation of training data using climates analogous to the target region rather than uniform expansion from unrelated areas (Harder et al., 7 Jul 2025). Because only quantile mapping is quantitatively evaluated in the reported experiments, the remaining items are best read as methodological recommendations rather than validated benchmark conclusions.

A common misconception in geographically transferred downscaling is that simply enlarging the training corpus will eliminate domain shift. RainShift directly contradicts this: expanding $80\times80$1 to $80\times80$2 generally helps, but residual performance drops remain and some target regions exhibit little benefit from added source data (Harder et al., 7 Jul 2025). Another misconception is that deterministic super-resolution alone is sufficient if the network is large enough; the benchmark instead reports that deterministic ResNet can be unstable under strong shifts, whereas GAN and diffusion baselines remain more reliable (Harder et al., 7 Jul 2025).

Within the broader research landscape, RainShift can be interpreted as a benchmark that formalizes an increasingly central question in climate ML: not whether a model can downscale precipitation accurately somewhere, but whether it can do so where high-resolution information is most needed and least observed. The reported evidence suggests that geographic transfer is not a peripheral evaluation setting but the core criterion for real-world applicability in global precipitation downscaling (Harder et al., 7 Jul 2025).