RainfallBench Nowcasting Benchmark

• RainfallBench is a specialized benchmark designed to evaluate short-horizon rainfall nowcasting by leveraging historical station data and GNSS-derived precipitable water vapor.
• It compiles 15-minute interval data from over 12,000 global GNSS stations (2018–2022) with key meteorological variables for multivariate-to-univariate forecasting.
• The evaluation framework spans 10 forecast settings, emphasizing both general and extreme rainfall prediction while highlighting robust RNN-based model performance.

Searching arXiv for the main benchmark paper and closely related rainfall benchmark papers to ground the article in current literature. arXiv search query: RainfallBench benchmark rainfall nowcasting PWV (Zhang et al., 28 Sep 2025) arXiv search query: RainBench precipitation forecasting benchmark (Witt et al., 2020) WeatherBench Probability (Garg et al., 2022) PostRainBench (Tang et al., 2023) KoMet (Kim et al., 2022) RainfallBench is a benchmark for rainfall nowcasting designed to evaluate short-horizon precipitation prediction from historical numerical meteorological records, explicitly incorporating GNSS-derived precipitable water vapor (PWV). It is presented as a rainfall-centric benchmark rather than a generic meteorological forecasting dataset, and formulates the task as multivariate-input, univariate-output forecasting of future rainfall within the next 0–3 hours using observations recorded at 15-minute intervals from more than 12,000 GNSS stations globally over 2018–2022 (Zhang et al., 28 Sep 2025).

1. Conceptual scope and benchmark identity

RainfallBench is introduced to address a gap in time-series forecasting benchmarks for meteorology: many existing benchmarks emphasize smoother or more periodic variables such as temperature and humidity, whereas rainfall nowcasting is characterized by zero inflation, temporal decay, non-stationarity, sparsity, and difficulty in extreme-event prediction. The benchmark is positioned as the first benchmark dedicated to rainfall nowcasting solely based on historical numerical meteorological records, explicitly incorporating PWV, and it is intended to test whether modern sequence models can nowcast rainfall from numerical station observations alone rather than from radar or satellite imagery (Zhang et al., 28 Sep 2025).

The benchmark’s task definition is multivariate-to-univariate. Given a historical multivariate sequence

$$\mathbf{X} = \{ \mathbf{x}_t \}_{t=1}^{T}, \quad \mathbf{x}_t \in \mathbb{R}^D,$$

the objective is to predict the future rainfall sequence

$$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$

In this formulation, the inputs contain meteorological covariates and past rainfall, while the output contains only future rainfall. This makes RainfallBench a direct nowcasting benchmark rather than a proxy task such as radar extrapolation or image sequence prediction (Zhang et al., 28 Sep 2025).

A defining feature is the explicit use of PWV. The paper frames PWV as a crucial indicator of rainfall that is absent in other datasets. This emphasis gives RainfallBench a distinct place within the precipitation-benchmark landscape: it evaluates localized numerical time series with an atmospheric-moisture variable derived from GNSS rather than gridded reanalysis-only predictors or remote-sensing imagery (Zhang et al., 28 Sep 2025).

2. Dataset composition and observational basis

RainfallBench combines meteorological observations from more than 12,000 GNSS stations globally, GNSS-derived PWV, and ERA5 reanalysis total precipitation for the target variable. The benchmark spans 2018–2022, with all variables recorded at 15-minute intervals. Each record contains six variables excluding timestamp: temperature at 2 meters above ground (t2m), surface pressure (sp), relative humidity (rh), wind speed (wind_speed), precipitable water vapor (PWV), and total precipitation (tp) (Zhang et al., 28 Sep 2025).

The paper states that tp is obtained from ERA5 reanalysis data. This suggests that RainfallBench evaluates rainfall nowcasting with station-side meteorological covariates and a reanalysis precipitation target rather than a purely gauge-only rainfall label. The benchmark is nevertheless described as being built from real-world GNSS-station observations, with PWV retrieved by inverting GNSS signal delays based on their proportional relationship with atmospheric water vapor (Zhang et al., 28 Sep 2025).

A representative dataset used throughout the main paper is the station JFNG. For JFNG, the exact time span is from February 14, 2018 00:00 to December 31, 2022 23:00, with 171,070 time steps and no missing entries. The paper states that JFNG is used as the primary representative dataset for analysis and experiments, while statistics for other stations are deferred to supplementary material. This means that the benchmark is globally sourced in construction, but the main empirical exposition is centered on a representative station (Zhang et al., 28 Sep 2025).

The train/validation/test split is 7:1:2. The paper also states that all reported results are computed using de-normalized actual rainfall values. The exact normalization formula is not specified in the provided text, although normalization is clearly applied during model development (Zhang et al., 28 Sep 2025).

Aspect	Specification
Time span	2018–2022
Sampling interval	15 minutes
Stations	More than 12,000 GNSS stations globally
Variables	t2m, sp, rh, wind_speed, PWV, tp
Target	tp (total precipitation)
Split	7:1:2

3. Task configuration and statistical difficulty

RainfallBench evaluates multiple forecasting settings formed by the Cartesian product of two input lengths and five output lengths:

$$\mathcal{L}_{\text{in}} = \{24, 48\}, \qquad \mathcal{L}_{\text{out}} = \{4, 6, 8, 10, 12\}.$$

At 15-minute resolution, input lengths of 24 and 48 correspond to 6 and 12 hours of history, while output lengths of 4, 6, 8, 10, and 12 correspond to 1, 1.5, 2, 2.5, and 3 hours of forecast horizon. The benchmark therefore defines 10 forecasting settings in total (Zhang et al., 28 Sep 2025).

The paper motivates RainfallBench by highlighting three statistical properties. First, it is zero-inflated. In the JFNG dataset, 121,572 records have $tp = 0$, accounting for 71% of the dataset. Second, it exhibits temporal dependency decay, characterized by the approximation

$\rho(k) \approx e^{-\lambda k}, \quad \lambda > 0.$

Third, it is non-stationary, assessed with the Augmented Dickey–Fuller-style regression

$$\Delta x_t = \alpha + \beta t + \gamma x_{t-1} + \sum_{i=1}^{p} \delta_i \Delta x_{t-i} + \varepsilon_t,$$

under the null hypothesis

$H_0 : \gamma = 0.$

The reported p-value is 0.4381, so the null cannot be rejected, which the paper interprets as evidence of non-stationarity (Zhang et al., 28 Sep 2025).

PWV is central to the benchmark’s physical motivation. In the correlation analysis using Pearson, Kendall, and Spearman coefficients, PWV shows the strongest positive correlation with tp; the example value given is a Pearson coefficient of 0.27. By contrast, t2m and sp show weak or negative correlations. The paper therefore identifies PWV as the most reliable indicator of imminent rainfall among the included variables (Zhang et al., 28 Sep 2025).

RainfallBench also defines a dedicated extreme rainfall evaluation. Following the T/CMSA 0013-2019 standard, extreme rainfall is defined as any 15-minute period with precipitation exceeding 2 mm. If $E$ denotes the set of extreme-rainfall time steps, the paper evaluates

$$\text{MSE}_{\text{extreme}} = \frac{1}{|E|} \sum_{t \in E} (\hat{y}_t - y_t)^2,$$

and

$$\text{MAE}_{\text{extreme}} = \frac{1}{|E|} \sum_{t \in E} |\hat{y}_t - y_t|.$$

This makes extreme-event performance an explicit benchmark axis rather than a secondary diagnostic (Zhang et al., 28 Sep 2025).

4. Evaluation protocol and benchmarked model families

RainfallBench uses MSE and MAE as its primary point-forecasting metrics and reports average leaderboard rankings over all input-output configurations. It additionally reports extreme-event versions of MSE and MAE restricted to the time steps satisfying the benchmark’s extreme-rainfall definition. The paper does not define a custom challenge metric beyond these general and extreme-restricted error measures (Zhang et al., 28 Sep 2025).

The benchmark evaluates 20 state-of-the-art models organized into six architectural families. The MLP-based family includes DLinear, Koopa, TimeMixer, and FilterTS. The RNN-based family includes SegRNN, xLSTM, and P-sLSTM. The CNN/TCN-based family includes TimesNet, TimeMixer++, and xPatch. The GNN-based family includes MSGNet and TimeFilter. The KAN-based family includes TimeKAN and MMK. The Transformer-based family includes Informer, PatchTST, iTransformer, TimeXer, PPDformer, and Informer + BFPF (Zhang et al., 28 Sep 2025).

This architectural range is a substantive part of the benchmark design. Rather than evaluating a narrow family of methods, RainfallBench is structured to test whether sequence models originally developed for smoother, more periodic time-series regimes remain competitive under rainfall-specific sparsity, recency sensitivity, and heavy-event rarity. The paper’s broader claim is that conventional benchmark rankings in generic time-series forecasting do not transfer automatically to rainfall nowcasting (Zhang et al., 28 Sep 2025).

5. Bi-Focus Precipitation Forecaster

To address zero inflation and temporal decay, the benchmark paper introduces the Bi-Focus Precipitation Forecaster (BFPF) as a plug-and-play module for Transformer-based models. BFPF contains two components: Non-Zero Focus, which emphasizes sparse but informative rainfall-related signals, and Temporal Focus, which biases attention toward more recent observations (Zhang et al., 28 Sep 2025).

In the Non-Zero Focus component, the benchmark defines the distance from a time step to the nearest zero-valued entry as

$$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$0

where $$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$1 and $$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$2 are the nearest zero positions to the left and right. A proximity weight is then constructed as

$$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$3

and used as an additive bias in the attention score tensor:

$$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$4

The stated intent is to encourage greater attention to non-zero entries and rainfall onsets embedded in long dry intervals (Zhang et al., 28 Sep 2025).

In the Temporal Focus component, the benchmark defines a linearly increasing positional bias

$$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$5

which is added to the attention tensor as

$$\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.$$6

Because the bias increases with position index, more recent keys receive larger attention bias. The paper presents this as an inductive prior aligned with the observed temporal decay of rainfall dependence (Zhang et al., 28 Sep 2025).

The benchmark further provides ablation evidence. Relative to vanilla Informer, Temporal Focus only, Non-Zero Focus only, and the full BFPF each improve some settings, while the combination of both modules yields the most balanced overall gains across horizons. The method section frames BFPF as plug-and-play, but the technical instantiation in the paper is specifically at the Transformer attention-score level (Zhang et al., 28 Sep 2025).

6. Empirical results, benchmark implications, and relation to adjacent resources

The principal empirical result is that RNN-based models dominate RainfallBench. The paper explicitly notes that two of the top three models are RNN-based and that RNNs appear especially well matched to rainfall’s short-memory, sparse, and recency-weighted structure. P-sLSTM is the best overall model across the main benchmark table, with representative results including 0.0083 / 0.0228 for 24(4) and 0.0413 / 0.0532 for 48(12) in MSE/MAE. SegRNN is also strong, with the best MSE at 48(8) and the best MAE at 48(10). Among Transformer-based models, Informer + BFPF is the strongest, and the paper states that it performs best for extreme rainfall (Zhang et al., 28 Sep 2025).

The benchmark also confirms several scaling properties. Errors increase monotonically with forecast horizon, and increasing input length from 24 to 48 generally reduces MSE. PWV ablation shows that including PWV usually lowers forecasting error; for example, under input length 48, the model with PWV outperforms the version without PWV at 48(4), 48(6), 48(8), and 48(12) in MSE, and at 48(4), 48(6), and 48(8) in MAE. The paper summarizes this as a significant reduction in nowcasting errors (Zhang et al., 28 Sep 2025).

Several limitations are explicit. Although the benchmark claims global station coverage, the main text centers experiments on the representative station JFNG. Region-wise robustness and transferability across stations are identified as future directions. The main text also provides limited detail on normalization and quality control, and BFPF, though described as plug-and-play, is practically implemented only for Transformer attention. A plausible implication is that RainfallBench is strongest as a benchmark for rainfall-specific sequence modeling behavior and less complete as a geographically stratified, fully specified operational benchmark (Zhang et al., 28 Sep 2025).

Within the broader literature, RainfallBench occupies a distinct niche. It differs from “RainBench” (Witt et al., 2020), which targets global precipitation forecasting from simulated satellite imagery, and from “PostRainBench” (Tang et al., 2023) and KoMet (Kim et al., 2022), which benchmark NWP post-processing. It is also complementary to “WeatherBench Probability” (Garg et al., 2022), which provides a probabilistic benchmarking scaffold but is not rainfall-specific, and to “RainShift” (Harder et al., 7 Jul 2025), which evaluates precipitation downscaling under geographic distribution shift. RainfallBench is therefore best understood as a specialized time-series nowcasting benchmark: short-horizon, station-based in covariates, explicitly PWV-aware, and designed to expose the distinctive statistical pathologies of rainfall prediction rather than the smoother dynamics emphasized in generic forecasting benchmarks (Witt et al., 2020, Garg et al., 2022, Tang et al., 2023, Kim et al., 2022, Harder et al., 7 Jul 2025).