---
title: RainfallBench Nowcasting Benchmark
url: https://www.emergentmind.com/topics/rainfallbench
type: topic
---

# RainfallBench Nowcasting Benchmark

Searching arXiv for the main benchmark paper and closely related rainfall benchmark papers to ground the article in current literature.
arXiv search query: RainfallBench benchmark rainfall nowcasting PWV 2509.25263
arXiv search query: RainBench precipitation forecasting benchmark 2012.09670 WeatherBench Probability 2205.00865 PostRainBench 2310.02676 KoMet 2206.15241
RainfallBench is a benchmark for rainfall nowcasting designed to evaluate short-horizon precipitation prediction from historical numerical meteorological records, explicitly incorporating GNSS-derived precipitable water vapor (PWV). It is presented as a rainfall-centric benchmark rather than a generic meteorological forecasting dataset, and formulates the task as multivariate-input, univariate-output forecasting of future rainfall within the next 0–3 hours using observations recorded at 15-minute intervals from more than 12,000 GNSS stations globally over 2018–2022 [2509.25263].

## 1. Conceptual scope and benchmark identity

RainfallBench is introduced to address a gap in time-series forecasting benchmarks for meteorology: many existing benchmarks emphasize smoother or more periodic variables such as temperature and humidity, whereas rainfall nowcasting is characterized by zero inflation, temporal decay, non-stationarity, sparsity, and difficulty in extreme-event prediction. The benchmark is positioned as the first benchmark dedicated to rainfall nowcasting solely based on historical numerical meteorological records, explicitly incorporating PWV, and it is intended to test whether modern sequence models can nowcast rainfall from numerical station observations alone rather than from radar or satellite imagery [2509.25263].

The benchmark’s task definition is multivariate-to-univariate. Given a historical multivariate sequence
$$
\mathbf{X} = \{ \mathbf{x}_t \}_{t=1}^{T}, \quad \mathbf{x}_t \in \mathbb{R}^D,
$$
the objective is to predict the future rainfall sequence
$$
\mathbf{y} = \{ y_{T+1}, y_{T+2}, \dots, y_{T+H} \}, \quad y_t \in \mathbb{R}.
$$
In this formulation, the inputs contain meteorological covariates and past rainfall, while the output contains only future rainfall. This makes RainfallBench a direct nowcasting benchmark rather than a proxy task such as radar extrapolation or image sequence prediction [2509.25263].

A defining feature is the explicit use of PWV. The paper frames PWV as a crucial indicator of rainfall that is absent in other datasets. This emphasis gives RainfallBench a distinct place within the precipitation-benchmark landscape: it evaluates localized numerical time series with an atmospheric-moisture variable derived from GNSS rather than gridded reanalysis-only predictors or remote-sensing imagery [2509.25263].

## 2. Dataset composition and observational basis

RainfallBench combines meteorological observations from more than 12,000 GNSS stations globally, GNSS-derived PWV, and ERA5 reanalysis total precipitation for the target variable. The benchmark spans 2018–2022, with all variables recorded at 15-minute intervals. Each record contains six variables excluding timestamp: temperature at 2 meters above ground (`t2m`), surface pressure (`sp`), relative humidity (`rh`), wind speed (`wind_speed`), precipitable water vapor (`PWV`), and total precipitation (`tp`) [2509.25263].

The paper states that `tp` is obtained from ERA5 reanalysis data. This suggests that RainfallBench evaluates rainfall nowcasting with station-side meteorological covariates and a reanalysis precipitation target rather than a purely gauge-only rainfall label. The benchmark is nevertheless described as being built from real-world GNSS-station observations, with PWV retrieved by inverting GNSS signal delays based on their proportional relationship with atmospheric water vapor [2509.25263].

A representative dataset used throughout the main paper is the station **JFNG**. For JFNG, the exact time span is from **February 14, 2018 00:00 to December 31, 2022 23:00**, with **171,070** time steps and **no missing entries**. The paper states that JFNG is used as the primary representative dataset for analysis and experiments, while statistics for other stations are deferred to supplementary material. This means that the benchmark is globally sourced in construction, but the main empirical exposition is centered on a representative station [2509.25263].

The train/validation/test split is **7:1:2**. The paper also states that all reported results are computed using de-normalized actual rainfall values. The exact normalization formula is not specified in the provided text, although normalization is clearly applied during model development [2509.25263].

| Aspect | Specification |
|---|---|
| Time span | 2018–2022 |
| Sampling interval | 15 minutes |
| Stations | More than 12,000 GNSS stations globally |
| Variables | `t2m`, `sp`, `rh`, `wind_speed`, `PWV`, `tp` |
| Target | `tp` (total precipitation) |
| Split | 7:1:2 |

## 3. Task configuration and statistical difficulty

RainfallBench evaluates multiple forecasting settings formed by the Cartesian product of two input lengths and five output lengths:
$$
\mathcal{L}_{\text{in}} = \{24, 48\}, \qquad \mathcal{L}_{\text{out}} = \{4, 6, 8, 10, 12\}.
$$
At 15-minute resolution, input lengths of 24 and 48 correspond to 6 and 12 hours of history, while output lengths of 4, 6, 8, 10, and 12 correspond to 1, 1.5, 2, 2.5, and 3 hours of forecast horizon. The benchmark therefore defines **10 forecasting settings** in total [2509.25263].

The paper motivates RainfallBench by highlighting three statistical properties. First, it is **zero-inflated**. In the JFNG dataset, **121,572** records have \(tp = 0\), accounting for **71%** of the dataset. Second, it exhibits **temporal dependency decay**, characterized by the approximation
$$
\rho(k) \approx e^{-\lambda k}, \quad \lambda > 0.
$$
Third, it is **non-stationary**, assessed with the Augmented Dickey–Fuller-style regression
$$
\Delta x_t = \alpha + \beta t + \gamma x_{t-1} + \sum_{i=1}^{p} \delta_i \Delta x_{t-i} + \varepsilon_t,
$$
under the null hypothesis
$$
H_0 : \gamma = 0.
$$
The reported **p-value is 0.4381**, so the null cannot be rejected, which the paper interprets as evidence of non-stationarity [2509.25263].

PWV is central to the benchmark’s physical motivation. In the correlation analysis using Pearson, Kendall, and Spearman coefficients, PWV shows the strongest positive correlation with `tp`; the example value given is a **Pearson coefficient of 0.27**. By contrast, `t2m` and `sp` show weak or negative correlations. The paper therefore identifies PWV as the most reliable indicator of imminent rainfall among the included variables [2509.25263].

RainfallBench also defines a dedicated **extreme rainfall evaluation**. Following the **T/CMSA 0013-2019 standard**, extreme rainfall is defined as any 15-minute period with precipitation exceeding **2 mm**. If \(E\) denotes the set of extreme-rainfall time steps, the paper evaluates
$$
\text{MSE}_{\text{extreme}} = \frac{1}{|E|} \sum_{t \in E} (\hat{y}_t - y_t)^2,
$$
and
$$
\text{MAE}_{\text{extreme}} = \frac{1}{|E|} \sum_{t \in E} |\hat{y}_t - y_t|.
$$
This makes extreme-event performance an explicit benchmark axis rather than a secondary diagnostic [2509.25263].

## 4. Evaluation protocol and benchmarked model families

RainfallBench uses **MSE** and **MAE** as its primary point-forecasting metrics and reports average leaderboard rankings over all input-output configurations. It additionally reports extreme-event versions of MSE and MAE restricted to the time steps satisfying the benchmark’s extreme-rainfall definition. The paper does not define a custom challenge metric beyond these general and extreme-restricted error measures [2509.25263].

The benchmark evaluates **20 state-of-the-art models** organized into six architectural families. The MLP-based family includes **DLinear**, **Koopa**, **TimeMixer**, and **FilterTS**. The RNN-based family includes **SegRNN**, **xLSTM**, and **P-sLSTM**. The CNN/TCN-based family includes **TimesNet**, **TimeMixer++**, and **xPatch**. The GNN-based family includes **MSGNet** and **TimeFilter**. The KAN-based family includes **TimeKAN** and **MMK**. The Transformer-based family includes **Informer**, **PatchTST**, **iTransformer**, **TimeXer**, **PPDformer**, and **Informer + BFPF** [2509.25263].

This architectural range is a substantive part of the benchmark design. Rather than evaluating a narrow family of methods, RainfallBench is structured to test whether sequence models originally developed for smoother, more periodic time-series regimes remain competitive under rainfall-specific sparsity, recency sensitivity, and heavy-event rarity. The paper’s broader claim is that conventional benchmark rankings in generic time-series forecasting do not transfer automatically to rainfall nowcasting [2509.25263].

## 5. Bi-Focus Precipitation Forecaster

To address zero inflation and temporal decay, the benchmark paper introduces the **Bi-Focus Precipitation Forecaster (BFPF)** as a plug-and-play module for Transformer-based models. BFPF contains two components: **Non-Zero Focus**, which emphasizes sparse but informative rainfall-related signals, and **Temporal Focus**, which biases attention toward more recent observations [2509.25263].

In the Non-Zero Focus component, the benchmark defines the distance from a time step to the nearest zero-valued entry as
$$
d_t = \begin{cases}
+\infty, & \text{if } x_t = 0 \\
\min\left(|t - z_l|,\ |z_r - t|\right), & \text{otherwise}
\end{cases}
$$
where \(z_l\) and \(z_r\) are the nearest zero positions to the left and right. A proximity weight is then constructed as
$$
w_k = \exp\left(-\frac{d_k}{\tau}\right),
$$
and used as an additive bias in the attention score tensor:
$$
\tilde{\mathbf{S}} = \mathbf{S} + \lambda \cdot \mathbf{W}.
$$
The stated intent is to encourage greater attention to non-zero entries and rainfall onsets embedded in long dry intervals [2509.25263].

In the Temporal Focus component, the benchmark defines a linearly increasing positional bias
$$
\mathbf{p} = \alpha \cdot \left[ \frac{0}{L_K}, \frac{1}{L_K}, \ldots, \frac{L_K - 1}{L_K} \right],
$$
which is added to the attention tensor as
$$
\tilde{\mathbf{S}}_{b,h,i,j} = \mathbf{S}_{b,h,i,j} + \mathbf{p}_j.
$$
Because the bias increases with position index, more recent keys receive larger attention bias. The paper presents this as an inductive prior aligned with the observed temporal decay of rainfall dependence [2509.25263].

The benchmark further provides ablation evidence. Relative to vanilla Informer, Temporal Focus only, Non-Zero Focus only, and the full BFPF each improve some settings, while the combination of both modules yields the most balanced overall gains across horizons. The method section frames BFPF as plug-and-play, but the technical instantiation in the paper is specifically at the Transformer attention-score level [2509.25263].

## 6. Empirical results, benchmark implications, and relation to adjacent resources

The principal empirical result is that **RNN-based models dominate** RainfallBench. The paper explicitly notes that two of the top three models are RNN-based and that RNNs appear especially well matched to rainfall’s short-memory, sparse, and recency-weighted structure. **P-sLSTM** is the best overall model across the main benchmark table, with representative results including **0.0083 / 0.0228** for **24(4)** and **0.0413 / 0.0532** for **48(12)** in MSE/MAE. **SegRNN** is also strong, with the best MSE at **48(8)** and the best MAE at **48(10)**. Among Transformer-based models, **Informer + BFPF** is the strongest, and the paper states that it performs best for **extreme rainfall** [2509.25263].

The benchmark also confirms several scaling properties. Errors increase monotonically with forecast horizon, and increasing input length from 24 to 48 generally reduces MSE. PWV ablation shows that including PWV usually lowers forecasting error; for example, under input length 48, the model with PWV outperforms the version without PWV at 48(4), 48(6), 48(8), and 48(12) in MSE, and at 48(4), 48(6), and 48(8) in MAE. The paper summarizes this as a significant reduction in nowcasting errors [2509.25263].

Several limitations are explicit. Although the benchmark claims global station coverage, the main text centers experiments on the representative station JFNG. Region-wise robustness and transferability across stations are identified as future directions. The main text also provides limited detail on normalization and quality control, and BFPF, though described as plug-and-play, is practically implemented only for Transformer attention. A plausible implication is that RainfallBench is strongest as a benchmark for rainfall-specific sequence modeling behavior and less complete as a geographically stratified, fully specified operational benchmark [2509.25263].

Within the broader literature, RainfallBench occupies a distinct niche. It differs from **“RainBench”** [2012.09670], which targets global precipitation forecasting from simulated satellite imagery, and from **“PostRainBench”** [2310.02676] and **KoMet** [2206.15241], which benchmark NWP post-processing. It is also complementary to **“WeatherBench Probability”** [2205.00865], which provides a probabilistic benchmarking scaffold but is not rainfall-specific, and to **“RainShift”** [2507.04930], which evaluates precipitation downscaling under geographic distribution shift. RainfallBench is therefore best understood as a specialized time-series nowcasting benchmark: short-horizon, station-based in covariates, explicitly PWV-aware, and designed to expose the distinctive statistical pathologies of rainfall prediction rather than the smoother dynamics emphasized in generic forecasting benchmarks [2012.09670] [2205.00865] [2310.02676] [2206.15241] [2507.04930].

Source: https://www.emergentmind.com/topics/rainfallbench