BFPF: Transformer Module for Rainfall Nowcasting

• The paper introduces BFPF, a plug-and-play module for transformer-based rainfall nowcasting that embeds rainfall-specific priors to tackle zero inflation and temporal decay.
• It applies a Non-Zero Focus to emphasize informative, nonzero rainfall signals and a Temporal Focus to prioritize recent data in the attention mechanism.
• Experimental results on RainfallBench demonstrate that BFPF improves extreme rainfall prediction, particularly over longer forecast horizons.

Searching arXiv for the BFPF paper and closely related precipitation forecasting work to ground the article in current literature. Bi-Focus Precipitation Forecaster (BFPF) is a plug-and-play module for transformer-based rainfall nowcasting introduced within the RainfallBench benchmark to address two rainfall-specific failure modes that generic time-series models often handle poorly: zero inflation and temporal decay (Zhang et al., 28 Sep 2025). In the paper’s formulation, rainfall nowcasting is a multivariate-to-univariate forecasting task in which recent meteorological observations $\mathbf{X}=\{\mathbf{x}_t\}_{t=1}^{T}$, with $\mathbf{x}_t\in\mathbb{R}^D$, are used to predict future rainfall $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$, with $y_t\in\mathbb{R}$, over horizons from 1 to 3 hours at 15-minute resolution (Zhang et al., 28 Sep 2025). “Bi-Focus” refers specifically to two additive attention biases inside a transformer: Non-Zero Focus, intended to emphasize informative rainfall-related positions in zero-dominated sequences, and Temporal Focus, intended to bias attention toward more recent observations (Zhang et al., 28 Sep 2025). In the source paper, BFPF is evaluated as an enhancement to Informer rather than as a standalone end-to-end forecaster, and its strongest reported benefit is on extreme rainfall prediction within a benchmark explicitly designed around sparse, non-stationary, event-driven precipitation series (Zhang et al., 28 Sep 2025).

1. Problem formulation and benchmark context

The defining experimental context for BFPF is RainfallBench, a benchmark for rainfall nowcasting built from five years of meteorological observations, from 2018 to 2022, sampled every 15 minutes and collected from more than 12,000 GNSS stations globally (Zhang et al., 28 Sep 2025). The benchmark uses six variables: temperature at 2 meters ($t2m$), surface pressure ($sp$), relative humidity ($rh$), wind speed ($wind\_speed$), precipitable water vapor (PWV), and total precipitation ($tp$) as the target (Zhang et al., 28 Sep 2025). The representative station analyzed in detail, JFNG, contains 171,070 time steps spanning 2018-02-14 00:00 to 2022-12-31 23:00 with no missing data (Zhang et al., 28 Sep 2025).

RainfallBench is organized around multi-temporal-scale evaluation. Input lengths are $\mathcal{L}_{\text{in}}=\{24,48\}$, corresponding to 6-hour and 12-hour histories, while forecasting horizons are $\mathbf{x}_t\in\mathbb{R}^D$0, corresponding to 1- to 3-hour prediction windows at 15-minute cadence (Zhang et al., 28 Sep 2025). The paper reports MSE and MAE on de-normalized actual rainfall values and additionally evaluates extreme rainfall using the T/CMSA 0013-2019 criterion of precipitation exceeding 2 mm in a 15-minute period (Zhang et al., 28 Sep 2025).

The benchmark motivation is central to the rationale for BFPF. Rainfall is described as zero-inflated, temporally decaying, non-stationary, event-driven, and sensitive to extremes (Zhang et al., 28 Sep 2025). The paper reports that at the JFNG station, 121,572 records, about 71% of the dataset, have zero precipitation (Zhang et al., 28 Sep 2025). It also models lag-$\mathbf{x}_t\in\mathbb{R}^D$1 rainfall autocorrelation as approximately exponential decay,

$\mathbf{x}_t\in\mathbb{R}^D$2

and supports non-stationarity with an ADF-style regression

$\mathbf{x}_t\in\mathbb{R}^D$3

for which a sample segment yields a p-value of 0.4381, failing to reject the unit-root null (Zhang et al., 28 Sep 2025). These benchmark properties motivate the two “foci” of BFPF more directly than a generic transformer modification would.

A further distinguishing feature of RainfallBench is the inclusion of PWV derived from GNSS signal delays. The paper reports that PWV has the strongest positive correlation with rainfall among the available variables, with Pearson correlation 0.27, and describes PWV as a key and timely indicator for imminent rainfall (Zhang et al., 28 Sep 2025). This does not define BFPF itself, but it situates the module within a forecasting setup where physically meaningful exogenous predictors matter.

2. Meaning of “Bi-Focus” and architectural role

BFPF is explicitly defined as a plug-and-play module for transformer-based models rather than a complete forecasting architecture (Zhang et al., 28 Sep 2025). In the reported experiments, it is attached to Informer and presented as “Informer (with BFPF)” (Zhang et al., 28 Sep 2025). The module changes attention computation by adding two domain-specific biases to the attention score tensor; it does not introduce a separate rainfall occurrence classifier, a zero-inflated likelihood, a distinct regression head, or a post-hoc calibration stage (Zhang et al., 28 Sep 2025).

The term “Bi-Focus” denotes two inductive biases. The first is Non-Zero Focus, intended to address zero inflation by altering attention toward informative rainfall-related positions in sequences dominated by zeros. The second is Temporal Focus, intended to address temporal decay by favoring recent observations during attention (Zhang et al., 28 Sep 2025). The paper presents these as the two overlooked challenges in transformer-based rainfall forecasting and uses them to explain why generic sequence models can underperform on rainfall despite performing well on more periodic variables (Zhang et al., 28 Sep 2025).

This positioning distinguishes BFPF from several other precipitation forecasting formulations. It is not a precipitation post-processor over NWP grids of the type used in "Self-Supervised Pre-Training for Precipitation Post-Processor" (An et al., 2023), which frames precipitation as a segmentation task over classes such as no rain, rain, and heavy rain. It is also not a radar-sequence encoder-forecaster of the kind used in "MS-nowcasting" (Klocek et al., 2021), where large-context radar inputs and HRRR conditioning are fused within ConvLSTM dynamics. Nor is it a deterministic-plus-probabilistic decomposition of mesoscale mean and convective residual as in "Skillful High-Resolution Ensemble Precipitation Forecasting with an Integrated Deep Learning Framework" (He et al., 6 Jan 2025). BFPF instead modifies the internal score tensor of a transformer to encode rainfall-specific priors (Zhang et al., 28 Sep 2025).

A plausible implication is that BFPF is best understood as an attention-side specialization for sparse meteorological sequences rather than as a full precipitation architecture. This interpretation is consistent with the paper’s experimental design, where the backbone remains Informer and the module’s effect is isolated through ablations (Zhang et al., 28 Sep 2025).

3. Non-Zero Focus

Non-Zero Focus is the component intended to cope with long dry stretches and sparse nonzero rainfall episodes (Zhang et al., 28 Sep 2025). Its first step is Non-Zero Context Encoding. For each time step $\mathbf{x}_t\in\mathbb{R}^D$4, the model computes the distance to the nearest zero-valued entry: $\mathbf{x}_t\in\mathbb{R}^D$5 where $\mathbf{x}_t\in\mathbb{R}^D$6 and $\mathbf{x}_t\in\mathbb{R}^D$7 are the nearest zero indices on the left and right (Zhang et al., 28 Sep 2025). The resulting distance matrix is $\mathbf{x}_t\in\mathbb{R}^D$8, with $\mathbf{x}_t\in\mathbb{R}^D$9 batch size and $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$0 sequence length (Zhang et al., 28 Sep 2025).

These distances are transformed into a proximity weight

$\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$1

where $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$2 is a temperature hyperparameter (Zhang et al., 28 Sep 2025). The paper then adds this as an attention bias: $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$3 where $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$4 is the original attention score tensor, $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$5 is the broadcasted bias tensor derived from $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$6, and $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$7 is a learned scaling factor (Zhang et al., 28 Sep 2025).

The intended interpretation in the paper is that this mechanism encourages greater attention to informative nonzero regions in zero-heavy rainfall sequences (Zhang et al., 28 Sep 2025). The source text also notes a minor inconsistency: while the narrative describes the module as emphasizing nonzero entries, the formula $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$8 decreases with distance to zero (Zhang et al., 28 Sep 2025). The safest reading is therefore the one stated in the source: Non-Zero Focus uses distance-to-zero-derived attention bias to alter attention toward informative rainfall-related positions, even if the exact monotonic semantics of the chosen weighting may appear counterintuitive.

This design choice is notable because it does not solve zero inflation by explicit occurrence–intensity decomposition. That distinguishes BFPF from methods that separate event discrimination and intensity estimation in the loss, such as the WMCE-based latent forecasting framework in "Accurate Precipitation Forecast by Efficiently Learning from Massive Atmospheric Variables and Unbalanced Distribution" (Li et al., 27 Mar 2026). In that work, the event threshold is embedded in a hybrid regression-classification objective. BFPF instead addresses sparsity inside attention, leaving the output formulation of the transformer backbone otherwise unchanged (Zhang et al., 28 Sep 2025).

4. Temporal Focus

Temporal Focus encodes the empirical prior that more recent observations are typically more informative for imminent rainfall than older ones (Zhang et al., 28 Sep 2025). The paper defines a linearly increasing positional bias over the key dimension: $\mathbf{y}=\{y_{T+1},y_{T+2},\dots,y_{T+H}\}$9 where $y_t\in\mathbb{R}$0 is a learnable scaling factor and $y_t\in\mathbb{R}$1 is the key sequence length (Zhang et al., 28 Sep 2025). This bias is added directly to each attention score: $y_t\in\mathbb{R}$2 Because $y_t\in\mathbb{R}$3 increases with position index $y_t\in\mathbb{R}$4, later positions in the input sequence receive larger attention bias and therefore greater influence on the forecast (Zhang et al., 28 Sep 2025).

The role of Temporal Focus becomes clearer when compared with the benchmark’s temporal-decay analysis. Since the paper models rainfall autocorrelation as approximately $y_t\in\mathbb{R}$5, an unmodified transformer attention mechanism may spread weight too uniformly across history or discover recency structure inefficiently (Zhang et al., 28 Sep 2025). Temporal Focus hard-codes a recency prior into the score tensor. This is a simpler mechanism than the learned temporal weighting of all encoder states used in the ConvLSTM nowcasting system of "MS-nowcasting" (Klocek et al., 2021), but both approaches pursue a related objective: avoiding an overly restrictive bottleneck that relies only on the latest state.

A plausible implication is that Temporal Focus is particularly well suited to short-horizon rainfall nowcasting, where the predictability window is strongly local in time. By contrast, in longer-range or subseasonal settings, stronger seasonality and regime dependence may demand richer lead-aware or climate-aware conditioning, as in the adaptive bias correction framework for weeks 3–4 and 5–6 forecasting (Mouatadid et al., 2022) or the seasonal quantile forecasting approach based on TFT (Civitarese et al., 2021). Within the 0–3 hour RainfallBench setting, however, the recency prior is well aligned with the task design (Zhang et al., 28 Sep 2025).

5. Experimental evidence and ablations

The paper reports that among more than 20 models across six major architecture families, recurrent models, especially P-sLSTM, achieve the strongest overall average performance on RainfallBench, but Informer with BFPF is the best transformer-based model and the best under the extreme rainfall setting among all baselines (Zhang et al., 28 Sep 2025). This distinction matters: BFPF materially strengthens transformer performance, but does not make the transformer family dominant on average across all architectures (Zhang et al., 28 Sep 2025).

For input length 24, vanilla Informer versus Informer with BFPF gives the following MSE/MAE pairs across horizons: 24(4), 0.0093/0.0281 versus 0.0104/0.0276; 24(6), 0.0176/0.0365 versus 0.0176/0.0323; 24(8), 0.0298/0.0471 versus 0.0263/0.0428; 24(10), 0.0358/0.0536 versus 0.0392/0.0525; 24(12), 0.0555/0.0738 versus 0.0428/0.0531 (Zhang et al., 28 Sep 2025). For input length 48, the corresponding results are 48(4), 0.0102/0.0262 versus 0.0093/0.0231; 48(6), 0.0191/0.0399 versus 0.0172/0.0355; 48(8), 0.0270/0.0440 versus 0.0263/0.0404; 48(10), 0.0347/0.0522 versus 0.0341/0.0500; 48(12), 0.0462/0.0633 versus 0.0425/0.0593 (Zhang et al., 28 Sep 2025). The largest gains appear at longer horizons, particularly 24(12) and several 48-step-input settings (Zhang et al., 28 Sep 2025).

The ablation study isolates the two parts of BFPF on Informer with input length 48. The baseline without the module gives 0.0102/0.0262, 0.0191/0.0399, 0.0270/0.0440, 0.0347/0.0522, and 0.0462/0.0633 for 48(4), 48(6), 48(8), 48(10), and 48(12), respectively (Zhang et al., 28 Sep 2025). Temporal Focus only gives 0.0093/0.0238, 0.0181/0.0363, 0.0271/0.0419, 0.0342/0.0534, and 0.0455/0.0621; Non-Zero Focus only gives 0.0091/0.0257, 0.0197/0.0446, 0.0254/0.0460, 0.0341/0.0565, and 0.0472/0.0621; and the full BFPF gives 0.0093/0.0231, 0.0172/0.0355, 0.0263/0.0404, 0.0341/0.0500, and 0.0425/0.0593 (Zhang et al., 28 Sep 2025). The paper interprets this as evidence that each component is useful in isolation but the combination is usually the most robust (Zhang et al., 28 Sep 2025).

The benchmark also evaluates PWV. For the BFPF-equipped model, without PWV the scores are 0.0094/0.0281, 0.0194/0.0370, 0.0272/0.0461, 0.0345/0.0462, and 0.0428/0.0530 across 48(4) to 48(12), while with PWV they are 0.0093/0.0231, 0.0172/0.0355, 0.0263/0.0404, 0.0341/0.0500, and 0.0425/0.0593 (Zhang et al., 28 Sep 2025). The paper concludes that PWV significantly improves nowcasting overall, though the gains are not uniform for every metric and horizon (Zhang et al., 28 Sep 2025).

A plausible reading of these results is that BFPF helps most where a transformer is structurally weakest on rainfall: long dry periods, abrupt event onset, and longer horizon degradation. The reported superiority under the extreme rainfall setting is particularly important because benchmark-average MSE and MAE can obscure behavior on sparse, high-impact events (Zhang et al., 28 Sep 2025).

6. Significance, limitations, and relation to broader precipitation forecasting

The significance of BFPF lies less in architectural scale than in domain-aware inductive bias. The source paper argues that rainfall nowcasting benefits from embedding rainfall-specific priors directly into attention rather than treating it as an ordinary multivariate forecasting problem (Zhang et al., 28 Sep 2025). This aligns with a broader pattern in precipitation research. Several recent systems obtain gains not merely from larger generic models, but from carefully targeted mechanisms for imbalance, context, calibration, or multiscale representation: self-supervised NWP-encoder transfer and continuous labeling for heavy-rain segmentation (An et al., 2023), efficient large-viewport radar context and HRRR conditioning (Klocek et al., 2021), latent occurrence–intensity objectives under extreme imbalance (Li et al., 27 Mar 2026), and two-stage classification-plus-reconstruction blending for extreme daily QPF (Wang et al., 6 Jul 2026).

At the same time, the limitations of BFPF are explicit. The paper does not specify optimizer type, learning rate, batch size, number of epochs, dropout, weight decay, or exact insertion depth inside Informer (Zhang et al., 28 Sep 2025). It also does not quantify parameter overhead or runtime cost, so the “plug-and-play” claim remains qualitative (Zhang et al., 28 Sep 2025). More substantively, BFPF does not make transformers best overall on RainfallBench; recurrent models remain stronger on average (Zhang et al., 28 Sep 2025). Gains are not uniform in every horizon and metric, and the paper provides no explicit failure-case study beyond noting that extreme rainfall remains hard for all models (Zhang et al., 28 Sep 2025).

Another important limitation is conceptual. BFPF does not model zero inflation through an explicit statistical likelihood, censoring mechanism, or two-part occurrence–amount formulation. In this respect it differs from Tobit-style precipitation post-processing approaches that treat zeros as censored observations (Johnson et al., 2021, Courbariaux et al., 2018), as well as from methods that directly separate event discrimination from intensity estimation in the loss (Li et al., 27 Mar 2026). It also does not implement a true dual-branch local/global architecture. “Bi-Focus” in the paper refers to two attention biases, not to two independently learned pathways (Zhang et al., 28 Sep 2025).

This suggests two distinct interpretations of BFPF’s place in the literature. In its narrow sense, BFPF is the specific Non-Zero Focus plus Temporal Focus attention-bias module proposed in RainfallBench (Zhang et al., 28 Sep 2025). In a broader editor’s sense, “bi-focus” can denote a design principle recurring across precipitation forecasting: separating two complementary forms of information that a monolithic regressor tends to conflate. Depending on task, those foci may be zero versus nonzero rainfall, recent versus older context, local radar detail versus broader meteorological context, occurrence versus intensity, or deterministic backbone versus probabilistic refinement. The RainfallBench BFPF instantiates one such separation at the attention-score level (Zhang et al., 28 Sep 2025).

From that perspective, the most defensible encyclopedic characterization is that BFPF is a transformer-side rainfall nowcasting enhancement whose methodological importance lies in codifying two empirically grounded rainfall priors—sparsity sensitivity and recency preference—inside attention. Its contribution is not to replace the forecasting backbone, but to show that relatively small, domain-specific structural biases can materially improve transformer behavior on sparse, event-driven precipitation series, especially for longer horizons and extreme rainfall (Zhang et al., 28 Sep 2025).