Weather Foundation Models (WFMs)

• Weather Foundation Models (WFMs) are large, transformer-based architectures trained on multivariate atmospheric data to capture intrinsic weather dynamics.
• They employ an encoder–backbone–decoder design with dual decoders to provide both dense spatial fields and hyperlocal, asset-specific forecasts.
• WFMs enhance grid resilience by delivering accurate predictions for temperature, precipitation, wind, and icing, enabling proactive infrastructure management.

Weather @@@@1@@@@ (WFMs) are large-scale, general-purpose neural architectures, typically transformer-based, pre-trained on vast, heterogeneous Earth system data to encode the fundamental statistical and dynamical relationships governing atmospheric processes. Their versatility enables efficient adaptation—often with minimal fine-tuning or parameter updates—to diverse downstream tasks across weather, climate, and coupled environmental domains. WFMs now underpin a new generation of data-driven meteorological intelligence, with demonstrated superlative skill over traditional numerical weather prediction (NWP) in multiple settings, including critical infrastructure risk management and energy systems.

1. Core Principles and Architectural Design

WFMs typically adopt a large backbone model—most commonly an encoder–backbone–decoder transformer—designed to process gridded, multivariate spatiotemporal fields as canonical "tokens" (Nguyen et al., 2023, Schmude et al., 20 Sep 2024, Bodnar et al., 28 Sep 2025). The encoder ingests heterogeneous meteorological inputs (e.g., multilevel temperature, wind, humidity, cloud proxies), constructs latent representations via learned projections, and the backbone—a stack of transformer or similar attention-based layers—models intrinsic spatial and temporal dynamics, exploiting global dependencies. The decoder (or decoders) maps these latent states onto application-specific outputs, which may be dense grids, localized forecasts, or domain-targeted quantities.

Modern WFMs often feature dual-headed decoders: a dense decoder yields conventional spatial fields, while a sparse (or asset-level) decoder produces hyper-local predictions at specific sites (e.g., transmission lines or wind turbines) (Bodnar et al., 28 Sep 2025).

Pre-training is accomplished on petabyte-scale, multidecadal observational and reanalysis datasets—potentially mixing public archives (e.g., ECMWF-IFS, MERRA-2) with proprietary asset streams. This furnishes the backbone with a universal "grammar" of atmospheric physics, enabling the model to interpolate and generalize across variable, resolution, and topological differences.

2. WFMs for Power Grid and Critical Infrastructure

A key demonstration of WFM capability is their deployment for weather-sensitive infrastructure, as exemplified by the fine-tuning of Silurian AI's 1.5B-parameter Generative Forecasting Transformer (GFT) on Hydro-Québec asset observations (Bodnar et al., 28 Sep 2025). The model ingested a rich archive of local observations (2016–2023), including transmission-line weather station data, wind-farm met masts, and direct icing sensor streams.

Through end-to-end post-training, the model was directly optimized to produce forecasts for five grid-critical variables:

• Surface (2 m) temperature
• Hourly precipitation
• Hub–height wind speed
• Wind–turbine icing risk
• Rime–ice accretion on overhead conductors

End-to-end post-training, as opposed to conventional model output statistics (MOS) or other site-specific regressors, enables the WFM to jointly recalibrate spatiotemporal dynamics, enforce multivariate physical coherence, and directly predict quantities—such as rime-ice risk—not available via standard NWP (Bodnar et al., 28 Sep 2025).

3. Quantitative Performance and Operational Metrics

Extensive out-of-sample validation (hold-out on 2024–2025 data) demonstrates the operational superiority of the post-trained WFM (termed GFT–HQ) over state-of-the-art NWP reference forecasts (ECMWF–IFS) in the following variables:

Variable	Reduction in MAE (%)	Additional Highlights
Surface temperature (2 m)	15	Robust across 6–72 h lead times, stable during peak load periods
Hourly precipitation	35	Enhanced spatial skill, river-basin improvements
Hub-height wind speed	15	High-wind prediction, improved turbine shutdown (cut-out) forecasts
Rime-ice (day-ahead detection)	—	Average precision (AP) of 0.72, ~8× skill over base rate (0.37),
		and >0.53 (ERA5 Makkonen proxy)

The model delivers several hours of actionable warning for rime-ice accretion—a capability not available in operational legacy systems. Hourly icing probabilities are aggregated using:

$q_t = 1 - \prod_h (1 - p_{t+h})$

where $p_{t+h}$ denotes the forecast probability at lead hour $h$, yielding an operational 24 h “any icing” event probability. Cost–loss optimization defines optimal alert thresholds:

$p^* = \frac{C_d}{\alpha L}$

with $C_d$ (dispatch cost), $L$ (miss loss), and $\alpha \in [0, 1]$ (mitigated loss fraction).

4. Methodological Innovations and Scientific Significance

The WFM paradigm establishes several significant advances over previous ensemble and post-processing workflows:

• Multivariate coherence: Joint fine-tuning allows the model to learn correlated atmospheric phenomena (e.g., the dependence of icing on temperature, wind, and precipitation), minimizing inconsistencies that plague variable-by-variable statistical corrections.
• Direct forecast of non-canonical targets: Model post-training enables direct probability estimation of engineering-relevant quantities (e.g., icing index, asset-level extremes) that are not retrievable as physical variables from traditional NWP.
• Hyper-local forecasting: Sparse (asset-specific) decoders, trained on site-level observations, provide operational forecasts at the scale of critical assets—surpassing the spatial granularity of typical NWP models.

These features allow a single WFM to replace disparate, post hoc adjustment models with a unified, physically-consistent, and dynamically-adaptive tool.

5. Implications for Grid Resilience and Decision-Making

The improved accuracy of WFMs in predicting temperature, precipitation, and wind speed translates into tangible enhancements in operational grid management:

• Load prediction: Refined temperature forecasts improve demand estimates and scheduling.
• Renewable output/curtailment: Better hub-height wind forecasts enhance wind-farm dispatch and maintenance planning; timely high-wind alerts reduce turbine downtime.
• Asset protection: Early and reliable rime-ice alerts allow for preemptive de-icing actions, dynamic line rating, and reduced outage duration.
• River basin management: Accurate precipitation forecasts benefit hydro-generation scheduling and flood risk assessment.

The ability to generate multivariate, hyperlocal, and actionable predictions with minimal additional training marks WFMs as an emerging backbone for operational grid-resilience intelligence.

6. Generality, Transfer, and Future Directions

This work demonstrates that post-training WFMs with modest volumes of high-quality local data can dramatically enhance site-specific prediction accuracy, even for variables and risk patterns absent from baseline NWP products (Bodnar et al., 28 Sep 2025). These findings generalize to other regions and asset classes contingent on the availability of minimal, targeted calibration datasets, positioning WFMs as a practical foundation for sector-specific meteorological services.

The universal architecture, data-driven adaptability, and demonstrated outperformance over state-of-the-art NWP benchmarks set a precedent for WFMs as a central enabler in operational meteorological informatics—bridging foundational atmospheric modeling with the mission-critical needs of modern infrastructure.