Reliability of data-driven weather models for extreme events

Determine whether AI-based data-driven weather forecasting models trained on reanalysis data, such as Pangu-Weather and related models, can reliably predict extreme weather events.

Background

The paper evaluates uncertainty quantification approaches for deterministic data-driven models, primarily Pangu-Weather, over Europe and shows competitive performance relative to a leading physics-based ensemble for many variables and lead times.

However, the evaluation focuses on general performance metrics (e.g., CRPS) and typical conditions rather than tail behavior, prompting the authors to highlight the unresolved question of model reliability for extremes. They note that targeted evaluation for extremes (e.g., via weighted scoring rules) would be a crucial next step to assess the models’ potential and limitations for predicting high-impact events.