Performance of ML Weather Forecasts on Extreme Events

Determine the predictive accuracy and reliability of machine-learning-based medium-range weather forecasting models, such as GraphCast, PanguWeather, and FourCastNet, for high-impact extreme weather events, quantifying their skill specifically in the extreme-value regime where extrapolation beyond the training distribution is required.

Background

Recent machine-learning (ML) weather models match or outperform ECMWF's high-resolution forecast (HRES) on headline metrics across many variables and lead times. However, extremes are rare in training data and often involve compounding factors, raising concerns about extrapolation and generalization.

The paper focuses on three extreme events—the 2021 Pacific Northwest heatwave, the 2023 South Asian humid heatwave, and the 2021 North American winter storm—to examine case-paper and impact-centric performance, highlighting the need to understand ML behavior under rare, high-impact conditions.