- The paper presents a detailed comparison between BMA and EMOS methods for calibrating ensemble weather forecasts.
- The study employs normal, truncated normal, and gamma distributions to correct under-dispersion in temperature and wind speed predictions.
- Findings indicate that while both methods improve calibration, BMA with bias correction often outperforms EMOS in achieving lower CRPS and better target coverage.
Comparison of BMA and EMOS Statistical Calibration Methods for Temperature and Wind Speed Ensemble Weather Prediction
The paper "Comparison of BMA and EMOS statistical calibration methods for temperature and wind speed ensemble weather prediction" (1312.3763) presents an analytical comparison between two notable statistical post-processing methods for ensemble weather prediction: Bayesian Model Averaging (BMA) and Ensemble Model Output Statistics (EMOS). Both methodologies aim to refine the precision of ensemble forecasts by correcting under-dispersive tendencies and calibrating ensemble predictions to yield more reliable probabilistic forecasts. The paper specifically focuses on 2-meter temperature and 10-meter wind speed forecasts from the ALADIN-HUNEPS ensemble prediction system implemented by the Hungarian Meteorological Service (HMS).
Introduction
Weather prediction relies primarily on Numerical Weather Prediction (NWP) models to simulate atmospheric processes. However, these models often exhibit under-dispersion, which can result in inaccurate forecasts. Probabilistic ensemble forecasts, generated by multiple runs with varied initial conditions, present a potential solution, offering a distribution of future atmospheric states. An additional layer of refinement — statistical post-processing — is required to yield more calibrated probabilities. Two prevalent post-processing methodologies are compared in this paper: Bayesian Model Averaging (BMA) and Ensemble Model Output Statistics (EMOS), as implemented in the R packages ensembleBMA and ensembleMOS respectively. The authors empirically validate these methods using temperature and wind speed forecasts generated by ALADIN-HUNEPS, a limited-area model ensemble prediction system.
ALADIN-HUNEPS Ensemble Description
The ALADIN-HUNEPS ensemble adopts dynamical downscaling of the global ARPEGE based PEARP system of Météo France, covering continental Europe. With a resolution of 12 km and consisting of 11 members, it includes a control member and 10 ensemble members with perturbed initial conditions. These perturbations are derived from singular vector and EDA-based perturbations, emphasizing forecast probability adjustments. Calibrating this ensemble is crucial due to its inherent under-dispersion, as demonstrated by verification rank histograms showing a suboptimal calibration before applying post-processing techniques.
BMA and EMOS Calibration Methods
Bayesian Model Averaging (BMA)
BMA was introduced by Raftery et al. (2005) and formulates the predictive PDF of a target weather variable as a mixture of component probability density functions (PDFs), each weighted based on its past performance. For temperature, BMA models employ normal component PDFs, adjusting mean parameters via linear regression and achieving variance estimation through maximum likelihood (ML) methods. Wind speed forecasts involve the use of skewed distributions such as the gamma distribution or mixtures of truncated normal distributions. Parameter estimation can either aim to maximize the likelihood function or minimize verification scores.
Ensemble Model Output Statistics (EMOS)
EMOS offers a different calibration technique where a single predictive density is parametrically dependent on ensemble members. For temperature, EMOS models employ a normal distribution, whereas for wind speed, a truncated normal PDF is used to accommodate the non-negativity constraint. Parameters are optimized by minimizing a proper scoring rule, the continuous ranked probability score (CRPS), computed over a rolling training period.
Results and Analysis
The practical examination of BMA and EMOS post-processing demonstrates substantial improvements in terms of probabilistic and point forecast performance compared to the raw ensemble forecasts. For temperature forecasts, both BMA and EMOS methods notably improve calibration, as indicated by more uniform PIT histograms compared to the under-dispersive raw ensemble. The paper finds that the BMA model incorporating linear bias correction yields the most balanced performance in terms of calibration and sharpness of predictive distributions.
For wind speed, the analysis shows that while both EMOS and BMA models perform better than the raw ensemble, the BMA model employing a truncated normal distribution for wind speed emerges as the most proficient method. This method achieves the lowest CRPS values and provides better approximations of the target coverage interval.
Conclusion
The comparative analysis in the paper establishes that both BMA and EMOS post-processing methods offer significant improvements over raw ensemble forecasts, specifically for temperature and wind speed. Despite these improvements, some nuances exist, such as EMOS providing better point forecasts due to more effective bias correction mechanisms, while BMA, especially with proper bias correction, tends to slightly outperform EMOS in other verification metrics such as CRPS. The findings underscore the importance of methodological choices tailored to the specific ensemble forecast characteristics, like bias and spread, to enhance forecast reliability and accuracy. Future developments might focus on integrating these approaches with advanced hybrid systems, potentially leveraging machine learning techniques to refine model parameter estimation and improve real-time adaptability.