Calibrating Bayesian UNet++ for Sub-Seasonal Forecasting (2403.16612v2)
Abstract: Seasonal forecasting is a crucial task when it comes to detecting the extreme heat and colds that occur due to climate change. Confidence in the predictions should be reliable since a small increase in the temperatures in a year has a big impact on the world. Calibration of the neural networks provides a way to ensure our confidence in the predictions. However, calibrating regression models is an under-researched topic, especially in forecasters. We calibrate a UNet++ based architecture, which was shown to outperform physics-based models in temperature anomalies. We show that with a slight trade-off between prediction error and calibration error, it is possible to get more reliable and sharper forecasts. We believe that calibration should be an important part of safety-critical machine learning applications such as weather forecasters.
- Seasonal arctic sea ice forecasting with probabilistic deep learning. Nature Communications, 12(1):5124, 2021.
- Machine learning for climate precipitation prediction modeling over south america. Remote Sensing, 13(13):2468, 2021.
- Overview of the coupled model intercomparison project phase 6 (cmip6) experimental design and organization. Geoscientific Model Development, 9:1937–1958, 2016.
- Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pages 1050–1059. PMLR, 2016.
- Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2):243–268, 2007.
- Bayesian neural networks: An introduction and survey. In Case Studies in Applied Bayesian Data Science, pages 45–87. Springer International Publishing, 2020. doi: 10.1007/978-3-030-42553-1˙3.
- Auto-encoding variational bayes, 2022.
- Being bayesian, even just a bit, fixes overconfidence in relu networks, 2020.
- Accurate uncertainties for deep learning using calibrated regression. In International Conference on Machine Learning, pages 2796–2804. PMLR, 2018.
- Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems, 30, 2017.
- A bayesian deep learning approach to near-term climate prediction. Journal of Advances in Modeling Earth Systems, 14(10):e2022MS003058, 2022.
- Predicting good probabilities with supervised learning. In International Conference on Machine Learning, pages 625–632, 2005.
- Climate model-driven seasonal forecasting approach with deep learning. Environmental Data Science, 2:e29, 2023. doi: 10.1017/eds.2023.24.