Machine Learning for Stochastic Parametrisation (2402.09471v1)
Abstract: Atmospheric models used for weather and climate prediction are traditionally formulated in a deterministic manner. In other words, given a particular state of the resolved scale variables, the most likely forcing from the sub-grid scale processes is estimated and used to predict the evolution of the large-scale flow. However, the lack of scale-separation in the atmosphere means that this approach is a large source of error in forecasts. Over recent years, an alternative paradigm has developed: the use of stochastic techniques to characterise uncertainty in small-scale processes. These techniques are now widely used across weather, sub-seasonal, seasonal, and climate timescales. In parallel, recent years have also seen significant progress in replacing parametrisation schemes using ML. This has the potential to both speed up and improve our numerical models. However, the focus to date has largely been on deterministic approaches. In this position paper, we bring together these two key developments, and discuss the potential for data-driven approaches for stochastic parametrisation. We highlight early studies in this area, and draw attention to the novel challenges that remain.
- A Arakawa and W. H. Schubert. Interaction of a Cumulus Cloud Ensemble with the Large-Scale Environment, Part I. Journal of the Atmospheric Sciences, 31:674–701, 1974.
- Stochastic parameterizations and model uncertainty in the Lorenz’96 system. Philosophical Transactions of the Royal Society A, 371(1991), 2013.
- Do GANs learn the distribution? some theory and empirics. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=BJehNfW0-.
- The quiet revolution of numerical weather prediction. Nature, 525(7567):47–55, sep 2015. 10.1038/nature14956. URL http://www.nature.com/doifinder/10.1038/nature14956.
- Improving atmospheric processes in earth system models with deep learning ensembles and stochastic parameterizations. arXiv preprint: 2402.03079, 2024.
- Large-scale dynamical response to subgrid-scale organization provided by cellular automata. Journal of the atmospheric sciences, 68(12):3132–3144, 2011.
- A stochastic parametrization for deep convection using cellular automata. Quarterly Journal of the Royal Meteorological Society, 139(675):1533–1543, 2013.
- Stochastic parameterization: Towards a new view of weather and climate models. Bulletin of the American Meteorological Society, 98:565–588, 2017.
- Enforcing analytic constraints in neural networks emulating physical systems. Physical Review Letters, 9(126):098302, 2021a.
- Climate-Invariant Machine Learning. arXiv preprint arXiv:2112.08440, 2021b.
- Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533–538, 2023.
- Interpreting and Stabilizing Machine-Learning Parametrizations of Convection. Journal of the Atmospheric Sciences, 77:4357–4375, 2020. 10.1175/JAS-D-20-0082.1.
- Prognostic validation of a neural network unified physics parameterization. Geophysical Research Letters, 2018.
- Stochastic representation of model uncertainties in the ECMWF Ensemble Prediction System. Quarterly Journal of the Royal Meteorological Society, 125(560):2887–2908, 1999. 10.1256/smsqj.56005.
- Machine learning emulation of gravity wave drag in numerical weather forecasting. Journal of Advances in Modeling Earth Systems, 13(7):e2021MS002477, 2021. https://doi.org/10.1029/2021MS002477. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2021MS002477. e2021MS002477 2021MS002477.
- A neural network approach for a fast and accurate computation of a longwave radiative budget. Journal of Applied Meteorology, 37(11):1385–1397, 1998. 10.1175/1520-0450(1998)037¡1385:annafa¿2.0.co;2.
- H. M. Christensen. Constraining stochastic parametrisation schemes using high-resolution simulations. Quarterly Journal of the Royal Meteorological Society, 146(727):938–962, 2020.
- H. M. Christensen and Driver O. G. A. The fractal nature of clouds in global storm-resolving models. Geophysical Research Letters, 48(23):n/a–n/a, 2021. doi.org/10.1029/2021GL095746. e2021GL095746.
- Simulating weather regimes: Impact of stochastic and perturbed parameter schemes in a simple atmospheric model. Climate Dynamics, 44(7):2195–2214, 2015a.
- Stochastic and perturbed parameter representations of model uncertainty in convection parameterization. Journal of the Atmospheric Sciences, 72(6):2525–2544, 2015b.
- Introducing independent patterns into the stochastically perturbed parametrization tendencies (SPPT) scheme. Quarterly Journal of the Royal Meteorological Society, 143(706):2168–2181, 2017a.
- Parametrization in weather and climate models. Oxford Research Encyclopedias, Climate Science, 12(12):e2020MS002226, 20202. https://doi.org/10.1029/2020MS002226. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2020MS002226. e2020MS002226 10.1029/2020MS002226.
- Stochastic parameterization and El Niño-southern oscillation. Journal of Climate, 30(1):17–38, 2017b. 10.1175/JCLI-D-16-0122.1.
- Fluctuations in an equilibrium convective ensemble. Part I: Theoretical formulation. Journal of the Atmospheric Sciences, 63(8):1996–2004, 2006. 10.1175/JAS3709.1.
- D. Crommelin and E. Vanden-Eijnden. Subgrid-scale parametrisation with conditional markov chains. Journal of the Atmospheric Sciences, 65(8):2661–2675, 2008.
- A. Dawson and T. N. Palmer. Simulating weather regimes: impact of model resolution and stochastic parametrisation. Climate Dynamics, 44(7):2177–2193, 2015.
- Stochastic parameterization of shallow cumulus convection estimated from high-resolution model data. Theoretical and Computational Fluid Dynamics, 27(1–2):133–148, 2013.
- Machine learning for stochastic parameterization: Generative adversarial networks in the lorenz’96 model. Journal of Advances in Modeling Earth Systems, 21(3), 2020. e2019MS001896.
- Could machine learning break the convection parameterization deadlock? Geophysical Research Letters, 2018.
- Machine learning the warm rain process. Journal of Advances in Modeling Earth Systems, 13(2):e2020MS002268, 2021. https://doi.org/10.1029/2020MS002268. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2020MS002268. e2020MS002268 2020MS002268.
- Stochastic Climate Theory. In Christian L. E. Franzke and Terence J.Editors O’Kane, editors, Nonlinear and Stochastic Climate Dynamics, page 209–240. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA, 2017. 10.1017/9781316339251.009.
- Stochastic-Deep Learning Parameterization of Ocean Momentum Forcing. Journal of Advances in Modeling the Earth System, 13(9):e2021MS002534, 2021.
- Critical Southern Ocean climate model biases traced to atmospheric model cloud errors. Nature Communications, 9(1), 2018. 10.1038/s41467-018-05634-2. URL http://dx.doi.org/10.1038/s41467-018-05634-2.
- Seas5: the new ecmwf seasonal forecast system. Geoscientific Model Development, 12(3):1087–1117, 2019.
- New approach to calculation of atmospheric model physics: Accurate and fast neural network emulation of longwave radiation in a climate model. Monthly Weather Review, 133(12):3721–3723, 2005.
- Graphcast: Learning skillful medium-range global weather forecasting. arXiv preprint arXiv:2212.12794, 2022.
- Stochastic representations of model uncertainties at ECMWF: state of the art and future vision. Quarterly Journal of the Royal Meteorological Society, 143(707):2315–2339, 2017. 10.1002/qj.3094.
- E. N. Lorenz. The predictability of a flow which possesses many scales of motion. Tellus, 21(3):289–307, 1969.
- E. N. Lorenz. Climatic predictability. In The Physical Basis of Climate and Climate Modeling, volume 16, pages 132–136. GARP Publication Series, 1975.
- The weather and climate: emergent laws and multifractal cascades. Cambridge University Press, 2013.
- Characterizing uncertainty in deep convection triggering using explainable machine learning. Journal of the Atmospheric Sciences, submitted.
- Stochastic parameterization of column physics using generative adversarial networks. Environmental Data Science, 1:e22, 2022. 10.1017/eds.2022.32.
- A climatology of atmospheric wavenumber spectra of wind and temperature observed by commercial aircraft. Journal of the Atmospheric Sciences, 42(9):950–960, 1985.
- A. Niculescu-Mizil and R. Caruana. Predicting good probabilities with supervised learning. In Proc. 22nd International Conference on Machine Learning (ICML’05), 2005. https://www.cs.cornell.edu/ alexn/papers/calibration.icml05.crc.rev3.pdf.
- Using machine learning to parameterize moist convection: potential for modeling of climate, climate change and extreme events. Journal of Advances in Modelling Earth Systems, 2018.
- Towards process-level representation of model uncertainties: Stochastically perturbed parametrisations in the ECMWF ensemble. Quarterly Journal of the Royal Meteorological Society, 143(702):408–422, 2017.
- T. N. Palmer. Stochastic weather and climate models. Nature reviews physics, 1(7), 2019. 10.1038/s42254-019-0062-2.
- Stochastic parametrization and model uncertainty, 2009.
- Raghul Parthipan. ML tools for modelling dynamical systems: existing deficiencies and new approaches. PhD thesis, University of Cambridge, 2024.
- Using probabilistic machine learning to better model temporal patterns in parameterizations: a case study with the lorenz 96 model. Geoscientific Model Development, 16(15):4501–4519, 2023.
- Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.11214, 2022.
- A stochastic parameterization for deep convection based on equilibrium statistics. Journal of the Atmospheric Sciences, 65(1):87–105, 2008. ISSN 00224928. 10.1175/2007JAS2263.1.
- A stochastic scale-aware parameterization of shallow cumulus convection across the convective gray zone. Journal of Advances in Modeling Earth Systems, 8(2):786–812, 2016. https://doi.org/10.1002/2016MS000634. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2016MS000634.
- Improved stochastic physics schemes for global weather and climate models. Quarterly Journal of the Royal Meteorological Society, 142(694):147–159, 2016.
- Parameterization of Stochastically Entraining Convection Using Machine Learning Technique. Journal of Advances in Modeling Earth Systems, 14(5), 2022. https://doi.org/10.1029/2021MS002817.
- Global energy spectrum of the general oceanic circulation. Nature communications, 13(1), 2022. 10.1038/s41467-022-33031-3.
- Progress towards a probabilistic Earth system model: Examining the impact of stochasticity in the atmosphere and land component of EC-Earth v3.2. Geoscientific Model Development, 12(7), 2019. 10.5194/gmd-12-3099-2019.
- Description of the arm large-scale forcing data from the constrained variational analysis ( varanal ) version 2 (august). 2019. https://www.arm.gov/publications/tech_reports/doe-sc-arm-tr-222.pdf.
- Evaluation of Machine Learning Classifiers for Predicting Deep Convection. Journal of Advances in Modeling Earth Systems, 11(6):1784–1802, 2019. https://doi.org/10.1029/2018MS001561. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2018MS001561.
- Accelerating radiation computations for dynamical models with targeted machine learning and code optimization. Journal of Advances in Modeling Earth Systems, 12(12):e2020MS002226, 2020. https://doi.org/10.1029/2020MS002226. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2020MS002226. e2020MS002226 10.1029/2020MS002226.
- Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Impact of Stochastic Physics and Model Resolution on the Simulation of Tropical Cyclones in Climate GCMs. Journal of Climate, 34(11):4315 – 4341, 2021. https://doi.org/10.1175/JCLI-D-20-0507.1. URL https://journals.ametsoc.org/view/journals/clim/34/11/JCLI-D-20-0507.1.xml.
- Non-local parameterization of atmospheric subgrid processes with neural networks. Journal of Advances in Modeling Earth Systems, 14(10):e2022MS002984, 2022. https://doi.org/10.1029/2022MS002984.
- The ecmwf model, progress and challenges. In Seminar on recent developments in numerical methods for atmosphere and ocean modelling, pages 1–14. ECMWF, 2013.
- Addressing model error through atmospheric stochastic physical parametrizations: impact on the coupled ECMWF seasonal forecasting system. Philosophical Transactions of the Royal Society A, 372, 2014.
- Modeling mixed type random variables. In 2017 Winter Simulation Conference (WSC), pages 1595–1606. IEEE, 2017.
- D. S. Wilks. Statistical Methods in the Atmospheric Sciences, volume 91 of International Geophysics Series. Elsevier, second edition, 2006.
- The impact of stochastic physics on the El Niño Southern Oscillation in the EC-Earth coupled model. Climate Dynamics, 53:2843–2859, 2019. 10.1007/s00382-019-04660-0. URL https://doi.org/10.1007/s00382-019-04660-0.
- J. Yuval and Paul A O’Gorman. Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions. Nature Communications, 11(3295):1–10, 2020. ISSN 2041-1723. 10.1038/s41467-020-17142-3. URL http://dx.doi.org/10.1038/s41467-020-17142-3.
- Use of neural networks for stable, accurate and physically consistent parameterization of subgrid atmospheric processes with good performance at reduced precision. Geophysical Research Letters, pages 1–15, 2021. ISSN 0094-8276. 10.1029/2020gl091363.
- Implementation and evaluation of a machine learned mesoscale eddy parameterization into a numerical ocean circulation model. arXiv preprint: 2303.00962, 2023.