Transfer Learning for Emulating Ocean Climate Variability across $CO_2$ forcing (2405.18585v3)
Abstract: With the success of ML applied to climate reaching further every day, emulators have begun to show promise not only for weather but for multi-year time scales in the atmosphere. Similar work for the ocean remains nascent, with state-of-the-art limited to models running for shorter time scales or only for regions of the globe. In this work, we demonstrate high-skill global emulation for surface ocean fields over 5-8 years of model rollout, accurately representing modes of variability for two different ML architectures (ConvNext and Transformers). In addition, we address the outstanding question of generalization, an essential consideration if the end-use of emulation is to model warming scenarios outside of the model training data. We show that 1) generalization is not an intrinsic feature of a data-driven emulator, 2) fine-tuning the emulator on only small amounts of additional data from a distribution similar to the test set can enable the emulator to perform well in a warmed climate, and 3) the forced emulators are robust to noise in the forcing.
- Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533–538, 2023.
- Ocean emulation with fourier neural operators: Double gyre. Authorea Preprints, 2023.
- Spherical fourier neural operators: Learning stable dynamics on the sphere. arXiv preprint arXiv:2306.03838, 2023.
- Oceannet: A principled neural operator-based digital twin for regional oceans. arXiv preprint arXiv:2310.00813, 2023.
- Simulated climate and climate change in the GFDL CM2.5 high-resolution coupled climate model. Journal of Climate, 25(8):2755–2781, 2012. ISSN 0894-8755. doi: 10.1175/JCLI-D-11-00316.1.
- An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
- Long-term prediction of the gulf stream meander using oceannet: a principled neural operator-based digital twin. EGUsphere, 2024:1–23, 2024.
- Deep residual convolutional neural network combining dropout and transfer learning for enso forecasting. Geophysical Research Letters, 48(24):e2021GL093531, 2021.
- Advancing parsimonious deep learning weather prediction using the healpix mes. Authorea Preprints, 2023.
- Neural general circulation models. arXiv preprint arXiv:2311.07222, 2023.
- A review of the predictability and prediction of enso. Journal of Geophysical Research: Oceans, 103(C7):14375–14393, 1998.
- Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012–10022, 2021.
- A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976–11986, 2022.
- Climax: A foundation model for weather and climate. arXiv preprint arXiv:2301.10343, 2023.
- Gencast: Diffusion-based ensemble forecasting for medium-range weather. arXiv preprint arXiv:2312.15796, 2023.
- Building ocean climate emulators. arXiv preprint arXiv:2402.04342, 2024.
- Explaining the physics of transfer learning in data-driven turbulence modeling. PNAS nexus, 2(3):pgad015, 2023.
- Ace: A fast, skillful learned global atmospheric model for climate prediction. arXiv preprint arXiv:2310.02074, 2023.
- Ai-goms: Large ai-driven global ocean modeling system. arXiv preprint arXiv:2308.03152, 2023.
- Fuxi-ens: A machine learning model for medium-range ensemble weather forecasting. arXiv preprint arXiv:2405.05925, 2024.