Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fuxi-DA: A Generalized Deep Learning Data Assimilation Framework for Assimilating Satellite Observations (2404.08522v1)

Published 12 Apr 2024 in cs.LG and physics.ao-ph

Abstract: Data assimilation (DA), as an indispensable component within contemporary Numerical Weather Prediction (NWP) systems, plays a crucial role in generating the analysis that significantly impacts forecast performance. Nevertheless, the development of an efficient DA system poses significant challenges, particularly in establishing intricate relationships between the background data and the vast amount of multi-source observation data within limited time windows in operational settings. To address these challenges, researchers design complex pre-processing methods for each observation type, leveraging approximate modeling and the power of super-computing clusters to expedite solutions. The emergence of deep learning (DL) models has been a game-changer, offering unified multi-modal modeling, enhanced nonlinear representation capabilities, and superior parallelization. These advantages have spurred efforts to integrate DL models into various domains of weather modeling. Remarkably, DL models have shown promise in matching, even surpassing, the forecast accuracy of leading operational NWP models worldwide. This success motivates the exploration of DL-based DA frameworks tailored for weather forecasting models. In this study, we introduces FuxiDA, a generalized DL-based DA framework for assimilating satellite observations. By assimilating data from Advanced Geosynchronous Radiation Imager (AGRI) aboard Fengyun-4B, FuXi-DA consistently mitigates analysis errors and significantly improves forecast performance. Furthermore, through a series of single-observation experiments, Fuxi-DA has been validated against established atmospheric physics, demonstrating its consistency and reliability.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. The quiet revolution of numerical weather prediction. Nature, 525(7567):47–55, 2015.
  2. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.11214, 2022.
  3. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533–538, 2023.
  4. Fuxi: a cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science, 6(1):190, 2023.
  5. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416–1421, 2023.
  6. Vilhelm Bjerknes. Das problem der wettervorhersage, betrachtet vom standpunkte der mechanik und der physik. Meteor. Z., 21:1–7, 1904.
  7. Edward N. Lorenz. Deterministic Nonperiodic Flow. J. Atmos. Sci., 20(2):130–148, March 1963.
  8. Andrew C Lorenc. Analysis methods for numerical weather prediction. Quarterly Journal of the Royal Meteorological Society, 112(474):1177–1194, 1986.
  9. The weather research and forecasting model’s community variational/ensemble data assimilation system: Wrfda. Bulletin of the American Meteorological Society, 93(6):831–843, 2012.
  10. Direct 4d-var assimilation of all-sky radiances. part i: Implementation. Quarterly Journal of the Royal Meteorological Society, 136(652):1868–1885, 2010.
  11. Satellite all-sky infrared radiance assimilation: Recent progress and future perspectives. Advances in Atmospheric Sciences, 39(1):9–21, 2022.
  12. Observations used in ECMWF. https://www.ecmwf.int/en/research/data-assimilation/observations. Accessed: 2024-04-05.
  13. All-sky satellite data assimilation at operational weather forecasting centres. Quarterly Journal of the Royal Meteorological Society, 144(713):1191–1217, 2018.
  14. Assimilation of satellite data in numerical weather prediction. part ii: Recent years. Quarterly Journal of the Royal Meteorological Society, 148(743):521–556, 2022.
  15. Identification and correction of radiative transfer modelling errors for atmospheric sounders: Airs and amsu-a. In ECMWF “Workshop on Assimilation of High Spectral Resolution Sounders in NWP, 2004.
  16. Extended assimilation and forecast experiments with a four-dimensional variational assimilation system. Quarterly Journal of the Royal Meteorological Society, 124(550):1861–1887, 1998.
  17. Ross N Bannister. A review of operational methods of variational and ensemble-variational data assimilation. Quarterly Journal of the Royal Meteorological Society, 143(703):607–633, 2017.
  18. Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review. IEEE/CAA Journal of Automatica Sinica, 10(6):1361–1387, 2023.
  19. Edward N Lorenz. Predictability: A problem partly solved. In Proc. Seminar on predictability, volume 1. Reading, 1996.
  20. Cornelis Boudewijn Vreugdenhil. Numerical methods for shallow-water flow, volume 13. Springer Science & Business Media, 2013.
  21. Variational data assimilation with a learned inverse observation operator. In International Conference on Machine Learning, pages 3449–3458. PMLR, 2021.
  22. Observation error covariance specification in dynamical systems for data assimilation using recurrent neural networks. Neural Computing and Applications, 34(16):13149–13167, 2022.
  23. Neural-network data assimilation using variational autoencoder. arXiv preprint arXiv:2308.16073, 2023.
  24. Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the lorenz 96 model. Journal of computational science, 44:101171, 2020.
  25. Deep data assimilation: integrating deep learning with data assimilation. Applied Sciences, 11(3):1114, 2021.
  26. Learning variational data assimilation models and solvers. Journal of Advances in Modeling Earth Systems, 13(10):e2021MS002572, 2021.
  27. Building tangent-linear and adjoint models for data assimilation with neural networks. Journal of Advances in Modeling Earth Systems, 13(9):e2021MS002521, 2021.
  28. 2022 ecmwf-esa workshop report: current status, progress and opportunities in machine learning for earth system observation and prediction. npj Climate and Atmospheric Science, 6(1):87, 2023.
  29. The growing impact of satellite observations sensitive to humidity, cloud and precipitation. Quarterly Journal of the Royal Meteorological Society, 143(709):3189–3206, 2017.
  30. Joshua Chun Kwang Lee and Xiang-Yu Huang. Modelling the Background Error Covariance Matrix: Applicability Over the Maritime Continent, pages 599–627. Springer International Publishing, Cham, 2022.
  31. Developing the science product algorithm testbed for chinese next-generation geostationary meteorological satellites: Fengyun-4 series. Journal of Meteorological Research, 31(4):708–719, 2017.
  32. Inferring convective weather characteristics with geostationary high spectral resolution ir window measurements: A look into the future. Journal of Atmospheric and Oceanic Technology, 26(8):1527–1541, 2009.
  33. Anthony P McNally. On the sensitivity of a 4d-var analysis system to satellite observations located at different times within the assimilation window. Quarterly Journal of the Royal Meteorological Society, 145(723):2806–2816, 2019.
  34. Automated meteorological reports from commercial aircraft. Bulletin of the American Meteorological Society, 84(2):203–216, 2003.
  35. Progress toward high-resolution, real-time radiosonde reports. Bulletin of the American Meteorological Society, 97(11):2149–2161, 2016.
  36. Fengyun-4B/AGRI observation. http://www.nsmc.org.cn/nsmc/en/instrument/AGRI.html. Accessed: 2024-04-11.
  37. Enhanced use of all-sky microwave observations sensitive to water vapour, cloud and precipitation, volume 620. ECMWF Reading, UK, 2010.
  38. The assimilation of ssmis radiances in numerical weather prediction models. IEEE Transactions on Geoscience and Remote Sensing, 46(4):884–900, 2008.
  39. Adaptive bias correction for satellite data in a numerical weather prediction system. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 133(624):631–642, 2007.
  40. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer, 2015.
  41. BA Harris and G Kelly. A satellite radiance-bias correction scheme for data assimilation. Quarterly Journal of the Royal Meteorological Society, 127(574):1453–1468, 2001.
  42. The evaluation of fy4a’s geostationary interferometric infrared sounder (giirs) long-wave temperature sounding channels using the grapes global 4d-var. Quarterly Journal of the Royal Meteorological Society, 146(728):1459–1476, 2020.
  43. JR Eyre. A bias correction scheme for simulated tovs brightness temperatures. ECMWF Tech. Memo. 186., 1992.
  44. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  45. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3–11, 2018.
  46. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1874–1883, 2016.
  47. Automatic differentiation in pytorch. 2017.
  48. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  49. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.
Citations (5)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com