Papers
Topics
Authors
Recent
Search
2000 character limit reached

MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling

Published 20 Aug 2024 in physics.ao-ph, cs.AI, and cs.CV | (2408.10854v1)

Abstract: In an era of frequent extreme weather and global warming, obtaining precise, fine-grained near-surface weather forecasts is increasingly essential for human activities. Downscaling (DS), a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions from global-scale forecast results. Previous downscaling methods, inspired by CNN and Transformer-based super-resolution models, lacked tailored designs for meteorology and encountered structural limitations. Notably, they failed to efficiently integrate topography, a crucial prior in the downscaling process. In this paper, we address these limitations by pioneering the selective state space model into the meteorological field downscaling and propose a novel model called MambaDS. This model enhances the utilization of multivariable correlations and topography information, unique challenges in the downscaling process while retaining the advantages of Mamba in long-range dependency modeling and linear computational complexity. Through extensive experiments in both China mainland and the continental United States (CONUS), we validated that our proposed MambaDS achieves state-of-the-art results in three different types of meteorological field downscaling settings. We will release the code subsequently.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (80)
  1. P. Stott, “How climate change affects extreme weather events,” Science, vol. 352, no. 6293, pp. 1517–1518, 2016.
  2. C. Lesk, P. Rowhani, and N. Ramankutty, “Influence of extreme weather disasters on global crop production,” Nature, vol. 529, no. 7584, pp. 84–87, 2016.
  3. P. Bauer, A. Thorpe, and G. Brunet, “The quiet revolution of numerical weather prediction,” Nature, vol. 525, no. 7567, pp. 47–55, 2015.
  4. R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, F. Alet, S. Ravuri, T. Ewalds, Z. Eaton-Rosen, W. Hu et al., “Learning skillful medium-range global weather forecasting,” Science, vol. 382, no. 6677, pp. 1416–1421, 2023.
  5. K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Accurate medium-range global weather forecasting with 3d neural networks,” Nature, vol. 619, no. 7970, pp. 533–538, 2023.
  6. K. Chen, T. Han, J. Gong, L. Bai, F. Ling, J.-J. Luo, X. Chen, L. Ma, T. Zhang, R. Su et al., “Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead,” arXiv preprint arXiv:2304.02948, 2023.
  7. T. Kurth, S. Subramanian, P. Harrington, J. Pathak, M. Mardani, D. Hall, A. Miele, K. Kashinath, and A. Anandkumar, “Fourcastnet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators,” in Proceedings of the platform for advanced scientific computing conference, 2023, pp. 1–11.
  8. D. Kochkov, J. Yuval, I. Langmore, P. Norgaard, J. Smith, G. Mooers, M. Klöwer, J. Lottes, S. Rasp, P. Düben et al., “Neural general circulation models for weather and climate,” Nature, pp. 1–7, 2024.
  9. F. Ling, Z. Lu, J.-J. Luo, L. Bai, S. K. Behera, D. Jin, B. Pan, H. Jiang, and T. Yamagata, “Diffusion model-based probabilistic downscaling for 180-year east asian climate reconstruction,” npj Climate and Atmospheric Science, vol. 7, no. 1, p. 131, 2024.
  10. E. R. Rodrigues, I. Oliveira, R. Cunha, and M. Netto, “Deepdownscale: A deep learning strategy for high-resolution weather forecast,” in 2018 IEEE 14th International Conference on e-Science (e-Science).   IEEE, 2018, pp. 415–422.
  11. J. Younis, S. Anquetin, and J. Thielen, “The benefit of high-resolution operational weather forecasts for flash flood warning,” Hydrology and Earth System Sciences, vol. 12, no. 4, pp. 1039–1051, 2008.
  12. A. d. De Wit, H. Boogaard, and C. Van Diepen, “Spatial resolution of precipitation and radiation: the effect on regional crop yield forecasts,” Agricultural and Forest Meteorology, vol. 135, no. 1-4, pp. 156–168, 2005.
  13. R. L. Wilby and T. M. Wigley, “Downscaling general circulation model output: a review of methods and limitations,” Progress in physical geography, vol. 21, no. 4, pp. 530–548, 1997.
  14. Z. Xu, Y. Han, and Z. Yang, “Dynamical downscaling of regional climate: A review of methods and limitations,” Science China Earth Sciences, vol. 62, pp. 365–375, 2019.
  15. Y. Sun, K. Deng, K. Ren, J. Liu, C. Deng, and Y. Jin, “Deep learning in statistical downscaling for deriving high spatial resolution gridded meteorological data: A systematic review,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 208, pp. 14–38, 2024.
  16. R. A. Pielke Sr and R. L. Wilby, “Regional climate downscaling: What’s the point?” Eos, Transactions American Geophysical Union, vol. 93, no. 5, pp. 52–53, 2012.
  17. J. T. Schoof and S. Pryor, “Downscaling temperature and precipitation: A comparison of regression-based methods and artificial neural networks,” International Journal of Climatology: A Journal of the Royal Meteorological Society, vol. 21, no. 7, pp. 773–790, 2001.
  18. R. J. Davy, M. J. Woods, C. J. Russell, and P. A. Coppin, “Statistical downscaling of wind variability from meteorological fields,” Boundary-layer meteorology, vol. 135, pp. 161–175, 2010.
  19. H. Chen, J. Guo, W. Xiong, S. Guo, and C.-Y. Xu, “Downscaling gcms using the smooth support vector machine method to predict daily precipitation in the hanjiang basin,” Advances in Atmospheric Sciences, vol. 27, pp. 274–284, 2010.
  20. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
  21. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  22. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  23. C. Irrgang, N. Boers, M. Sonnewald, E. A. Barnes, C. Kadow, J. Staneva, and J. Saynisch-Wagner, “Towards neural earth system modelling by integrating artificial intelligence in earth system science,” Nature Machine Intelligence, vol. 3, no. 8, pp. 667–674, 2021.
  24. Z. Wang, J. Chen, and S. C. Hoi, “Deep learning for image super-resolution: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 10, pp. 3365–3387, 2020.
  25. T. Vandal, E. Kodra, S. Ganguly, A. Michaelis, R. Nemani, and A. R. Ganguly, “Deepsd: Generating high resolution climate change projections through single image super-resolution,” in Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, 2017, pp. 1663–1672.
  26. K. Höhlein, M. Kern, T. Hewson, and R. Westermann, “A comparative study of convolutional neural network models for wind field downscaling,” Meteorological Applications, vol. 27, no. 6, p. e1961, 2020.
  27. L. Sun and Y. Lan, “Statistical downscaling of daily temperature and precipitation over china using deep learning neural models: Localization and comparison with other methods,” International Journal of Climatology, vol. 41, no. 2, pp. 1128–1147, 2021.
  28. R. Tie, C. Shi, G. Wan, X. Hu, L. Kang, and L. Ge, “Cldassd: reconstructing fine textures of the temperature field using super-resolution technology,” Advances in Atmospheric Sciences, vol. 39, no. 1, pp. 117–130, 2022.
  29. K. Stengel, A. Glaws, D. Hettinger, and R. N. King, “Adversarial super-resolution of climatological wind and solar data,” Proceedings of the National Academy of Sciences, vol. 117, no. 29, pp. 16 805–16 815, 2020.
  30. L. Harris, A. T. McRae, M. Chantry, P. D. Dueben, and T. N. Palmer, “A generative deep learning approach to stochastic downscaling of precipitation forecasts,” Journal of Advances in Modeling Earth Systems, vol. 14, no. 10, p. e2022MS003120, 2022.
  31. P. Hess, M. Drüke, S. Petri, F. M. Strnad, and N. Boers, “Physically constrained generative adversarial networks for improving precipitation fields from earth system models,” Nature Machine Intelligence, vol. 4, no. 10, pp. 828–839, 2022.
  32. J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte, “Swinir: Image restoration using swin transformer,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 1833–1844.
  33. Z. Wang, X. Cun, J. Bao, W. Zhou, J. Liu, and H. Li, “Uformer: A general u-shaped transformer for image restoration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 17 683–17 693.
  34. F. Gerges, M. C. Boufadel, E. Bou-Zeid, H. Nassif, and J. T. Wang, “A novel deep learning approach to the statistical downscaling of temperatures for monitoring climate change,” in Proceedings of the 2022 6th International Conference on Machine Learning and Soft Computing, 2022, pp. 1–7.
  35. L. Xiang, J. Guan, J. Xiang, L. Zhang, and F. Zhang, “Spatiotemporal model based on transformer for bias correction and temporal downscaling of forecasts,” Frontiers in Environmental Science, vol. 10, p. 1039764, 2022.
  36. X. Zhong, F. Du, L. Chen, Z. Wang, and H. Li, “Investigating transformer-based models for spatial downscaling and correcting biases of near-surface temperature and wind-speed forecasts,” Quarterly Journal of the Royal Meteorological Society, vol. 150, no. 758, pp. 275–289, 2024.
  37. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  38. Q. Fournier, G. M. Caron, and D. Aloise, “A practical survey on faster and lighter transformers,” ACM Computing Surveys, vol. 55, no. 14s, pp. 1–40, 2023.
  39. A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023.
  40. X. Wang, S. Wang, Y. Ding, Y. Li, W. Wu, Y. Rong, W. Kong, J. Huang, S. Li, H. Yang et al., “State space model for new-generation network alternative to transformers: A survey,” arXiv preprint arXiv:2404.09516, 2024.
  41. B. N. Patro and V. S. Agneeswaran, “Mamba-360: Survey of state space models as transformer alternative for long sequence modelling: Methods, applications, and challenges,” arXiv preprint arXiv:2404.16112, 2024.
  42. L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: Efficient visual representation learning with bidirectional state space model,” arXiv preprint arXiv:2401.09417, 2024.
  43. R. Xu, S. Yang, Y. Wang, B. Du, and H. Chen, “A survey on vision mamba: Models, applications and challenges,” arXiv preprint arXiv:2404.18861, 2024.
  44. H. Guo, J. Li, T. Dai, Z. Ouyang, X. Ren, and S.-T. Xia, “Mambair: A simple baseline for image restoration with state-space model,” arXiv preprint arXiv:2402.15648, 2024.
  45. Y. Shi, B. Xia, X. Jin, X. Wang, T. Zhao, X. Xia, X. Xiao, and W. Yang, “Vmambair: Visual state space model for image restoration,” arXiv preprint arXiv:2403.11423, 2024.
  46. Y. Lu, S. Wang, Z. Wang, P. Xia, T. Zhou et al., “Lfmamba: Light field image super-resolution with state space model,” arXiv preprint arXiv:2406.12463, 2024.
  47. Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang, Q. Ye, and Y. Liu, “Vmamba: Visual state space model,” 2024. [Online]. Available: https://arxiv.org/abs/2401.10166
  48. H. Cao, Y. Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-unet: Unet-like pure transformer for medical image segmentation,” in European conference on computer vision.   Springer, 2022, pp. 205–218.
  49. H. Hersbach, B. Bell, P. Berrisford, S. Hirahara, A. Horányi, J. Muñoz-Sabater, J. Nicolas, C. Peubey, R. Radu, D. Schepers et al., “The era5 global reanalysis,” Quarterly Journal of the Royal Meteorological Society, vol. 146, no. 730, pp. 1999–2049, 2020.
  50. D. C. Dowell, C. R. Alexander, E. P. James, S. S. Weygandt, S. G. Benjamin, G. S. Manikin, B. T. Blake, J. M. Brown, J. B. Olson, M. Hu et al., “The high-resolution rapid refresh (hrrr): An hourly updating convection-allowing forecast model. part i: Motivation and system description,” Weather and Forecasting, vol. 37, no. 8, pp. 1371–1395, 2022.
  51. C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295–307, 2015.
  52. J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1646–1654.
  53. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18.   Springer, 2015, pp. 234–241.
  54. Y. Sha, D. J. Gagne II, G. West, and R. Stull, “Deep-learning-based gridded downscaling of surface meteorological variables in complex terrain. part i: Daily maximum and minimum 2-m temperature,” Journal of Applied Meteorology and Climatology, vol. 59, no. 12, pp. 2057–2073, 2020.
  55. A. Doury, S. Somot, S. Gadat, A. Ribes, and L. Corre, “Regional climate model emulator based on deep learning: Concept and first evaluation of a novel hybrid downscaling approach,” Climate Dynamics, vol. 60, no. 5, pp. 1751–1779, 2023.
  56. Y. Sha, D. J. Gagne II, G. West, and R. Stull, “Deep-learning-based gridded downscaling of surface meteorological variables in complex terrain. part ii: Daily precipitation,” Journal of Applied Meteorology and Climatology, vol. 59, no. 12, pp. 2075–2092, 2020.
  57. A. Y. Sun and G. Tang, “Downscaling satellite and reanalysis precipitation products using attention-based deep convolutional neural nets,” Frontiers in Water, vol. 2, p. 536743, 2020.
  58. C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang et al., “Photo-realistic single image super-resolution using a generative adversarial network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4681–4690.
  59. Z. Liu, H. Chen, L. Bai, W. Li, K. Chen, Z. Wang, W. Ouyang, Z. Zou, and Z. Shi, “Observation-guided meteorological field downscaling at station scale: A benchmark and a new method,” arXiv preprint arXiv:2401.11960, 2024.
  60. W. Li, Z. Liu, K. Chen, H. Chen, S. Liang, Z. Zou, and Z. Shi, “Deepphysinet: Bridging deep learning and atmospheric physics for accurate and continuous weather modeling,” arXiv preprint arXiv:2401.04125, 2024.
  61. K. Chen, B. Chen, C. Liu, W. Li, Z. Zou, and Z. Shi, “Rsmamba: Remote sensing image classification with state space model,” IEEE Geoscience and Remote Sensing Letters, 2024.
  62. Z. Wan, Y. Wang, S. Yong, P. Zhang, S. Stepputtis, K. Sycara, and Y. Xie, “Sigma: Siamese mamba network for multi-modal semantic segmentation,” arXiv preprint arXiv:2404.04256, 2024.
  63. Q. Zhu, Y. Cai, Y. Fang, Y. Yang, C. Chen, L. Fan, and A. Nguyen, “Samba: Semantic segmentation of remotely sensed images with state space model,” arXiv preprint arXiv:2404.01705, 2024.
  64. S. Zhao, H. Chen, X. Zhang, P. Xiao, L. Bai, and W. Ouyang, “Rs-mamba for large remote sensing image dense prediction,” arXiv preprint arXiv:2404.02668, 2024.
  65. H. Zhang, K. Chen, C. Liu, H. Chen, Z. Zou, and Z. Shi, “Cdmamba: Remote sensing image change detection with mamba,” arXiv preprint arXiv:2406.04207, 2024.
  66. J. Ma, F. Li, and B. Wang, “U-mamba: Enhancing long-range dependency for biomedical image segmentation,” arXiv preprint arXiv:2401.04722, 2024.
  67. Y. Yang, Z. Xing, and L. Zhu, “Vivim: a video vision mamba for medical video object segmentation,” arXiv preprint arXiv:2401.14168, 2024.
  68. Y. Xiao, Q. Yuan, K. Jiang, Y. Chen, Q. Zhang, and C.-W. Lin, “Frequency-assisted mamba for remote sensing image super-resolution,” arXiv preprint arXiv:2405.04964, 2024.
  69. J. D. Hamilton, “State-space models,” Handbook of econometrics, vol. 4, pp. 3039–3080, 1994.
  70. L. R. Medsker, L. Jain et al., “Recurrent neural networks,” Design and Applications, vol. 5, no. 64-67, p. 2, 2001.
  71. K. Chen, T. Chen, P. Ye, H. Chen, K. Chen, T. Han, W. Ouyang, and L. Bai, “Fnp: Fourier neural processes for arbitrary-resolution data assimilation,” arXiv preprint arXiv:2406.01645, 2024.
  72. J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016.
  73. P. Harder, A. Hernandez-Garcia, V. Ramesh, Q. Yang, P. Sattegeri, D. Szwarcman, C. Watson, and D. Rolnick, “Hard-constrained deep learning for climate downscaling,” Journal of Machine Learning Research, vol. 24, no. 365, pp. 1–40, 2023.
  74. M. MacFerrin, C. Amante, K. Carignan, M. Love, and E. Lim, “The earth topography 2022 (etopo 2022) global dem dataset,” Earth System Science Data Discussions, vol. 2024, pp. 1–24, 2024.
  75. P. Charbonnier, L. Blanc-Feraud, G. Aubert, and M. Barlaud, “Two deterministic half-quadratic regularization algorithms for computed imaging,” in Proceedings of 1st international conference on image processing, vol. 2.   IEEE, 1994, pp. 168–172.
  76. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  77. B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 136–144.
  78. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, “Image super-resolution using very deep residual channel attention networks,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 286–301.
  79. P. Ren, N. B. Erichson, S. Subramanian, O. San, Z. Lukic, and M. W. Mahoney, “Superbench: A super-resolution benchmark dataset for scientific machine learning,” arXiv preprint arXiv:2306.14070, 2023.
  80. S. Rasp, P. D. Dueben, S. Scher, J. A. Weyn, S. Mouatadid, and N. Thuerey, “Weatherbench: a benchmark data set for data-driven weather forecasting,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 11, p. e2020MS002203, 2020.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up