SRNDiff: Short-term Rainfall Nowcasting with Condition Diffusion Model (2402.13737v1)
Abstract: Diffusion models are widely used in image generation because they can generate high-quality and realistic samples. This is in contrast to generative adversarial networks (GANs) and variational autoencoders (VAEs), which have some limitations in terms of image quality.We introduce the diffusion model to the precipitation forecasting task and propose a short-term precipitation nowcasting with condition diffusion model based on historical observational data, which is referred to as SRNDiff. By incorporating an additional conditional decoder module in the denoising process, SRNDiff achieves end-to-end conditional rainfall prediction. SRNDiff is composed of two networks: a denoising network and a conditional Encoder network. The conditional network is composed of multiple independent UNet networks. These networks extract conditional feature maps at different resolutions, providing accurate conditional information that guides the diffusion model for conditional generation.SRNDiff surpasses GANs in terms of prediction accuracy, although it requires more computational resources.The SRNDiff model exhibits higher stability and efficiency during training than GANs-based approaches, and generates high-quality precipitation distribution samples that better reflect future actual precipitation conditions. This fully validates the advantages and potential of diffusion models in precipitation forecasting, providing new insights for enhancing rainfall prediction.
- Nowcasting challenges during the beijing olympics: Successes, failures, and implications for future nowcasting systems. Weather and Forecasting, 25(6):1691–1714, 2010.
- Improving high-impact numerical weather prediction with lidar and drone observations. Bulletin of the American Meteorological Society, 101(7):E1036–E1051, 2020.
- Making the black box more transparent: Understanding the physical implications of machine learning. Bulletin of the American Meteorological Society, 100(11):2175–2199, 2019.
- Learning spatiotemporal features with 3d convolutional networks. Proceedings of the IEEE international conference on computer vision, 379(2194):4489–4497, 2015.
- Prediction of flow based on a cnn-lstm combined deep learning approach. Water, 14:993, 2022.
- Recurrent flow networks: A recurrent latent variable model for density estimation of urban mobility. Pattern Recognition, 129:108752, 2022.
- Smaat-unet: Precipitation nowcasting using a small attention-unet architecture. Pattern Recognition Letters, 145:178–186, 2021.
- Skilful precipitation nowcasting using deep generative models of radar. Nature, 597(7878):672–677, 2021.
- Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
- Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
- Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494, 2022.
- High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022.
- U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234–241. Springer, 2015.
- Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.
- U2-net: Going deeper with nested u-structure for salient object detection. Pattern recognition, 106:107404, 2020.
- A spatiotemporal deep learning model st-lstm-sa for hourly rainfall forecasting using radar echo images. Journal of Hydrology, 609:127748, 2022.
- Deep learning for precipitation nowcasting: A benchmark and a new model. Advances in neural information processing systems, 30, 2017.
- The reconstitution predictive network for precipitation nowcasting. Neurocomputing, 507:1–15, 2022.
- Robust recurrent neural networks for time series forecasting. Neurocomputing, 526:143–157, 2023.
- Dual motion gan for future-flow embedded video prediction. In proceedings of the IEEE international conference on computer vision, pages 1744–1752, 2017.
- Mocogan: Decomposing motion and content for video generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1526–1535, 2018.
- Imaginator: Conditional spatio-temporal gan for video generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1160–1169, 2020.
- Two-stage ua-gan for precipitation nowcasting. Remote Sensing, 14(23):5948, 2022.
- Increasing the accuracy and resolution of precipitation forecasts using deep generative models. In International conference on artificial intelligence and statistics, pages 10555–10571. PMLR, 2022.
- A generative deep learning approach to stochastic downscaling of precipitation forecasts. Journal of Advances in Modeling Earth Systems, 14(10):e2022MS003120, 2022.
- Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
- Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR, 2015.
- Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
- Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
- Video diffusion models, 2022.
- Medsegdiff: Medical image segmentation with diffusion probabilistic model. In Medical Imaging with Deep Learning, 2023.
- Rotate to attend: Convolutional triplet attention module. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3139–3148, 2021.