Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SFTformer: A Spatial-Frequency-Temporal Correlation-Decoupling Transformer for Radar Echo Extrapolation (2402.18044v1)

Published 28 Feb 2024 in cs.CV

Abstract: Extrapolating future weather radar echoes from past observations is a complex task vital for precipitation nowcasting. The spatial morphology and temporal evolution of radar echoes exhibit a certain degree of correlation, yet they also possess independent characteristics. {Existing methods learn unified spatial and temporal representations in a highly coupled feature space, emphasizing the correlation between spatial and temporal features but neglecting the explicit modeling of their independent characteristics, which may result in mutual interference between them.} To effectively model the spatiotemporal dynamics of radar echoes, we propose a Spatial-Frequency-Temporal correlation-decoupling Transformer (SFTformer). The model leverages stacked multiple SFT-Blocks to not only mine the correlation of the spatiotemporal dynamics of echo cells but also avoid the mutual interference between the temporal modeling and the spatial morphology refinement by decoupling them. Furthermore, inspired by the practice that weather forecast experts effectively review historical echo evolution to make accurate predictions, SFTfomer incorporates a joint training paradigm for historical echo sequence reconstruction and future echo sequence prediction. Experimental results on the HKO-7 dataset and ChinaNorth-2021 dataset demonstrate the superior performance of SFTfomer in short(1h), mid(2h), and long-term(3h) precipitation nowcasting.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. W.-c. Woo and W.-k. Wong, “Operational application of optical flow techniques to radar-based rainfall nowcasting,” Atmosphere, vol. 8, no. 3, p. 48, 2017.
  2. G. Ayzel, M. Heistermann, and T. Winterrath, “Optical flow models as an open benchmark for radar-based precipitation nowcasting (rainymotion v0. 1),” Geoscientific Model Development, vol. 12, no. 4, pp. 1387–1402, 2019.
  3. M. Dixon and G. Wiener, “Titan: Thunderstorm identification, tracking, analysis, and nowcasting—a radar-based methodology,” Journal of atmospheric and oceanic technology, vol. 10, no. 6, pp. 785–797, 1993.
  4. J. Johnson, P. L. MacKeen, A. Witt, E. D. W. Mitchell, G. J. Stumpf, M. D. Eilts, and K. W. Thomas, “The storm cell identification and tracking algorithm: An enhanced wsr-88d algorithm,” Weather and forecasting, vol. 13, no. 2, pp. 263–276, 1998.
  5. A. del Moral, T. Rigo, and M. C. Llasat, “A radar-based centroid tracking algorithm for severe weather surveillance: Identifying split/merge processes in convective systems,” Atmospheric Research, vol. 213, pp. 110–120, 2018.
  6. L. Li, W. Schmid, and J. Joss, “Nowcasting of motion and growth of precipitation with radar over a complex orography,” Journal of applied meteorology and climatology, vol. 34, no. 6, pp. 1286–1300, 1995.
  7. S. Mecklenburg, J. Joss, and W. Schmid, “Improving the nowcasting of precipitation in an alpine region with an enhanced radar echo tracking algorithm,” Journal of Hydrology, vol. 239, no. 1-4, pp. 46–68, 2000.
  8. Q. Liang, Y. Feng, W. Deng, S. Hu, Y. Huang, Q. Zeng, and Z. Chen, “A composite approach of radar echo extrapolation based on trec vectors in combination with model-predicted winds,” Advances in Atmospheric Sciences, vol. 27, no. 5, pp. 1119–1130, 2010.
  9. M. Babaeizadeh, C. Finn, D. Erhan, R. H. Campbell, and S. Levine, “Stochastic variational video prediction,” arXiv preprint arXiv:1710.11252, 2017.
  10. Y. Wang, J. Zhang, H. Zhu, M. Long, J. Wang, and P. S. Yu, “Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9154–9162, 2019.
  11. N. Srivastava, E. Mansimov, and R. Salakhudinov, “Unsupervised learning of video representations using lstms,” in International conference on machine learning, pp. 843–852, PMLR, 2015.
  12. Y. Wang, L. Jiang, M.-H. Yang, L.-J. Li, M. Long, and L. Fei-Fei, “Eidetic 3d lstm: A model for video prediction and beyond,” in International conference on learning representations, 2018.
  13. D. Weissenborn, O. Täckström, and J. Uszkoreit, “Scaling autoregressive video models,” arXiv preprint arXiv:1906.02634, 2019.
  14. R. Rakhimov, D. Volkhonskiy, A. Artemov, D. Zorin, and E. Burnaev, “Latent video transformer,” arXiv preprint arXiv:2006.10704, 2020.
  15. M. Ranzato, A. Szlam, J. Bruna, M. Mathieu, R. Collobert, and S. Chopra, “Video (language) modeling: a baseline for generative models of natural videos,” arXiv preprint arXiv:1412.6604, 2014.
  16. N. Kalchbrenner, A. Oord, K. Simonyan, I. Danihelka, O. Vinyals, A. Graves, and K. Kavukcuoglu, “Video pixel networks,” in International Conference on Machine Learning, pp. 1771–1779, PMLR, 2017.
  17. X. Shi, Z. Chen, H. Wang, D. Yeung, W. Wong, and W. Woo, “Convolutional LSTM network: A machine learning approach for precipitation nowcasting,” CoRR, vol. abs/1506.04214, 2015.
  18. Y. Wang, H. Wu, J. Zhang, Z. Gao, J. Wang, P. S. Yu, and M. Long, “Predrnn: A recurrent neural network for spatiotemporal predictive learning,” CoRR, vol. abs/2103.09504, 2021.
  19. X. Shi, Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-k. Wong, and W.-c. Woo, “Deep learning for precipitation nowcasting: A benchmark and a new model,” Advances in neural information processing systems, vol. 30, 2017.
  20. H. Wu, Z. Yao, J. Wang, and M. Long, “Motionrnn: A flexible model for video prediction with spacetime-varying motions,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15435–15444, 2021.
  21. L. R. Medsker and L. Jain, “Recurrent neural networks,” Design and Applications, vol. 5, no. 64-67, p. 2, 2001.
  22. Y. Wang, Z. Gao, M. Long, J. Wang, and P. S. Yu, “Predrnn++: Towards A resolution of the deep-in-time dilemma in spatiotemporal predictive learning,” CoRR, vol. abs/1804.06300, 2018.
  23. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” CoRR, vol. abs/1706.03762, 2017.
  24. H. Bi, Y. Feng, Z. Yan, Y. Mao, W. Diao, H. Wang, and X. Sun, “Not just learning from others but relying on yourself: A new perspective on few-shot segmentation in remote sensing,” IEEE Transactions on Geoscience and Remote Sensing, 2023.
  25. Y. Mao, K. Chen, W. Diao, X. Sun, X. Lu, K. Fu, and M. Weinmann, “Beyond single receptive field: A receptive field fusion-and-stratification network for airborne laser scanning point cloud classification,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 188, pp. 45–61, 2022.
  26. Y. Mao, K. Chen, L. Zhao, W. Chen, D. Tang, W. Liu, Z. Wang, W. Diao, X. Sun, and K. Fu, “Elevation estimation-driven building 3d reconstruction from single-view remote sensing imagery,” IEEE Transactions on Geoscience and Remote Sensing, 2023.
  27. Y. Yao, T. Chen, H. Bi, X. Cai, G. Pei, G. Yang, Z. Yan, X. Sun, X. Xu, and H. Zhang, “Automated object recognition in high-resolution optical remote sensing imagery,” National Science Review, vol. 10, no. 6, p. nwad122, 2023.
  28. F. Yao, W. Lu, H. Yang, L. Xu, C. Liu, L. Hu, H. Yu, N. Liu, C. Deng, D. Tang, et al., “Ringmo-sense: Remote sensing foundation model for spatiotemporal prediction via spatiotemporal evolution disentangling,” IEEE Transactions on Geoscience and Remote Sensing, 2023.
  29. X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” Advances in neural information processing systems, vol. 28, 2015.
  30. J. Jing, Q. Li, X. Peng, Q. Ma, and S. Tang, “Hprnn: A hierarchical sequence prediction model for long-term weather radar echo extrapolation,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4142–4146, IEEE, 2020.
  31. J. Jing, Q. Li, L. Ma, L. Chen, and L. Ding, “Remnet: Recurrent evolution memory-aware network for accurate long-term weather radar echo extrapolation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–13, 2022.
  32. Y. Wang, M. Long, J. Wang, Z. Gao, and P. S. Yu, “Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms,” Advances in neural information processing systems, vol. 30, 2017.
  33. Y. Wang, Z. Gao, M. Long, J. Wang, and S. Y. Philip, “Predrnn++: Towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning,” in International Conference on Machine Learning, pp. 5123–5132, PMLR, 2018.
  34. Y. Wang, H. Wu, J. Zhang, Z. Gao, J. Wang, S. Y. Philip, and M. Long, “Predrnn: A recurrent neural network for spatiotemporal predictive learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 2, pp. 2208–2225, 2022.
  35. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  36. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, pp. 11106–11115, 2021.
  37. H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,” Advances in Neural Information Processing Systems, vol. 34, pp. 22419–22430, 2021.
  38. I. Beltagy, M. E. Peters, and A. Cohan, “Longformer: The long-document transformer,” arXiv preprint arXiv:2004.05150, 2020.
  39. D. So, Q. Le, and C. Liang, “The evolved transformer,” in International conference on machine learning, pp. 5877–5886, PMLR, 2019.
  40. W. Hua, Z. Dai, H. Liu, and Q. Le, “Transformer quality in linear time,” in International Conference on Machine Learning, pp. 9099–9117, PMLR, 2022.
  41. T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting,” in International Conference on Machine Learning, pp. 27268–27286, PMLR, 2022.
  42. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
  43. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012–10022, 2021.
  44. Z. Tong, Y. Song, J. Wang, and L. Wang, “Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training,” Advances in neural information processing systems, vol. 35, pp. 10078–10093, 2022.
  45. Z. Gao, C. Tan, L. Wu, and S. Z. Li, “Simvp: Simpler yet better video prediction,” 2022.
  46. Z. Chang, X. Zhang, S. Wang, S. Ma, Y. Ye, X. Xinguang, and W. Gao, “Mau: A motion-aware unit for video prediction and beyond,” Advances in Neural Information Processing Systems, vol. 34, pp. 26950–26962, 2021.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now