Distribution-aware Interactive Attention Network and Large-scale Cloud Recognition Benchmark on FY-4A Satellite Image (2401.03182v1)
Abstract: Accurate cloud recognition and warning are crucial for various applications, including in-flight support, weather forecasting, and climate research. However, recent deep learning algorithms have predominantly focused on detecting cloud regions in satellite imagery, with insufficient attention to the specificity required for accurate cloud recognition. This limitation inspired us to develop the novel FY-4A-Himawari-8 (FYH) dataset, which includes nine distinct cloud categories and uses precise domain adaptation methods to align 70,419 image-label pairs in terms of projection, temporal resolution, and spatial resolution, thereby facilitating the training of supervised deep learning networks. Given the complexity and diversity of cloud formations, we have thoroughly analyzed the challenges inherent to cloud recognition tasks, examining the intricate characteristics and distribution of the data. To effectively address these challenges, we designed a Distribution-aware Interactive-Attention Network (DIAnet), which preserves pixel-level details through a high-resolution branch and a parallel multi-resolution cross-branch. We also integrated a distribution-aware loss (DAL) to mitigate the imbalance across cloud categories. An Interactive Attention Module (IAM) further enhances the robustness of feature extraction combined with spatial and channel information. Empirical evaluations on the FYH dataset demonstrate that our method outperforms other cloud recognition networks, achieving superior performance in terms of mean Intersection over Union (mIoU). The code for implementing DIAnet is available at https://github.com/icey-zhang/DIAnet.
- X. Wang, M. Min, F. Wang, J. Guo, B. Li, and S. Tang, “Intercomparisons of cloud mask products among fengyun-4a, himawari-8, and modis,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 11, pp. 8827–8839, 2019.
- Y. Sun, L. Su, Y. Luo, H. Meng, Z. Zhang, W. Zhang, and S. Yuan, “Irdclnet: Instance segmentation of ship images based on interference reduction and dynamic contour learning in foggy scenes,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 9, pp. 6029–6043, 2022.
- Y. Sun, L. Su, S. Yuan, and H. Meng, “Danet: Dual-branch activation network for small object instance segmentation of ship images,” IEEE Trans. Circuits Syst. Video Technol., 2023.
- L. Zhang, X. Zhang, Q. Wang, W. Wu, X. Chang, and J. Liu, “Rpmg-fss: Robust prior mask guided few-shot semantic segmentation,” IEEE Trans. Circuits Syst. Video Technol., 2023.
- Z. Li, H. Shen, H. Li, G. Xia, P. Gamba, and L. Zhang, “Multi-feature combined cloud and cloud shadow detection in gaofen-1 wide field of view imagery,” Remote sens. of environ., vol. 191, pp. 342–358, 2017.
- C. Gerhardt, F. Weidner, and W. Broll, “Skycloud: Neural network-based sky and cloud segmentation from natural images,” in International Conference on Image, Vision and Computing (ICIVC). IEEE, 2023, pp. 343–351.
- Z. Li, H. Shen, Q. Cheng, Y. Liu, S. You, and Z. He, “Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors,” ISPRS J. Photogramm. Remote Sens., vol. 150, pp. 197–212, 2019.
- S. Platnick, M. D. King, S. A. Ackerman, W. P. Menzel, B. A. Baum, J. C. Riédi, and R. A. Frey, “The modis cloud products: Algorithms and examples from terra,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 2, pp. 459–473, 2003.
- O. Hagolle, M. Huc, D. V. Pascual, and G. Dedieu, “A multi-temporal method for cloud detection, applied to formosat-2, venμ𝜇\muitalic_μs, landsat and sentinel-2 images,” Remote Sens. of Environ., vol. 114, no. 8, pp. 1747–1755, 2010.
- J. Zhang, H. Wang, Y. Wang, Q. Zhou, and Y. Li, “Deep network based on up and down blocks using wavelet transform and successive multi-scale spatial attention for cloud detection,” Remote Sens. of Environ., vol. 261, p. 112483, 2021.
- J. Zhang, J. Wu, H. Wang, Y. Wang, and Y. Li, “Cloud detection method using cnn based on cascaded feature attention and channel attention,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1–17, 2021.
- Y. Chen, Q. Weng, L. Tang, L. Wang, H. Xing, and Q. Liu, “Developing an intelligent cloud attention network to support global urban green spaces mapping,” ISPRS J. Photogramm. Remote Sens., vol. 198, pp. 197–209, 2023.
- Z. Li, H. Shen, Q. Weng, Y. Zhang, P. Dou, and L. Zhang, “Cloud and cloud shadow detection for optical satellite imagery: Features, algorithms, validation, and prospects,” ISPRS J. Photogramm. Remote Sens., vol. 188, pp. 89–108, 2022.
- S. Dev, Y. H. Lee, and S. Winkler, “Multi-level semantic labeling of sky/cloud images,” in IEEE International Conference on Image Processing (ICIP). IEEE, 2015, pp. 636–640.
- A. Taravat, F. Del Frate, C. Cornaro, and S. Vergari, “Neural networks and support vector machine algorithms for automatic cloud classification of whole-sky ground-based images,” IEEE Geosci. Remote Sens. Lett., vol. 12, no. 3, pp. 666–670, 2014.
- W. Zhuo, Z. Cao, and Y. Xiao, “Cloud classification of ground-based images using texture–structure features,” Journal of Atmospheric and Oceanic Technology, vol. 31, no. 1, pp. 79–92, 2014.
- M. J. Hughes and D. J. Hayes, “Automated detection of cloud and cloud shadow in single-date landsat imagery using neural networks and spatial post-processing,” Remote Sens., vol. 6, no. 6, pp. 4907–4926, 2014.
- S. Mohajerani and P. Saeedi, “Cloud-net: An end-to-end cloud detection algorithm for landsat 8 imagery,” in IEEE International Geoscience and Remote Sensing Symposium (IGARSS), July 2019, pp. 1029–1032.
- M. Domnich, I. Sünter, H. Trofimov, O. Wold, F. Harun, A. Kostiukhin, M. Järveoja, M. Veske, T. Tamm, K. Voormansik et al., “Kappamask: Ai-based cloudmask processor for sentinel-2,” Remote Sens., vol. 13, no. 20, p. 4100, 2021.
- C. Aybar, L. Ysuhuaylas, J. Loja, K. Gonzales, F. Herrera, L. Bautista, R. Yali, A. Flores, L. Diaz, N. Cuenca et al., “Cloudsen12, a global dataset for semantic understanding of cloud and cloud shadow in sentinel-2,” Scientific data, vol. 9, no. 1, p. 782, 2022.
- X. Du and H. Wu, “Gated aggregation network for cloud detection in remote sensing image,” The Vis. Comput., pp. 1–20, 2023.
- S. Mohajerani and P. Saeedi, “Cloud-Net+: A Cloud Segmentation CNN for Landsat 8 Remote Sensing Imagery Optimized with Filtered Jaccard Loss Function,” vol. 2001.08768, 2020.
- J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang, W. Liu, and B. Xiao, “Deep high-resolution representation learning for visual recognition,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, 2020.
- A. Shrivastava, A. Gupta, and R. Girshick, “Training region-based object detectors with online hard example mining,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 761–769.
- L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 801–818.
- O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer, 2015, pp. 234–241.
- Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation,” IEEE Trans Med Imaging., vol. 39, no. 6, pp. 1856–1867, 2019.
- N. He, L. Fang, and A. Plaza, “Hybrid first and second order attention unet for building segmentation in remote sensing images,” Science China Information Sciences, vol. 63, pp. 1–12, 2020.
- T. Takikawa, D. Acuna, V. Jampani, and S. Fidler, “Gated-scnn: Gated shape cnns for semantic segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2019, pp. 5229–5238.