Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adjacent-Level Feature Cross-Fusion With 3-D CNN for Remote Sensing Image Change Detection (2302.05109v2)

Published 10 Feb 2023 in cs.CV

Abstract: Deep learning-based change detection (CD) using remote sensing images has received increasing attention in recent years. However, how to effectively extract and fuse the deep features of bi-temporal images for improving the accuracy of CD is still a challenge. To address that, a novel adjacent-level feature fusion network with 3D convolution (named AFCF3D-Net) is proposed in this article. First, through the inner fusion property of 3D convolution, we design a new feature fusion way that can simultaneously extract and fuse the feature information from bi-temporal images. Then, to alleviate the semantic gap between low-level features and high-level features, we propose an adjacent-level feature cross-fusion (AFCF) module to aggregate complementary feature information between the adjacent levels. Furthermore, the full-scale skip connection strategy is introduced to improve the capability of pixel-wise prediction and the compactness of changed objects in the results. Finally, the proposed AFCF3D-Net has been validated on the three challenging remote sensing CD datasets: the Wuhan building dataset (WHU-CD), the LEVIR building dataset (LEVIR-CD), and the Sun Yat-Sen University dataset (SYSU-CD). The results of quantitative analysis and qualitative comparison demonstrate that the proposed AFCF3D-Net achieves better performance compared to other state-of-the-art methods. The code for this work is available at https://github.com/wm-Githuber/AFCF3D-Net.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. D. Lu, P. Mausel, E. Brondizio, and E. Moran, “Change detection techniques,” International journal of remote sensing, vol. 25, no. 12, pp. 2365–2401, 2004.
  2. Y. Ye, B. Zhu, T. Tang, C. Yang, Q. Xu, and G. Zhang, “A robust multimodal remote sensing image registration method and system using steerable filters with first-and second-order gradients,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 188, pp. 331–350, 2022.
  3. B. Zhu, C. Yang, J. Dai, J. Fan, Y. Qin, and Y. Ye, “R 2 fd 2: Fast and robust matching of multimodal remote sensing images via repeatable feature detector and rotation-invariant feature descriptor,” IEEE Transactions on Geoscience and Remote Sensing, 2023.
  4. A. Shafique, G. Cao, Z. Khan, M. Asad, and M. Aslam, “Deep learning-based change detection in remote sensing images: A review,” Remote Sensing, vol. 14, no. 4, p. 871, 2022.
  5. Y. Byun, Y. Han, and T. Chae, “Image fusion-based change detection for flood extent extraction using bi-temporal very high-resolution satellite images,” Remote Sensing, vol. 7, no. 8, pp. 10 347–10 363, 2015.
  6. Z. Yetgin, “Unsupervised change detection of satellite images using local gradual descent,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 5, pp. 1919–1929, 2011.
  7. K. Ding, C. Huo, Y. Xu, Z. Zhong, and C. Pan, “Sparse hierarchical clustering for vhr image change detection,” IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 3, pp. 577–581, 2014.
  8. Y. Deng, J. Chen, S. Yi, A. Yue, Y. Meng, J. Chen, and Y. Zhang, “Feature guided multitask change detection network,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens, vol. 15, pp. 9667–9679, 2022.
  9. A. Mondini, F. Guzzetti, P. Reichenbach, M. Rossi, M. Cardinali, and F. Ardizzone, “Semi-automatic recognition and mapping of rainfall induced shallow landslides using optical satellite images,” Remote sensing of environment, vol. 115, no. 7, pp. 1743–1757, 2011.
  10. P. J. Howarth and G. M. Wickware, “Procedures for change detection using landsat digital data,” International Journal of Remote Sensing, vol. 2, no. 3, pp. 277–291, 1981.
  11. H. Fang, P. Du, X. Wang, C. Lin, and P. Tang, “Unsupervised change detection based on weighted change vector analysis and improved markov random field for high spatial resolution imagery,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2021.
  12. C. Wu, B. Du, X. Cui, and L. Zhang, “A post-classification change detection method based on iterative slow feature analysis and bayesian soft fusion,” Remote Sensing of Environment, vol. 199, pp. 241–255, 2017.
  13. X. Peng, R. Zhong, Z. Li, and Q. Li, “Optical remote sensing image change detection based on attention mechanism and image difference,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 9, pp. 7296–7307, 2020.
  14. Z. Li, C. Yan, Y. Sun, and Q. Xin, “A densely attentive refinement network for change detection based on very-high-resolution bitemporal remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–18, 2022.
  15. Z. Li, W. Shi, M. Hao, and H. Zhang, “Unsupervised change detection using spectral features and a texture difference measure for vhr remote-sensing images,” International journal of remote sensing, vol. 38, no. 23, pp. 7302–7315, 2017.
  16. J. Im and J. R. Jensen, “A change detection model based on neighborhood correlation image analysis and decision tree classification,” Remote Sensing of Environment, vol. 99, no. 3, pp. 326–340, 2005.
  17. M. Wang, Z. Han, P. Yang, B. Zhu, M. Hao, J. Fan, and Y. Ye, “Exploiting neighbourhood structural features for change detection,” Remote Sensing Letters, vol. 14, no. 4, pp. 346–356, 2023.
  18. L. Li, X. Li, Y. Zhang, L. Wang, and G. Ying, “Change detection for high-resolution remote sensing imagery using object-oriented change vector analysis method,” in 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).   IEEE, 2016, pp. 2873–2876.
  19. H. Liu, M. Yang, J. Chen, J. Hou, and M. Deng, “Line-constrained shape feature for building change detection in vhr remote sensing imagery,” ISPRS International Journal of Geo-Information, vol. 7, no. 10, p. 410, 2018.
  20. F. Pan, Z. Wu, Q. Liu, Y. Xu, and Z. Wei, “Dcff-net: A densely connected feature fusion network for change detection in high-resolution remote sensing images,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 11 974–11 985, 2021.
  21. Q. Li, R. Zhong, X. Du, and Y. Du, “Transunetcd: A hybrid transformer network for change detection in optical remote-sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–19, 2022.
  22. L. Zhou, Y. Ye, T. Tang, K. Nan, and Y. Qin, “Robust matching for sar and optical images using multiscale convolutional gradient features,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2021.
  23. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
  24. D. Peng, Y. Zhang, and H. Guan, “End-to-end change detection for high resolution satellite images using improved unet++,” Remote Sensing, vol. 11, no. 11, p. 1382, 2019.
  25. R. C. Daudt, B. Le Saux, and A. Boulch, “Fully convolutional siamese networks for change detection,” in 2018 25th IEEE International Conference on Image Processing (ICIP).   IEEE, 2018, pp. 4063–4067.
  26. P. Chen, B. Zhang, D. Hong, Z. Chen, X. Yang, and B. Li, “Fccdn: Feature constraint network for vhr image change detection,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 187, pp. 101–119, 2022.
  27. C. Zhang, P. Yue, D. Tapete, L. Jiang, B. Shangguan, L. Huang, and G. Liu, “A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 166, pp. 183–200, 2020.
  28. T. Lei, J. Wang, H. Ning, X. Wang, D. Xue, Q. Wang, and A. K. Nandi, “Difference enhancement and spatial–spectral nonlocal network for change detection in vhr remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–13, 2021.
  29. D. Wang, X. Chen, M. Jiang, S. Du, B. Xu, and J. Wang, “Ads-net: An attention-based deeply supervised network for remote sensing image change detection,” International Journal of Applied Earth Observation and Geoinformation, vol. 101, p. 102348, 2021.
  30. Y. Luo, X. Li, S. Chen, C. Xia, and L. Zhao, “Imnn-lwec: A novel infrared small target detection based on spatial–temporal tensor model,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–22, 2022.
  31. C. Feichtenhofer, “X3d: Expanding architectures for efficient video recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 203–213.
  32. D. Tran, H. Wang, L. Torresani, and M. Feiszli, “Video classification with channel-separated convolutional networks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 5552–5561.
  33. Q. Ding, Z. Shao, X. Huang, and O. Altan, “Dsa-net: A novel deeply supervised attention-guided network for building change detection in high-resolution remote sensing images,” International Journal of Applied Earth Observation and Geoinformation, vol. 105, p. 102591, 2021.
  34. L. S. Khalid, G. Jati, W. Caesarendra, and W. Jatmiko, “Change detection from areal imagery drones using siamese u-net with spatial attention module,” in 7th International Workshop on Big Data and Information Security, IWBIS 2022.   Institute of Electrical and Electronics Engineers Inc., 2022, pp. 51–58.
  35. D. B. Mesquita, R. F. dos Santos, D. G. Macharet, M. F. Campos, and E. R. Nascimento, “Fully convolutional siamese autoencoder for change detection in uav aerial images,” IEEE Geoscience and Remote Sensing Letters, vol. 17, no. 8, pp. 1455–1459, 2019.
  36. Y. Liu, C. Pang, Z. Zhan, X. Zhang, and X. Yang, “Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model,” IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 5, pp. 811–815, 2020.
  37. Q. Shi, M. Liu, S. Li, X. Liu, F. Wang, and L. Zhang, “A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection,” IEEE transactions on geoscience and remote sensing, vol. 60, pp. 1–16, 2021.
  38. S. Fang, K. Li, J. Shao, and Z. Li, “Snunet-cd: A densely connected siamese network for change detection of vhr images,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2021.
  39. Q. Guo, J. Zhang, S. Zhu, C. Zhong, and Y. Zhang, “Deep multiscale siamese network with parallel convolutional structure and self-attention for change detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–12, 2021.
  40. H. Chen, Z. Qi, and Z. Shi, “Remote sensing image change detection with transformers,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–14, 2021.
  41. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  42. X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,” arXiv preprint arXiv:2010.04159, 2020.
  43. Y. Ye, X. Ren, B. Zhu, T. Tang, X. Tan, Y. Gui, and Q. Yao, “An adaptive attention fusion mechanism convolutional network for object detection in remote sensing images,” Remote Sensing, vol. 14, no. 3, p. 516, 2022.
  44. W. Liu, S. Chen, L. Guo, X. Zhu, and J. Liu, “Cptr: Full transformer network for image captioning,” arXiv preprint arXiv:2101.10804, 2021.
  45. S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang, P. H. Torr et al., “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6881–6890.
  46. J. Gu, H. Kwon, D. Wang, W. Ye, M. Li, Y.-H. Chen, L. Lai, V. Chandra, and D. Z. Pan, “Multi-scale high-resolution vision transformer for semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 094–12 103.
  47. J. Lanchantin, T. Wang, V. Ordonez, and Y. Qi, “General multi-label image classification with transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 16 478–16 488.
  48. X. Song, Z. Hua, and J. Li, “Remote sensing image change detection transformer network based on dual-feature mixed attention,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–16, 2022.
  49. Z. Mao, X. Tong, Z. Luo, and H. Zhang, “Mfatnet: Multi-scale feature aggregation via transformer for remote sensing image change detection,” Remote Sensing, vol. 14, no. 21, p. 5379, 2022.
  50. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  51. J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141.
  52. S. Ji, W. Xu, M. Yang, and K. Yu, “3d convolutional neural networks for human action recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 1, pp. 221–231, 2012.
  53. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497.
  54. J. Mäyrä, S. Keski-Saari, S. Kivinen, T. Tanhuanpää, P. Hurskainen, P. Kullberg, L. Poikolainen, A. Viinikka, S. Tuominen, T. Kumpula et al., “Tree species classification from airborne hyperspectral and lidar data using 3d convolutional neural networks,” Remote Sensing of Environment, vol. 256, p. 112322, 2021.
  55. Q. Dou, L. Yu, H. Chen, Y. Jin, X. Yang, J. Qin, and P.-A. Heng, “3d deeply supervised network for automated segmentation of volumetric medical images,” Medical image analysis, vol. 41, pp. 40–54, 2017.
  56. A. Saveliev, M. Uzdiaev, and M. Dmitrii, “Aggressive action recognition using 3d cnn architectures,” in 2019 12th International Conference on Developments in eSystems Engineering (DeSE).   IEEE, 2019, pp. 890–895.
  57. J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6299–6308.
  58. D. Tran, H. Wang, L. Torresani, J. Ray, Y. LeCun, and M. Paluri, “A closer look at spatiotemporal convolutions for action recognition,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 6450–6459.
  59. H. Chen and Z. Shi, “A spatial-temporal attention-based method and a new dataset for remote sensing image change detection,” Remote Sensing, vol. 12, no. 10, p. 1662, 2020.
  60. D. Peng, L. Bruzzone, Y. Zhang, H. Guan, H. Ding, and X. Huang, “Semicdnet: A semisupervised convolutional neural network for change detection in high resolution remote-sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 7, pp. 5891–5906, 2020.
  61. R. Liu, Z. Cheng, L. Zhang, and J. Li, “Remote sensing image change detection based on information transmission and attention mechanism,” IEEE Access, vol. 7, pp. 156 349–156 359, 2019.
  62. X. Xiang, D. Tian, N. Lv, and Q. Yan, “Fcdnet: A change detection network based on full-scale skip connections and coordinate attention,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2022.
  63. J. Chen, Z. Yuan, J. Peng, L. Chen, H. Huang, J. Zhu, Y. Liu, and H. Li, “Dasnet: Dual attentive fully convolutional siamese networks for change detection in high-resolution satellite images,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 1194–1206, 2020.
  64. C. Zhang, G. Li, and S. Du, “Multi-scale dense networks for hyperspectral remote sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 11, pp. 9201–9222, 2019.
  65. H. Huang, L. Lin, R. Tong, H. Hu, Q. Zhang, Y. Iwamoto, X. Han, Y.-W. Chen, and J. Wu, “Unet 3+: A full-scale connected unet for medical image segmentation,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2020, pp. 1055–1059.
  66. Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4.   Springer, 2018, pp. 3–11.
  67. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18.   Springer, 2015, pp. 234–241.
  68. S. Ji, S. Wei, and M. Lu, “Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 1, pp. 574–586, 2018.
  69. Y. Feng, H. Xu, J. Jiang, H. Liu, and J. Zheng, “Icif-net: Intra-scale cross-interaction and inter-scale feature fusion network for bitemporal remote sensing images change detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–13, 2022.
  70. W. G. C. Bandara and V. M. Patel, “A transformer-based siamese network for change detection,” in IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium.   IEEE, 2022, pp. 207–210.
  71. H. Yang, Y. Chen, W. Wu, S. Pu, X. Wu, Q. Wan, and W. Dong, “A lightweight siamese neural network for building change detection using remote sensing images,” Remote Sensing, vol. 15, no. 4, p. 928, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Yuanxin Ye (11 papers)
  2. Mengmeng Wang (73 papers)
  3. Liang Zhou (44 papers)
  4. Guangyang Lei (1 paper)
  5. Jianwei Fan (6 papers)
  6. Yao Qin (41 papers)
Citations (24)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now