ELGC-Net: Efficient Local-Global Context Aggregation for Remote Sensing Change Detection (2403.17909v1)
Abstract: Deep learning has shown remarkable success in remote sensing change detection (CD), aiming to identify semantic change regions between co-registered satellite image pairs acquired at distinct time stamps. However, existing convolutional neural network and transformer-based frameworks often struggle to accurately segment semantic change regions. Moreover, transformers-based methods with standard self-attention suffer from quadratic computational complexity with respect to the image resolution, making them less practical for CD tasks with limited training data. To address these issues, we propose an efficient change detection framework, ELGC-Net, which leverages rich contextual information to precisely estimate change regions while reducing the model size. Our ELGC-Net comprises a Siamese encoder, fusion modules, and a decoder. The focus of our design is the introduction of an Efficient Local-Global Context Aggregator module within the encoder, capturing enhanced global context and local spatial information through a novel pooled-transpose (PT) attention and depthwise convolution, respectively. The PT attention employs pooling operations for robust feature extraction and minimizes computational cost with transposed attention. Extensive experiments on three challenging CD datasets demonstrate that ELGC-Net outperforms existing methods. Compared to the recent transformer-based CD approach (ChangeFormer), ELGC-Net achieves a 1.4% gain in intersection over union metric on the LEVIR-CD dataset, while significantly reducing trainable parameters. Our proposed ELGC-Net sets a new state-of-the-art performance in remote sensing change detection benchmarks. Finally, we also introduce ELGC-Net-LW, a lighter variant with significantly reduced computational complexity, suitable for resource-constrained settings, while achieving comparable performance. Project url https://github.com/techmn/elgcnet.
- J. Yin, J. Dong, N. A. Hamm, Z. Li, J. Wang, H. Xing, and P. Fu, “Integrating remote sensing and geospatial big data for urban land use mapping: A review,” International Journal of Applied Earth Observation and Geoinformation, vol. 103, p. 102514, 2021.
- M. Kucharczyk and C. H. Hugenholtz, “Remote sensing of natural hazard-related disasters with small drones: Global trends, biases, and research opportunities,” Remote Sensing of Environment, vol. 264, p. 112577, 2021.
- Q. Li, R. Zhong, X. Du, and Y. Du, “Transunetcd: A hybrid transformer network for change detection in optical remote-sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–19, 2022.
- P. Rosin, “Thresholding for change detection,” in Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271). IEEE, 1998, pp. 274–279.
- P. L. Rosin and E. Ioannidis, “Evaluation of global image thresholding for change detection,” Pattern recognition letters, vol. 24, no. 14, pp. 2345–2356, 2003.
- J. Im and J. R. Jensen, “A change detection model based on neighborhood correlation image analysis and decision tree classification,” Remote Sensing of Environment, vol. 99, no. 3, pp. 326–340, 2005.
- F. Bovolo, S. Marchesi, and L. Bruzzone, “A framework for automatic and unsupervised detection of multiple changes in multitemporal images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 6, pp. 2196–2212, 2011.
- M. Volpi, D. Tuia, F. Bovolo, M. Kanevski, and L. Bruzzone, “Supervised change detection in vhr images using contextual information and support vector machines,” International Journal of Applied Earth Observation and Geoinformation, vol. 20, pp. 77–85, 2013.
- B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and C. Zoppetti, “Nonparametric change detection in multitemporal sar images based on mean-shift clustering,” IEEE transactions on geoscience and remote sensing, vol. 51, no. 4, pp. 2022–2031, 2013.
- R. Shang, L. Qi, L. Jiao, R. Stolkin, and Y. Li, “Change detection in sar images by artificial immune multi-objective clustering,” Engineering Applications of Artificial Intelligence, vol. 31, pp. 53–67, 2014.
- R. C. Daudt, B. Le Saux, and A. Boulch, “Fully convolutional siamese networks for change detection,” in ICIP. IEEE, 2018, pp. 4063–4067.
- M. Zhang, G. Xu, K. Chen, M. Yan, and X. Sun, “Triplet-based semantic relation learning for aerial remote sensing image change detection,” IEEE Geoscience and Remote Sensing Letters, vol. 16, no. 2, pp. 266–270, 2018.
- J. Chen, Z. Yuan, J. Peng, L. Chen, H. Huang, J. Zhu, Y. Liu, and H. Li, “Dasnet: Dual attentive fully convolutional siamese networks for change detection in high-resolution satellite images,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 1194–1206, 2020.
- C. Zhang, P. Yue, D. Tapete, L. Jiang, B. Shangguan, L. Huang, and G. Liu, “A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 166, pp. 183–200, 2020.
- H. Chen and Z. Shi, “A spatial-temporal attention-based method and a new dataset for remote sensing image change detection,” Remote Sensing, vol. 12, no. 10, p. 1662, 2020.
- W. G. C. Bandara and V. M. Patel, “A transformer-based siamese network for change detection,” arXiv preprint arXiv:2201.01293, 2022.
- T. Yan, Z. Wan, and P. Zhang, “Fully transformer network for change detection of remote sensing images,” arXiv preprint arXiv:2210.00757, 2022.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” NeurIPS, vol. 30, 2017.
- S. Fang, K. Li, J. Shao, and Z. Li, “Snunet-cd: A densely connected siamese network for change detection of vhr images,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2022.
- H. Chen, Z. Qi, and Z. Shi, “Remote sensing image change detection with transformers,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–14, 2021.
- Y. Liu, C. Pang, Z. Zhan, X. Zhang, and X. Yang, “Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model,” IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 5, pp. 811–815, 2020.
- Q. Ke and P. Zhang, “Hybrid-transcd: A hybrid transformer remote sensing image change detection network via token aggregation,” ISPRS International Journal of Geo-Information, vol. 11, no. 4, p. 263, 2022.
- C. P, J. I, N. K, M. B, and L. E, “Digital change detection methods in ecosystem monitoring: a review,” International Journal of Remote Sensing, vol. 25, no. 9, p. 1565, 2004.
- A. Singh, “Review article digital change detection techniques using remotely-sensed data,” International journal of remote sensing, vol. 10, no. 6, pp. 989–1003, 1989.
- R. J. Radke, S. Andra, O. Al-Kofahi, and B. Roysam, “Image change detection algorithms: a systematic survey,” IEEE transactions on image processing, vol. 14, no. 3, pp. 294–307, 2005.
- Y. Bazi, L. Bruzzone, and F. Melgani, “An unsupervised approach based on the generalized gaussian model to automatic change detection in multitemporal sar images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 4, pp. 874–887, 2005.
- M. Zanetti, F. Bovolo, and L. Bruzzone, “Rayleigh-rice mixture parameter estimation via em algorithm for change detection in multispectral images,” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5004–5016, 2015.
- J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in CVPR, 2015, pp. 3431–3440.
- M. Fiaz, A. Mahmood, and S. K. Jung, “Video object segmentation using guided feature and directional deep appearance learning,” in DAVIS Challenge on Video Object Segmentation—CVPR, Workshops, vol. 19, 2020.
- M. Fiaz, A. Mahmood, K. Y. Baek, S. S. Farooq, and S. K. Jung, “Improving object tracking by added noise and channel attention,” Sensors, vol. 20, no. 13, p. 3780, 2020.
- C. Feichtenhofer, A. Pinz, and R. P. Wildes, “Spatiotemporal multiplier networks for video action recognition,” in CVPR, 2017, pp. 4768–4777.
- W. Shi, M. Zhang, R. Zhang, S. Chen, and Z. Zhan, “Change detection based on artificial intelligence: State-of-the-art and challenges,” Remote Sensing, vol. 12, no. 10, p. 1688, 2020.
- K. Zhao, J. Kang, J. Jung, and G. Sohn, “Building extraction from satellite images using mask r-cnn with building boundary regularization.” in CVPR Workshops, 2018, pp. 247–251.
- Y. Liu, J. Liu, H. Pu, Y. Liu, S. Song et al., “Instance segmentation of outdoor sports ground from high spatial resolution remote sensing imagery using the improved mask r-cnn,” International Journal of Geosciences, vol. 10, no. 10, p. 884, 2019.
- S. Ham, Y. Oh, K. Choi, and I. Lee, “Semantic segmentation and unregistered building detection from uav images using a deconvolutional network,” International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 42, no. 2, 2018.
- S. Khan, X. He, F. Porikli, M. Bennamoun, F. Sohel, and R. Togneri, “Learning deep structured network for weakly supervised change detection,” in IJCAI, 2017, pp. 2008–2015.
- P. F. Alcantarilla, S. Stent, G. Ros, R. Arroyo, and R. Gherardi, “Street-view change detection with deconvolutional networks,” Autonomous Robots, vol. 42, no. 7, pp. 1301–1322, 2018.
- M. Papadomanolaki, S. Verma, M. Vakalopoulou, S. Gupta, and K. Karantzalos, “Detecting urban changes with recurrent neural networks from multitemporal sentinel-2 data,” in IGARSS. IEEE, 2019, pp. 214–217.
- E. Guo, X. Fu, J. Zhu, M. Deng, Y. Liu, Q. Zhu, and H. Li, “Learning to measure change: Fully convolutional siamese metric networks for scene change detection,” arXiv preprint arXiv:1810.09111, 2018.
- M. Noman, M. Fiaz, H. Cholakkal, S. Narayan, R. M. Anwar, S. Khan, and F. S. Khan, “Remote sensing change detection with transformers trained from scratch,” arXiv preprint arXiv:2304.06710, 2022.
- A. A. Aleissaee, A. Kumar, R. M. Anwer, S. Khan, H. Cholakkal, G.-S. Xia, and F. S. Khan, “Transformers in remote sensing: A survey,” Remote Sensing, vol. 15, no. 7, 2023.
- H. Jiang, X. Hu, K. Li, J. Zhang, J. Gong, and M. Zhang, “Pga-siamnet: Pyramid feature-based attention-guided siamese network for remote sensing orthoimagery building change detection,” Remote Sensing, vol. 12, no. 3, p. 484, 2020.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, 2016, pp. 770–778.
- F. Song, S. Zhang, T. Lei, Y. Song, and Z. Peng, “Mstdsnet-cd: Multiscale swin transformer and deeply supervised network for change detection of the fast-growing urban regions,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2022.
- Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in ICCV, 2021.
- C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in CVPR, 2015, pp. 1–9.
- X. Chu, Z. Tian, B. Zhang, X. Wang, X. Wei, H. Xia, and C. Shen, “Conditional positional encodings for vision transformers,” arXiv preprint arXiv:2102.10882, 2021.
- T. Xiao, M. Singh, E. Mintun, T. Darrell, P. Dollár, and R. Girshick, “Early convolutions help transformers see better,” NeurIPS, vol. 34, pp. 30 392–30 400, 2021.
- Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y. Li, “Maxvit: Multi-axis vision transformer,” in ECCV, 2022, pp. 459–479.
- M. Fiaz, H. Cholakkal, R. M. Anwer, and F. S. Khan, “Sat: Scale-augmented transformer for person search,” in WACV, 2023, pp. 4820–4829.
- M. Lebedev, Y. V. Vizilter, O. Vygolov, V. A. Knyaz, and A. Y. Rubis, “Change detection in remote sensing images using conditional adversarial networks,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018.
- S. Mehta and M. Rastegari, “Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer,” in ICLR, 2022.
- Y. Li, G. Yuan, Y. Wen, J. Hu, G. Evangelidis, S. Tulyakov, Y. Wang, and J. Ren, “Efficientformer: Vision transformers at mobilenet speed,” arXiv preprint arXiv:2206.01191, 2022.
- Mubashir Noman (12 papers)
- Mustansar Fiaz (21 papers)
- Hisham Cholakkal (78 papers)
- Salman Khan (244 papers)
- Fahad Shahbaz Khan (225 papers)