Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TransLandSeg: A Transfer Learning Approach for Landslide Semantic Segmentation Based on Vision Foundation Model (2403.10127v1)

Published 15 Mar 2024 in cs.CV

Abstract: Landslides are one of the most destructive natural disasters in the world, posing a serious threat to human life and safety. The development of foundation models has provided a new research paradigm for large-scale landslide detection. The Segment Anything Model (SAM) has garnered widespread attention in the field of image segmentation. However, our experiment found that SAM performed poorly in the task of landslide segmentation. We propose TransLandSeg, which is a transfer learning approach for landslide semantic segmentation based on a vision foundation model (VFM). TransLandSeg outperforms traditional semantic segmentation models on both the Landslide4Sense dataset and the Bijie landslide dataset. Our proposed adaptive transfer learning (ATL) architecture enables the powerful segmentation capability of SAM to be transferred to landslide detection by training only 1.3% of the number of the parameters of SAM, which greatly improves the training efficiency of the model. Finally we also conducted ablation experiments on models with different ATL structures, concluded that the deployment location and residual connection of ATL play an important role in TransLandSeg accuracy improvement.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Z. Lu, Y. Peng, W. Li, J. Yu, D. Ge, and W. Xiang, “An Iterative Classification and Semantic Segmentation Network for Old Landslide Detection Using High-Resolution Remote Sensing Images.” arXiv, Apr. 24, 2023.
  2. S. Plank, A. Twele, and S. Martinis, “Landslide Mapping in Vegetated Areas Using Change Detection Based on Optical and Polarimetric SAR Data,” Remote Sens., vol. 8, no. 4, p. 307, Apr. 2016, doi: 10.3390/rs8040307.
  3. H. Zhang, “Remote Sensing Data Processing of Urban Land Using Based on Artificial Neural Network,” Sci. Program., vol. 2022, pp. 1–8, Mar. 2022, doi: 10.1155/2022/6994179.
  4. R. N. Keyport, T. Oommen, T. R. Martha, K. S. Sajinkumar, and J. S. Gierke, “A comparative analysis of pixel- and object-based detection of landslides from very high-resolution images,” Int. J. Appl. Earth Obs. Geoinformation, vol. 64, pp. 1–11, Feb. 2018, doi: 10.1016/j.jag.2017.08.015.
  5. A. Mondini, M. Santangelo, M. Rocchetti, E. Rossetto, A. Manconi, and O. Monserrat, “Sentinel-1 SAR Amplitude Imagery for Rapid Landslide Detection,” Remote Sens., vol. 11, no. 7, p. 760, Mar. 2019, doi: 10.3390/rs11070760.
  6. X. Tang, Z. Tu, Y. Wang, M. Liu, D. Li, and X. Fan, “Automatic Detection of Coseismic Landslides Using a New Transformer Method,” Remote Sens., vol. 14, no. 12, p. 2884, Jun. 2022, doi: 10.3390/rs14122884.
  7. T. Ren, W. Gong, L. Gao, F. Zhao, and Z. Cheng, “An Interpretation Approach of Ascending–Descending SAR Data for Landslide Identification,” Remote Sens., vol. 14, no. 5, p. 1299, Mar. 2022, doi: 10.3390/rs14051299.
  8. F. Chen, B. Yu, and B. Li, “A practical trial of landslide detection from single-temporal Landsat8 images using contour-based proposals and random forest: a case study of national Nepal,” Landslides, vol. 15, no. 3, pp. 453–464, Mar. 2018, doi: 10.1007/s10346-017-0884-x.
  9. A. Mohan, A. K. Singh, B. Kumar, and R. Dwivedi, “Review on remote sensing methods for landslide detection using machine and deep learning,” Trans. Emerg. Telecommun. Technol., vol. 32, no. 7, p. e3998, Jul. 2021, doi: 10.1002/ett.3998.
  10. W. Zhao, A. Li, X. Nan, Z. Zhang, and G. Lei, “Postearthquake Landslides Mapping From Landsat-8 Data for the 2015 Nepal Earthquake Using a Pixel-Based Change Detection Method,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 10, no. 5, pp. 1758–1768, May 2017, doi: 10.1109/JSTARS.2017.2661802.
  11. Z. Li, W. Shi, P. Lu, L. Yan, Q. Wang, and Z. Miao, “Landslide mapping from aerial photographs using change detection-based Markov random field,” Remote Sens. Environ., vol. 187, pp. 76–90, Dec. 2016, doi: 10.1016/j.rse.2016.10.008.
  12. T. Chen, J. Trinder, and R. Niu, “Object-Oriented Landslide Mapping Using ZY-3 Satellite Imagery, Random Forest and Mathematical Morphology, for the Three-Gorges Reservoir, China,” Remote Sens., vol. 9, no. 4, p. 333, Mar. 2017, doi: 10.3390/rs9040333.
  13. T.-A. Bui, P.-J. Lee, K.-Y. Lum, C. Loh, and K. Tan, “Deep Learning for Landslide Recognition in Satellite Architecture,” IEEE Access, vol. 8, pp. 143665–143678, 2020, doi: 10.1109/ACCESS.2020.3014305.
  14. E. Shelhamer, J. Long, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 640–651, Apr. 2017, doi: 10.1109/TPAMI.2016.2572683.
  15. L. P. Soares, H. C. Dias, and C. H. Grohmann, “Landslide Segmentation with U-Net: Evaluating Different Sampling Methods and Patch Sizes.” arXiv, Jul. 13, 2020.
  16. X. Zheng, L. Han, G. He, N. Wang, G. Wang, and L. Feng, “Semantic Segmentation Model for Wide-Area Coseismic Landslide Extraction Based on Embedded Multichannel Spectral–Topographic Feature Fusion: A Case Study of the Jiuzhaigou Ms7.0 Earthquake in Sichuan, China,” Remote Sens., vol. 15, no. 4, p. 1084, Feb. 2023, doi: 10.3390/rs15041084.
  17. B. Li and J. Li, “Methods for landslide detection based on lightweight YOLOv4 convolutional neural network,” Earth Sci. Inform., vol. 15, no. 2, pp. 765–775, Jun. 2022, doi: 10.1007/s12145-022-00764-0.
  18. H. Yu, Y. Ma, L. Wang, Y. Zhai, and X. Wang, “A landslide intelligent detection method based on CNN and RSG_R,” in 2017 IEEE International Conference on Mechatronics and Automation (ICMA), Takamatsu, Japan: IEEE, Aug. 2017, pp. 40–44. doi: 10.1109/ICMA.2017.8015785.
  19. B. Fang, G. Chen, L. Pan, R. Kou, and L. Wang, “GAN-Based Siamese Framework for Landslide Inventory Mapping Using Bi-Temporal Optical Remote Sensing Images,” IEEE Geosci. Remote Sens. Lett., vol. 18, no. 3, pp. 391–395, Mar. 2021, doi: 10.1109/LGRS.2020.2979693.
  20. S. Ji, D. Yu, C. Shen, W. Li, and Q. Xu, “Landslide detection from an open satellite imagery and digital elevation model dataset using attention boosted convolutional neural networks,” Landslides, vol. 17, no. 6, pp. 1337–1352, Jun. 2020, doi: 10.1007/s10346-020-01353-2.
  21. H. Cai, T. Chen, R. Niu, and A. Plaza, “Landslide Detection Using Densely Connected Convolutional Networks and Environmental Conditions,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 14, pp. 5235–5247, 2021, doi: 10.1109/JSTARS.2021.3079196.
  22. Y. Liang, Y. Zhang, Y. Li, and J. Xiong, “Automatic Identification for the Boundaries of InSAR Anomalous Deformation Areas Based on Semantic Segmentation Model,” Remote Sens., vol. 15, no. 21, p. 5262, Nov. 2023, doi: 10.3390/rs15215262.
  23. P. Lv, L. Ma, Q. Li, and F. Du, “ShapeFormer: A Shape-Enhanced Vision Transformer Model for Optical Remote Sensing Image Landslide Detection,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 16, pp. 2681–2689, 2023, doi: 10.1109/JSTARS.2023.3253769.
  24. Z. Yang, C. Xu, and L. Li, “Landslide Detection Based on ResU-Net with Transformer and CBAM Embedded: Two Examples with Geologically Different Environments,” Remote Sens., vol. 14, no. 12, p. 2885, Jun. 2022, doi: 10.3390/rs14122885.
  25. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,arXiv - CS - Machine Learning .”
  26. “How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers,arXiv - CS - Computer Vision and Pattern Recognition.”
  27. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” arXiv:1810.04805, May 2019.
  28. K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast.” arXiv, Nov. 03, 2022.
  29. Z. Qiu, Y. Hu, H. Li, and J. Liu, “Learnable Ophthalmology SAM.” arXiv, Apr. 26, 2023.
  30. K. Zhang and D. Liu, “Customized Segment Anything Model for Medical Image Segmentation.” arXiv, Oct. 17, 2023.
  31. E. B. Zaken, S. Ravfogel, and Y. Goldberg, “BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models.” arXiv, Sep. 05, 2022.
  32. X. He, C. Li, P. Zhang, J. Yang, and X. E. Wang, “Parameter-efficient Model Adaptation for Vision Transformers.” arXiv, Jul. 13, 2023.
  33. O. Ghorbanzadeh, H. Shahabi, A. Crivellari, S. Homayouni, T. Blaschke, and P. Ghamisi, “Landslide detection using deep learning and object-based image analysis,” Landslides, vol. 19, no. 4, pp. 929–939, Apr. 2022, doi: 10.1007/s10346-021-01843-x.
  34. H. He, C. Li, R. Yang, H. Zeng, L. Li, and Y. Zhu, “Multisource Data Fusion and Adversarial Nets for Landslide Extraction from UAV-Photogrammetry-Derived Data,” Remote Sens., vol. 14, no. 13, p. 3059, Jun. 2022, doi: 10.3390/rs14133059.
  35. Z. Peng, Z. Xu, Z. Zeng, X. Yang, and W. Shen, “SAM-PARSER: Fine-tuning SAM Efficiently by Parameter Space Reconstruction.” arXiv, Dec. 18, 2023.
  36. Y. Li, M. Hu, and X. Yang, “Polyp-SAM: Transfer SAM for Polyp Segmentation”.
  37. “Visual Prompting: Modifying Pixel Space to Adapt Pre-trained Models,arXiv - CS - Computer Vision and Pattern Recognition .”
  38. “Visual Prompt Tuning,arXiv - CS - Computer Vision and Pattern Recognition.”

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now