2000 character limit reached
Learning from Unlabelled Data with Transformers: Domain Adaptation for Semantic Segmentation of High Resolution Aerial Images (2404.11299v1)
Published 17 Apr 2024 in cs.CV and cs.LG
Abstract: Data from satellites or aerial vehicles are most of the times unlabelled. Annotating such data accurately is difficult, requires expertise, and is costly in terms of time. Even if Earth Observation (EO) data were correctly labelled, labels might change over time. Learning from unlabelled data within a semi-supervised learning framework for segmentation of aerial images is challenging. In this paper, we develop a new model for semantic segmentation of unlabelled images, the Non-annotated Earth Observation Semantic Segmentation (NEOS) model. NEOS performs domain adaptation as the target domain does not have ground truth semantic segmentation masks. The distribution inconsistencies between the target and source domains are due to differences in acquisition scenes, environment conditions, sensors, and times. Our model aligns the learned representations of the different domains to make them coincide. The evaluation results show that NEOS is successful and outperforms other models for semantic segmentation of unlabelled data.
- “DIAL: Deep interactive and active learning for semantic segmentation in remote sensing,” IEEE J Sel Top Appl Ear Obs Rem Se, v. 15, p. 3376-3389, 2022.
- “Artificial intelligence to advance Earth observation: a perspective,” arXiv preprint arXiv:2305.08413, 2023.
- “Data Fusion Contest 2017 (DFC2017),” IEEE GRSS, 2022. DOI: 10.21227/e56j-eh82.
- “An interpretable deep semantic segmentation method for Earth Observation,” In IEEE Int Conf Intelligent Systems (IS), 2022.
- “Semi-supervised semantic segmentation in earth observation: The MiniFrance suite, dataset analysis and multi-task network,” Machine Learning (111), p. 3125-3160, 2022.
- Scott Workman et al., “Wide-area image geolocalization with aerial reference imagery,” In ICCV, pp. 1–9, 2015.
- “Where am I looking at? Joint location and orientation estimation by cross-view matching,” In CVPR, pp. 4064–4072, 2020.
- “Are these from the same place? Seeing the unseen in cross-view image geo-localization,” In WACV, p. 3753-3761, 2021.
- Jiangtao Peng et al., “Domain adaptation in remote sensing image classification: A survey,” IEEE J Sel Top Appl Earth Obs Remote Sen, v. 15, p. 9842-9859, 2022.
- “Domain-adversarial training of neural networks,” JMLR, 2016.
- Xiaofeng Liu et al., “Deep unsupervised domain adaptation: A review of recent advances and perspectives,” APSIPA Trans. Signal Inf. Process. 11 (1), 2022.
- “Benchmarking domain adaptation methods on aerial datasets,” Sensors, MDPI, 2021.
- Ying Chen et al., “Semantic segmentation in aerial images using class-aware unsupervised domain adaptation,” ACM SIGSPATIAL Int Workshop GEOAI, p. 9-16, 2021.
- “SegFormer: Simple and efficient design for semantic segmentation with Transformers,” In NeurIPS, 34:12077-12090, 2021.
- Libo Wang et al., “UNetFormer: A UNet-like Transformer for efficient semantic segmentation of remote sensing urban scene imagery,” ISPRS Journal Photogrammetry Remote Sensing, v. 190, p. 196-214, 2022.
- “Segment anything,” arXiv:2304.02643, 2023.
- Yabin Zhang et al., “Domain-symmetric networks for adversarial domain adaptation,” In CVPR, 2019.
- Jian Liang et al., “Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation,” In ICML, PMLR 119, 2020.
- “Universal domain adaptation,” In CVPR, p. 2720-2729, 2019.
- Valerio Marsocci et al., “GeoMultiTaskNet: Remote sensing unsupervised domain adaptation using geographical coordinates,” Workshop CVPR, 2023.
- “FixMatch: Simplifying semi-supervised learning with consistency and confidence,” In NeurIPS, v. 33, p. 596-608, 2020.
- “MSMatch: Semi-supervised multispectral scene classification with few labels,” IEEE Journal Selected Topics Applied Earth Observations and Remote Sensing, 14:11643-11654, 2021.
- “AerialFormer: Multi-resolution Transformer for aerial image segmentation,” arXiv:2306.06842, 2023.
- X. He et al., “Swin Transformer embedding U-Net for RS image semantic segmentation,” T Geo RS (60), 2022.
- “samgeo: A Python package for segmenting geospatial data with SAM,” Journal of Open Source Software, 8(89), 5663, 2023.
- A. Hancharenka, “SAMEO: Segment anything EO tools,” GitHub, 2023.
- “Learning from unlabelled data: Domain adaptation for semantic segmentation,” GitHub repository, 2023. http://github.com/ESA-PhiLab/Learning_from_Unlabeled_Data_for_Domain_Adaptation_for_Semantic_Segmentation.
- Marius Cordts et al., “The Cityscapes dataset for semantic urban scene understanding,” In Proc. CVPR, 2016.
- “Robustness of SAM: Segment anything under corruptions and beyond,” arXiv:2306.07713, 2023.
- “Segment anything in high quality,” In NeurIPS, Poster Session 4, 2023.
- “Fast segment anything,” arXiv:2306.12156, 2023.
- “Semantic segment anything,” 2023. http://github.com/fudan-zvg/Semantic-Segment-Anything.
- Feng Li et al., “Semantic-SAM: Segment and recognize anything at any granularity,” arXiv:2307.04767, 2023.
- “A semantic segmentation-guided approach for ground-to-aerial image matching,” IGARSS, 2024.
- Francesco Pro. (2023), “Ground-to-Aerial Image Matching for Geospatial Applications,” [Unpublished Master’s thesis]. Sapienza University of Rome.
- Nikolaos Dionelis (16 papers)
- Francesco Pro (3 papers)
- Luca Maiano (10 papers)
- Irene Amerini (22 papers)
- Bertrand Le Saux (59 papers)