Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data (2103.16607v2)

Abstract: Remote sensing and automatic earth monitoring are key to solve global-scale challenges such as disaster prevention, land use monitoring, or tackling climate change. Although there exist vast amounts of remote sensing data, most of it remains unlabeled and thus inaccessible for supervised learning algorithms. Transfer learning approaches can reduce the data requirements of deep learning algorithms. However, most of these methods are pre-trained on ImageNet and their generalization to remote sensing imagery is not guaranteed due to the domain gap. In this work, we propose Seasonal Contrast (SeCo), an effective pipeline to leverage unlabeled data for in-domain pre-training of remote sensing representations. The SeCo pipeline is composed of two parts. First, a principled procedure to gather large-scale, unlabeled and uncurated remote sensing datasets containing images from multiple Earth locations at different timestamps. Second, a self-supervised algorithm that takes advantage of time and position invariance to learn transferable representations for remote sensing applications. We empirically show that models trained with SeCo achieve better performance than their ImageNet pre-trained counterparts and state-of-the-art self-supervised learning methods on multiple downstream tasks. The datasets and models in SeCo will be made public to facilitate transfer learning and enable rapid progress in remote sensing applications.

• The paper introduces Seasonal Contrast (SeCo), a novel method that exploits seasonal variations in unlabeled Sentinel-2 images to boost remote sensing performance.
• It employs self-supervised contrastive learning on temporally diverse data, capturing both invariant and variant features to enhance tasks like land-cover classification.
• Experimental results show superior label efficiency and accuracy over traditional methods, highlighting SeCo's potential in automated Earth observation.

Overview of "Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data"

The paper presents a novel approach to leveraging unlabeled remote sensing data for remote sensing applications using an unsupervised learning pipeline called Seasonal Contrast (SeCo). Given the abundance of remote sensing data that remains untapped due to the lack of labels, the research introduces a method that harnesses the temporal aspect of these images for representation learning. The primary objective is to develop a representation for remote sensing imagery that can address a variety of downstream tasks without relying on costly labeled datasets.

Methodological Contributions

The proposed SeCo pipeline consists of two main components:

1. Dataset Construction: The authors propose an automatic process to collect large scale, unlabeled, and uncurated datasets from Sentinel-2 satellite images. This procedure eliminates the need for manual curation and annotation by sampling data that represents diverse geographical locations across seasonal snapshots.
2. Self-Supervised Learning Model: SeCo leverages a self-supervised learning model that uses temporal variations (seasonal changes) to define a contrastive learning task. By deriving positive pairs from different temporal instances of the same location, SeCo generates richer, semantically meaningful representations. The model design employs multiple embedding sub-spaces to capture both invariant and variant factors from these temporal changes, enhancing the applicability of learned features across various remote sensing tasks.

Experimental Results and Observations

SeCo's performance was evaluated on tasks such as land-cover classification and change detection using datasets including BigEarthNet, EuroSAT, and OSCD. Key results indicate:

• Superior Performance: The SeCo pre-training substantially surpasses standard methods like ImageNet pre-training and state-of-the-art self-supervised learning models (e.g., MoCo-v2) across several metrics. For instance, it displayed marked improvements in label efficiency, achieving competitive accuracy in BigEarthNet classification tasks with reduced labeled data.
• Temporal Variance Handling: By introducing multiple embedding sub-spaces, the method shows strong capabilities in balancing temporal invariance and variance, which is crucial for applications like change detection where temporal information is crucial.
• Dataset Effectiveness: The results corroborate the efficacy of the proposed dataset sampling strategy, suggesting that unsupervised methods can effectively harness the vast data reserves of remote sensing imagery.

Implications and Future Directions

The research presents significant implications for the field of geospatial analytics, emphasizing that domain-specific unsupervised pre-training can outperform conventional methods reliant on curated datasets. It underscores the potential to utilize hidden temporal structures within satellite data to generate flexible and generalizable representations.

Future developments could explore the application of SeCo across different spectral bands beyond RGB and extend its applications to more dynamic Earth monitoring challenges, such as real-time disaster response systems and detailed climate analysis. Additionally, integrating further domain-specific self-supervised tasks might enhance the adaptability and richness of learned representations.

Overall, the paper provides a compelling case for the adoption of unsupervised learning methodologies in remote sensing, unlocking new pathways for automated Earth observation and monitoring at scale.