SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion (1906.07789v1)

Abstract: The availability of curated large-scale training data is a crucial factor for the development of well-generalizing deep learning methods for the extraction of geoinformation from multi-sensor remote sensing imagery. While quite some datasets have already been published by the community, most of them suffer from rather strong limitations, e.g. regarding spatial coverage, diversity or simply number of available samples. Exploiting the freely available data acquired by the Sentinel satellites of the Copernicus program implemented by the European Space Agency, as well as the cloud computing facilities of Google Earth Engine, we provide a dataset consisting of 180,662 triplets of dual-pol synthetic aperture radar (SAR) image patches, multi-spectral Sentinel-2 image patches, and MODIS land cover maps. With all patches being fully georeferenced at a 10 m ground sampling distance and covering all inhabited continents during all meteorological seasons, we expect the dataset to support the community in developing sophisticated deep learning-based approaches for common tasks such as scene classification or semantic segmentation for land cover mapping.

• The paper introduces SEN12MS, a comprehensive multi-sensor remote sensing dataset designed to fill gaps in annotated data for deep learning.
• It details the use of Google Earth Engine and advanced workflows to curate cloud-free Sentinel-2 images with rigorous quality control.
• The dataset advances applications in scene classification and semantic segmentation with global coverage and seasonal subsets.

An Analysis of the SEN12MS Dataset for Advanced Remote Sensing Applications

The paper “SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion” by Schmitt and colleagues is a comprehensive effort to address significant gaps in the availability of annotated datasets for remote sensing applications. The authors introduce SEN12MS, a large-scale dataset providing a robust platform for developing deep learning models aimed at extracting geoinformation from multi-sensor remote sensing data.

Contributions and Dataset Composition

SEN12MS is constructed leveraging data from the Sentinel-1 and Sentinel-2 satellites of the European Space Agency's Copernicus program, with supplementary land cover information from MODIS. The dataset consists of 180,662 triplets of data patches: dual-polarized SAR images from Sentinel-1, multi-spectral images from Sentinel-2, and MODIS land cover maps. Each patch is meticulously georeferenced and measures 256 x 256 pixels at a 10-meter ground sampling distance, broadening spatial coverage to encompass all inhabited continents across diverse meteorological seasons.

This dataset is an enhancement over previous datasets which have limitations such as restricted spatial coverage or insufficient sample sizes. Notably, SEN12MS offers comprehensive spectral information and global coverage, distinguishing it from other remote sensing datasets. Its detailed heterogeneity is expected to improve the robustness and generalization capabilities of deep learning models applied to remote sensing tasks.

Technical Considerations

The authors employed advanced methodologies to curate the dataset, utilizing Google Earth Engine (GEE) for data preparation. Their sophisticated mosaicking workflow ensures cloud-free Sentinel-2 images, addressing a common issue in optical remote sensing imagery. Moreover, rigorous manual quality control was conducted to filter out flawed data, resulting in a reliable collection of patches.

The structure of the dataset into seasonal subsets based on meteorological definitions of the northern hemisphere demonstrates the dataset’s alignment with climatic variations. However, this also implies that researchers may need to re-organize data subsets for semantic modeling based on other climatic regions.

The dataset is publicly available under a CC-BY license, facilitating open-access research and encouraging collaborative advancements in the field.

Implications and Applications

SEN12MS is promising for the remote sensing community, particularly for tasks such as scene classification and semantic segmentation. The diverse nature of the dataset supports the potential development of predictive models with enhanced accuracy and generalization. The paper presents baseline experiments using ResNet-110 and fully convolutional DenseNet models for land cover classification, demonstrating significant improvements over existing MODIS-derived products in high-resolution applications for urban areas like Munich and Rome.

Beyond immediate practical applications, SEN12MS serves as a fertile ground for future research into data fusion, trans-domain learning, and the development of robust, globally-offer models. Researchers may explore innovative methods to integrate this dataset with other remote sensing datasets or augment it through transfer learning techniques.

Conclusion

The SEN12MS dataset is a critical resource for advancing remote sensing analysis using deep learning. Its extensive coverage and detailed content provide the necessary foundation to overcome current limitations in land cover mapping and other geospatial tasks. The dataset not only enables enhanced modeling possibilities but also sets a benchmark for future data fusion research and remote sensing applications. As the field progresses, the SEN12MS will likely become integral to the development of next-generation AI solutions in geoinformation sciences.