TorchGeo: Deep Learning With Geospatial Data

Abstract: Remotely sensed geospatial data are critical for applications including precision agriculture, urban planning, disaster monitoring and response, and climate change research, among others. Deep learning methods are particularly promising for modeling many remote sensing tasks given the success of deep neural networks in similar computer vision tasks and the sheer volume of remotely sensed imagery available. However, the variance in data collection methods and handling of geospatial metadata make the application of deep learning methodology to remotely sensed data nontrivial. For example, satellite imagery often includes additional spectral bands beyond red, green, and blue and must be joined to other geospatial data sources that can have differing coordinate systems, bounds, and resolutions. To help realize the potential of deep learning for remote sensing applications, we introduce TorchGeo, a Python library for integrating geospatial data into the PyTorch deep learning ecosystem. TorchGeo provides data loaders for a variety of benchmark datasets, composable datasets for generic geospatial data sources, samplers for geospatial data, and transforms that work with multispectral imagery. TorchGeo is also the first library to provide pre-trained models for multispectral satellite imagery (e.g., models that use all bands from the Sentinel-2 satellites), allowing for advances in transfer learning on downstream remote sensing tasks with limited labeled data. We use TorchGeo to create reproducible benchmark results on existing datasets and benchmark our proposed method for preprocessing geospatial imagery on the fly. TorchGeo is open source and available on GitHub: https://github.com/microsoft/torchgeo.

• The paper introduces TorchGeo, a Python library that simplifies remote sensing by integrating heterogeneous geospatial data with PyTorch through on-the-fly reprojection and resampling.
• The paper details dataset loaders, geospatial samplers, and multispectral augmentation methods that enable seamless training on benchmark and custom satellite imagery.
• The paper demonstrates competitive performance in classification and segmentation tasks, validating TorchGeo’s efficiency against traditional pre-processed workflows.

TorchGeo: Deep Learning With Geospatial Data

TorchGeo introduces a Python library designed to integrate geospatial data into the PyTorch ecosystem. This integration addresses common challenges faced in applying deep learning to remote sensing tasks, such as handling heterogeneous data sources with differing coordinate systems and resolutions.

Incorporating Geospatial Data in Deep Learning

Geospatial data, including satellite imagery, poses unique challenges due to its varied spectral, spatial, and temporal characteristics. Unlike conventional vision datasets, satellite data often contains multiple spectral bands (e.g., Sentinel-2's 12 bands), which increases the complexity of integration into deep learning models. The challenge further extends to aligning datasets with different coordinate reference systems and resolutions (Figure 1).

Figure 1: An illustration of the challenges in sampling from heterogeneous geospatial data layers.

TorchGeo offers a solution through its ability to perform alignment steps—reprojecting and resampling—during data loading. This process enables users to bypass manual pre-processing and directly train models on pixel-aligned data. Additionally, TorchGeo provides data loaders for benchmark datasets and compositional loaders for arbitrary geospatial data sources, facilitating easy integration into machine learning pipelines.

Design and Implementation of TorchGeo

The core components of TorchGeo include:

• Dataset Loaders: Loaders for common benchmark datasets and for generic geospatial inputs, such as Landsat or Sentinel imagery (Figure 2).
• Geospatial Samplers: Tools to sample data using spatial and temporal coordinates, enabling efficient batch creation for training.
• Pre-Trained Models: Models that utilize multispectral imagery inputs, enhancing transfer learning performance for remote sensing applications.
• Transforms and Augmentation: Methods specifically designed for multispectral imagery augmentation.

  Figure 2: Different layers of geospatial data often have differing coordinate reference systems and spatial resolutions.

The library adheres to PyTorch-like conventions, reducing the learning curve for new users. TorchGeo's flexibility allows it to perform on-the-fly alignment of datasets without pre-processing overhead.

Experimental Results and Benchmarks

TorchGeo's effectiveness is demonstrated through various experiments across datasets such as So2Sat, LandCover.ai, Chesapeake Land Cover, and others. The library's sampling mechanisms are benchmarked, displaying efficient data loading rates comparable to pre-processed datasets.

In classification and semantic segmentation tasks, TorchGeo achieves competitive performance, often matching or surpassing results from specialized methods pre-trained in-domain. For instance, models trained on ImageNet demonstrate significant improvements in generalization performance for tasks like land cover mapping across diverse geographic areas.

Conclusion

TorchGeo significantly eases the integration of geospatial data into deep learning pipelines, promoting reproducible research and streamlined experimentation. By providing tools for data alignment, model training, and augmentation, TorchGeo empowers researchers to leverage the vast quantities of remote sensing data available for AI applications. The library not only accelerates existing workflows but also opens avenues for future research in geospatial machine learning, including self-supervised learning and multimodal data fusion.