Land Cover Classification via Multi-temporal Spatial Data by Recurrent Neural Networks

Abstract: Nowadays, modern earth observation programs produce huge volumes of satellite images time series (SITS) that can be useful to monitor geographical areas through time. How to efficiently analyze such kind of information is still an open question in the remote sensing field. Recently, deep learning methods proved suitable to deal with remote sensing data mainly for scene classification (i.e. Convolutional Neural Networks - CNNs - on single images) while only very few studies exist involving temporal deep learning approaches (i.e Recurrent Neural Networks - RNNs) to deal with remote sensing time series. In this letter we evaluate the ability of Recurrent Neural Networks, in particular the Long-Short Term Memory (LSTM) model, to perform land cover classification considering multi-temporal spatial data derived from a time series of satellite images. We carried out experiments on two different datasets considering both pixel-based and object-based classification. The obtained results show that Recurrent Neural Networks are competitive compared to state-of-the-art classifiers, and may outperform classical approaches in presence of low represented and/or highly mixed classes. We also show that using the alternative feature representation generated by LSTM can improve the performances of standard classifiers.

• The paper demonstrates that LSTM networks effectively capture temporal dependencies in satellite time series to enhance land cover classification accuracy.
• It shows that LSTM outperforms conventional classifiers like Random Forest and SVM, especially for complex and underrepresented classes.
• The study’s experiments on the THAU and REUNION ISLAND datasets highlight LSTM's dual role as a classifier and a feature transformer in remote sensing.

Land Cover Classification via Multi-temporal Spatial Data with Recurrent Neural Networks

This paper presents an investigation into the application of Recurrent Neural Networks (RNN), specifically Long Short-Term Memory (LSTM) models, for land cover classification using multi-temporal spatial data derived from time series of satellite images. The authors explore the feasibility and effectiveness of RNNs in this domain, contributing to the ongoing development of methodologies for processing remote sensing data.

The research is centered on utilizing the temporal nature intrinsic to satellite image time series (SITS) for land cover classification, diverging from conventional approaches that often neglect temporal dependencies. Remote sensing data contain rich temporal information that traditional classifiers such as Random Forest (RF) and Support Vector Machine (SVM) do not fully exploit because they treat each image independently without modeling temporal correlations. Incorporating LSTM networks, a subset of RNNs designed to manage sequences and temporal dependencies, allows the proposed model to capture these correlations effectively.

The authors conduct experiments on two multi-temporal remote sensing datasets: the THAU dataset from France and the REUNION ISLAND dataset. The THAU dataset consists of object-based classification data derived from very high spatial resolution images, while the REUNION ISLAND dataset offers pixel-based classification data sourced from an extensive Landsat 8 image time series. The choice of these datasets provides a robust evaluation environment for the proposed methodology across different spatial resolutions and thematic contexts.

For the THAU dataset, numerical results indicate that the LSTM-based classifier outperforms traditional methods such as RF and SVM in terms of overall accuracy, F-measure, and Kappa statistics. Notably, LSTM shows significant improvement in classifying low-represented and complex land cover classes, demonstrating its strength in capturing crucial temporal signals that may be pivotal for distinguishing such classes.

Similarly, on the REUNION ISLAND dataset, LSTM achieves competitive results, again surpassing RF and SVM in important metrics, and when combined with SVM using the feature representation produced by LSTM, it garners the best performance in F-measure. This highlights the versatility and potency of LSTM not only as a standalone classifier but also as a feature transformer in multi-temporal data scenarios.

The study substantiates the practicality of LSTM networks in addressing the limitations of conventional classifiers for land cover classification through improved representation learning that respects temporal dependencies. Unlike classical models that overlook these dependencies, LSTM-based approaches enhance class discrimination, especially for classes affected by temporal patterns. This capability is substantial given the complex and seasonally dynamic nature of land covers, particularly in classes driven by agricultural practices.

In conclusion, the deployment of LSTM architectures for land cover classification represents a meaningful progression in the use of deep learning for remote sensing applications. These models, by inherently capturing temporal correlations, render themselves suitable for complex spatiotemporal datasets offered by contemporary earth observation programs. Future work may include exploring more intricate LSTM architectures, integrating them with other deep learning paradigms, and extending these methods to additional datasets and applications to further evaluate their potential in remote sensing analytics.

The findings of this study hold significant implications for both practical applications and theoretical advancements in remote sensing and machine learning, promoting a more nuanced understanding of temporal dynamics in land cover classification tasks. This could steer future research towards more sophisticated techniques for exploiting the vast multitemporal data produced by satellite imaging systems.