A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community (1709.00308v2)

Abstract: In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.

A Comprehensive Survey of Deep Learning in Remote Sensing

The paper "A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools, and Challenges for the Community" offers an exhaustive review of the applications and challenges of implementing deep learning (DL) techniques within the domain of remote sensing (RS). Deep learning, which has roots in neural networks, has demonstrated significant successes in fields such as computer vision (CV) and NLP. However, the application of DL to RS presents unique challenges and opportunities, given the specialized nature of the data involved.

Key Contributions

The paper meticulously details the state-of-the-art research in RS employing DL, categorizing the advancements into several critical areas:

1. Comprehensive Literature Survey: The paper reviews 207 application papers and 57 survey papers in RS, extending previous surveys to provide a broader overview of the field.
2. Unique Challenges: It highlights unsolved challenges specific to RS, such as the scarcity of labeled data, complexity in modeling, and the various types of data like spectral, spatial, and temporal data.
3. DL in RS Applications: The survey organizes DL applications in RS into areas such as classification, change detection, data fusion, and image enhancement. Each area is further enriched by detailed discussions of research papers that have contributed significantly to these domains.
4. Educational Resources: The paper provides a high-level overview of DL for researchers engaging with DL in RS for the first time and lists popular DL tools with their pros and cons.
5. Datasets and Tools: It discusses publicly available RS datasets and lists well-known DL tools, offering insights into their suitability for various tasks within RS.

Strong Numerical Results and Bold Claims

The paper addresses the saturation of results on common datasets such as Indian Pines and Pavia, noting that many methods achieve over 99% accuracy—indicating the need for new challenging datasets. It also discusses how RS DL systems must address the often-poor transferability of models across different datasets due to the varied nature of RS data.

Theoretical and Practical Implications

On the theoretical side, the paper calls for enhanced understanding of DL systems tailor-made for RS, pointing out the need for architectures and learning techniques that can handle RS-specific data complexities. Practically, the research suggests adopting DL techniques such as transfer learning, data augmentation, and fusion of heterogeneous data sources to advance RS applications.

Future Directions in AI

Looking forward, the survey suggests further exploration in areas such as semantic understanding of DL models, training with limited data, and optimal architectures for RS data types. It emphasizes fostering interdisciplinary collaboration to bridge the gap between DL innovations in CV and their adaptation to RS challenges.

Conclusion

This comprehensive survey establishes a foundation for future research in the application of deep learning within remote sensing. By identifying existing barriers and proposing directions for future inquiry, this work is a valuable resource for researchers aiming to harness the full potential of DL in the ever-evolving RS landscape.