HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification (1902.06701v3)

Abstract: Hyperspectral image (HSI) classification is widely used for the analysis of remotely sensed images. Hyperspectral imagery includes varying bands of images. Convolutional Neural Network (CNN) is one of the most frequently used deep learning based methods for visual data processing. The use of CNN for HSI classification is also visible in recent works. These approaches are mostly based on 2D CNN. Whereas, the HSI classification performance is highly dependent on both spatial and spectral information. Very few methods have utilized the 3D CNN because of increased computational complexity. This letter proposes a Hybrid Spectral Convolutional Neural Network (HybridSN) for HSI classification. Basically, the HybridSN is a spectral-spatial 3D-CNN followed by spatial 2D-CNN. The 3D-CNN facilitates the joint spatial-spectral feature representation from a stack of spectral bands. The 2D-CNN on top of the 3D-CNN further learns more abstract level spatial representation. Moreover, the use of hybrid CNNs reduces the complexity of the model compared to 3D-CNN alone. To test the performance of this hybrid approach, very rigorous HSI classification experiments are performed over Indian Pines, Pavia University and Salinas Scene remote sensing datasets. The results are compared with the state-of-the-art hand-crafted as well as end-to-end deep learning based methods. A very satisfactory performance is obtained using the proposed HybridSN for HSI classification. The source code can be found at \url{https://github.com/gokriznastic/HybridSN}.

• The paper introduces a novel hybrid CNN architecture that combines 3D and 2D convolutions to effectively capture both spectral and spatial features.
• It achieves outstanding accuracy, reaching up to 100% on benchmark datasets while reducing the computational load compared to pure 3D CNN models.
• Experimental validation on multiple datasets underscores its potential for real-time applications in remote sensing and environmental monitoring.

HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification

Introduction

The paper "HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification" presents a novel hybrid Convolutional Neural Network (CNN) architecture that combines 3D and 2D CNNs for hyperspectral image (HSI) classification. This architecture, termed HybridSN, intends to capture both spectral and spatial information effectively, while addressing computational complexity issues typically associated with 3D CNNs alone.

Core Contribution

HybridSN leverages a two-stage feature extraction process:

1. 3D-CNN: Initially, the 3D CNN layers perform joint spectral-spatial feature extraction from a stack of spectral bands. This enables the model to capture the essential correlations among spectral bands, which is crucial for HSI classification.
2. 2D-CNN: Subsequently, the 2D CNN layers focus on higher-level spatial feature extraction, leveraging the features produced by the 3D convolution layers.

This hybrid approach not only retains valuable spatial and spectral information but also mitigates the computational overhead typically associated with 3D CNNs.

Experimental Validation

The HybridSN model was rigorously evaluated on three benchmark hyperspectral datasets: Indian Pines, University of Pavia, and Salinas Scene. The evaluation metrics included Overall Accuracy (OA), Average Accuracy (AA), and Kappa Coefficient, with comparisons made against several state-of-the-art methods: SVM, 2D-CNN, 3D-CNN, M3D-CNN, and SSRN.

Results

• Indian Pines Dataset: HybridSN achieved an OA of 99.75%, outperforming SSRN (99.19%) and significantly surpassing classic methods like SVM (85.30%) and standalone CNNs (2D-CNN: 89.48%, 3D-CNN: 91.10%).
• University of Pavia Dataset: HybridSN demonstrated an OA of 99.98%, equal to or better than SSRN and considerably higher than methods such as M3D-CNN (95.76%).
• Salinas Scene Dataset: HybridSN reached 100% OA, outperforming all compared methods. This underscores its robustness and capability in handling extensive spectral information effectively.

Computational Efficiency

HybridSN's efficiency was another focal point of the paper. Although 3D-CNN layers introduce computational intensity, the subsequent 2D-CNN layers help balance this by reducing redundant computations. This design choice results in lower overall complexity compared to using deep 3D CNNs alone. As reported, the training and testing times for HybridSN were competitive, often faster than vanilla 3D-CNNs.

Implications and Future Work

The implications of this research are twofold:

1. Practical Applications: Leveraging both spatial and spectral features without prohibitive computational costs makes HybridSN suitable for real-time applications in remote sensing, environmental monitoring, and agricultural management.
2. Theoretical Developments: The introduced hybrid approach can inspire further innovations in network architectures for various high-dimensional data applications, emphasizing the blend of different convolution strategies.

Future work might focus on extending this architecture to other types of high-dimensional data or incorporating additional types of processing layers, such as attention mechanisms, to further enhance classification performance and efficiency.

Conclusion

The HybridSN model effectively bridges the gap between computational efficiency and high accuracy in hyperspectral image classification. Its hierarchical design proposes a significant advance in the field, underscoring the utility of combining 3D and 2D convolutional layers for nuanced data representation. The experimental results validate HybridSN's superiority over current state-of-the-art methods, establishing it as a robust and efficient tool for HSI classification tasks.