Bidirectional-Convolutional LSTM Based Spectral-Spatial Feature Learning for Hyperspectral Image Classification (1703.07910v1)

Abstract: This paper proposes a novel deep learning framework named bidirectional-convolutional long short term memory (Bi-CLSTM) network to automatically learn the spectral-spatial feature from hyperspectral images (HSIs). In the network, the issue of spectral feature extraction is considered as a sequence learning problem, and a recurrent connection operator across the spectral domain is used to address it. Meanwhile, inspired from the widely used convolutional neural network (CNN), a convolution operator across the spatial domain is incorporated into the network to extract the spatial feature. Besides, to sufficiently capture the spectral information, a bidirectional recurrent connection is proposed. In the classification phase, the learned features are concatenated into a vector and fed to a softmax classifier via a fully-connected operator. To validate the effectiveness of the proposed Bi-CLSTM framework, we compare it with several state-of-the-art methods, including the CNN framework, on three widely used HSIs. The obtained results show that Bi-CLSTM can improve the classification performance as compared to other methods.

• The paper proposes a novel Bidirectional-Convolutional LSTM (Bi-CLSTM) network that effectively combines bidirectional LSTM for spectral sequence learning with convolutional layers for spatial feature extraction, significantly enhancing hyperspectral image classification.
• Experimental validation on multiple datasets shows that the Bi-CLSTM achieved superior classification performance (higher OA, AA, Kappa) compared to traditional and state-of-the-art methods, notably reaching 96.78% OA on the Indian Pines dataset.
• The Bi-CLSTM framework's enhanced spectral-spatial feature extraction improves precision for applications like environmental monitoring and urban mapping, suggesting potential for future advancements using attention mechanisms or efficient training techniques.

Bidirectional-Convolutional LSTM for Hyperspectral Image Classification

The paper highlights the development of a Bidirectional-Convolutional Long Short Term Memory (Bi-CLSTM) network specifically designed for hyperspectral image (HSI) classification. By addressing both spectral and spatial features, the proposed framework strives to revolutionize image classification processes by leveraging deep learning architectures. It enhances the capability to extract robust features automatically, thereby improving classification accuracy compared to traditional methods.

Methodology

The proposed Bi-CLSTM framework integrates two fundamental components: Long Short Term Memory (LSTM) and Convolutional Neural Networks (CNN), optimized for spectral-spatial feature learning. The LSTM network addresses the spectral dimension under a sequence learning paradigm, whereas the CNN-inspired convolutional operator processes the spatial dimension. The pivotal feature of this architecture is its bidirectional recurrent network, allowing it to comprehensively capture correlations across spectral bands, simultaneously learning spatial hierarchies.

The framework utilizes a recurrent connection across the spectral dimension to assimilate the interrelated nature of spectral channels. This bidirectional connectivity leverages both previous and future states within the sequence, enabling a contextually richer feature representation. In the spatial domain, convolutional operators supplant fully connected layers, thus ensuring better spatial feature extraction without sacrificing spectral integrity.

Experimental Validation

To verify its efficacy, the Bi-CLSTM model was tested on three distinct hyperspectral datasets – Indian Pines, Pavia University, and Kennedy Space Center (KSC). These datasets presented varied conditions, aiding in the robustness validation of the proposed model across different environments and sensor types (AVIRIS and ROSIS). Classification performance was measured against several state-of-the-art methods, including MVC and RLDE, as well as traditional feature extraction methods like PCA, LDA, NWFE, and CNN. The Bi-CLSTM demonstrated superior classification performance with higher Overall Accuracy (OA), Average Accuracy (AA), and Kappa coefficient ($\kappa$).

Results

The experimental results corroborate the effectiveness of the Bi-CLSTM. For instance, the Bi-CLSTM achieved a 96.78% OA on the Indian Pines dataset, significantly outperforming traditional methods which topped at 92.31% with MDA. Similar improvements were noted across other datasets, indicating the model's potential for enhancing HSI classification's precision and consistency. The integration of dropout and data augmentation techniques also contributed to mitigate overfitting, thereby refining the Bi-CLSTM's performance, particularly with limited training samples.

Implications and Future Directions

The Bi-CLSTM framework's capacity to enhance spectral-spatial feature extraction underscores its potential applications in various domains requiring precise environmental monitoring, urban mapping, or agricultural assessment using hyperspectral imagery. Given this capability, future work may focus on further optimizing network architectures or extending the application of Bi-CLSTM to other types of sequential data where bidirectional feature extraction can offer insights.

Moreover, as DL frameworks become more sophisticated, incorporating additional components such as attention mechanisms or expanding into generative models may boost classification performance further. In practice, an exploration of efficient training techniques or robust feature selection approaches could also pave the way for more streamlined and automatic HSI classification solutions.

In summary, the Bi-CLSTM model presents a considerable advancement in hyperspectral image classification, promising improved analytical precision vital for diverse scientific and industrial applications.