MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification (2501.04944v1)

Abstract: Transformer has been extensively explored for hyperspectral image (HSI) classification. However, transformer poses challenges in terms of speed and memory usage because of its quadratic computational complexity. Recently, the Mamba model has emerged as a promising approach, which has strong long-distance modeling capabilities while maintaining a linear computational complexity. However, representing the HSI is challenging for the Mamba due to the requirement for an integrated spatial and spectral understanding. To remedy these drawbacks, we propose a novel HSI classification model based on a Mamba model, named MambaHSI, which can simultaneously model long-range interaction of the whole image and integrate spatial and spectral information in an adaptive manner. Specifically, we design a spatial Mamba block (SpaMB) to model the long-range interaction of the whole image at the pixel-level. Then, we propose a spectral Mamba block (SpeMB) to split the spectral vector into multiple groups, mine the relations across different spectral groups, and extract spectral features. Finally, we propose a spatial-spectral fusion module (SSFM) to adaptively integrate spatial and spectral features of a HSI. To our best knowledge, this is the first image-level HSI classification model based on the Mamba. We conduct extensive experiments on four diverse HSI datasets. The results demonstrate the effectiveness and superiority of the proposed model for HSI classification. This reveals the great potential of Mamba to be the next-generation backbone for HSI models. Codes are available at https://github.com/li-yapeng/MambaHSI .

• The paper introduces MambaHSI, a novel hyperspectral image classification model leveraging Mamba, a state space model, to efficiently process spatial and spectral information.
• MambaHSI addresses the computational complexity of transformer models by utilizing Mamba's linear complexity for modeling long-range dependencies in large datasets.
• Empirical evaluations demonstrate MambaHSI's superior performance compared to existing methods across multiple datasets, setting new benchmarks in classification accuracy.

An Overview of MambaHSI: A Novel SSM-based Model for Hyperspectral Image Classification

The paper entitled "MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification" introduces an innovative approach to hyperspectral image (HSI) classification by leveraging the capabilities of state space models, specifically the Mamba model. This research notably advances the domain of HSI classification, addressing the limitations of traditional methods by adopting a novel framework that simultaneously models long-range dependencies and integrates spatial-spectral information effectively.

Key Contributions

1. Novel Architecture: The proposed MambaHSI framework presents an image-level HSI classification model that uniquely employs Mamba, a state space model (SSM), for maintaining linear computational complexity while modeling long-range dependencies. The framework effectively encapsulates this through distinct spatial and spectral Mamba blocks, which handle pixel-level spatial features and spectral sequence information, respectively.
2. Efficient Long-range Modeling: A significant challenge in HSI classification is the computational burden associated with transformer models due to their quadratic complexity in sequence length. MambaHSI circumvents this by leveraging Mamba's linear complexity for sequence processing, which is crucial for handling long-range dependencies in large-scale hyperspectral datasets.
3. Integration of Spatial-Spectral Features: The inclusion of a spatial-spectral fusion module is pivotal in the architecture of MambaHSI. This module adaptively fuses the spatial and spectral features, guided by their relative significance, to provide a comprehensive feature set for the classification task.
4. Robust Performance: The empirical evaluations of MambaHSI demonstrate its superior performance across multiple widely-used hyperspectral datasets when compared against existing CNN-based, transformer-based, and traditional machine learning methods. It is noteworthy that even with a basic Mamba implementation, MambaHSI sets new benchmarks in accuracy, supporting its potential as a next-generation HSI classification backbone.

Strong Numerical Results

The research undertakes extensive experiments on datasets such as Pavia University, Houston, WHU-Hi-HanChuan, and WHU-Hi-HongHu, showcasing the robustness of MambaHSI. Notably, the model achieves significant improvements in overall accuracy (OA) and average accuracy (AA) compared to its contemporaries. For instance, on the Houston dataset, MambaHSI attains an OA of 94.46% and AA of 95.34%, which emphasizes its efficacy in capturing fine-grained spatial dependencies and spectral correlations.

Implications and Future Directions

The results and framework presented in this work innovate the landscape of HSI classification, demonstrating the untapped potential of state space models in this domain. The success of MambaHSI not only indicates the efficacy of SSMs in processing spatial-spectral data but also suggests their broader applicability in various machine learning tasks that demand efficient long-range dependency modeling.

Looking forward, the integration of more advanced variants of the Mamba model, such as those tailored for visual attention tasks, could propel further improvements in both efficiency and accuracy. Moreover, extending this model to applications like weakly supervised classification and HSI clustering could substantially impact fields like remote sensing and environmental monitoring.

In conclusion, MambaHSI marks a significant stride in hyperspectral image classification, efficiently bridging the gap between deep learning's potential and the practical constraints of computational complexity. Its ability to process whole images at once, coupled with outstanding classification accuracy, positions it as a pivotal model in advancing remote sensing techniques and real-world applications.