Dual-stage Hyperspectral Image Classification Model with Spectral Supertoken (2407.07307v2)

Abstract: Hyperspectral image classification, a task that assigns pre-defined classes to each pixel in a hyperspectral image of remote sensing scenes, often faces challenges due to the neglect of correlations between spectrally similar pixels. This oversight can lead to inaccurate edge definitions and difficulties in managing minor spectral variations in contiguous areas. To address these issues, we introduce the novel Dual-stage Spectral Supertoken Classifier (DSTC), inspired by superpixel concepts. DSTC employs spectrum-derivative-based pixel clustering to group pixels with similar spectral characteristics into spectral supertokens. By projecting the classification of these tokens onto the image space, we achieve pixel-level results that maintain regional classification consistency and precise boundary. Moreover, recognizing the diversity within tokens, we propose a class-proportion-based soft label. This label adaptively assigns weights to different categories based on their prevalence, effectively managing data distribution imbalances and enhancing classification performance. Comprehensive experiments on WHU-OHS, IP, KSC, and UP datasets corroborate the robust classification capabilities of DSTC and the effectiveness of its individual components. Code will be publicly available at https://github.com/laprf/DSTC.

• The paper introduces a dual-stage model that improves hyperspectral image classification by leveraging spectral derivative-based pixel clustering.
• It refines boundary detection and handles spectral variations using a novel spectral supertoken mechanism within a transformer framework.
• Quantitative evaluations on benchmark datasets demonstrate enhanced accuracy and edge precision, outperforming several contemporary deep learning models.

Dual-stage Hyperspectral Image Classification Model with Spectral Supertoken

The paper "Dual-stage Hyperspectral Image Classification Model with Spectral Supertoken" presents a new framework to address the intricate challenges associated with hyperspectral image classification (HSIC). The challenges in HSIC primarily stem from the mismanagement of correlations among spectrally similar pixels, which often lead to inaccuracies in boundary detection and difficulties in handling slight spectral variations within homogeneous regions. The manuscript introduces the Dual-stage Spectral Supertoken Classifier (DSTC) as an innovative approach to tackle such issues effectively.

Methodology

The Dual-stage Spectral Supertoken Classifier operates in two definitive stages:

1. Spectrum-derivative-based Pixel Clustering: The first stage is pivotal as it clusters pixels based on spectral similarities, leading to the formation of spectral supertokens. This process employs spectral derivative features in determining the clustering attributes, thereby enhancing boundary definition accuracy and reducing classification errors within contiguous regions. The incorporation of both first-order and second-order spectral derivatives aids in refining spectral features that are typically obscured in initial analyses.
2. Classification via Spectral Supertokens: In the second stage, these spectral supertokens undergo classification using a Transformer-based architecture. This not only facilitates robust classification performance but concurrently manages computational efficiency by considerable reduction in primitive image data.

Further, the paper introduces class-proportion-based soft labels as a novel form of result supervision. By assigning a soft label instead of a hard label (a single definitive class), the DSTC manages the varied representation within each supertoken effectively. This approach addresses data imbalance issues and substantially improves the model's classification accuracy across datasets.

Results and Implications

Extensive quantitative and qualitative evaluations conducted on WHU-OHS, IP, KSC, and UP datasets underscore the DSTC's robust classification capabilities. Notably, it achieves impressive overall accuracy and F1 scores, outperforming several contemporary methods, including A2S2K, Capsule, and various CNN-based architectures. Of particular interest is the DSTC's ability to maintain edge precision and region consistency, which are critical in hyperspectral imaging applications.

The implications extend significantly to real-world sectors like urban planning, agriculture, and resource mining, where accurate land cover mapping and monitoring are pertinent. The methodology's adaptability with Transformer models hints at potential expandability into more dynamic contexts, possibly in domains requiring real-time hyperspectral data analysis.

Future Directions

Given the novel integration of pixel clustering and token classification, future research could explore the adaptability of DSTC on larger-scale or multi-source hyperspectral datasets. Furthermore, advancing the spectral clustering algorithms to support CUDA acceleration can enhance DSTC's real-time inference capabilities. Continued exploration of soft label assignments based on varying class proportions might yield further improvement in classification robustness, especially in datasets exhibiting significant class imbalances.

In conclusion, the Dual-stage Spectral Supertoken Classifier opens up new avenues in hyperspectral image classification by effectively balancing regional consistency and computational efficiency. Its contribution to the handling of spectral data distribution and image semantics marks a forward step in the use of deep learning models for complex data types like hyperspectral imagery.