Graph Convolutional Networks for Hyperspectral Image Classification (2008.02457v2)

Abstract: To read the final version please go to IEEE TGRS on IEEE Xplore. Convolutional neural networks (CNNs) have been attracting increasing attention in hyperspectral (HS) image classification, owing to their ability to capture spatial-spectral feature representations. Nevertheless, their ability in modeling relations between samples remains limited. Beyond the limitations of grid sampling, graph convolutional networks (GCNs) have been recently proposed and successfully applied in irregular (or non-grid) data representation and analysis. In this paper, we thoroughly investigate CNNs and GCNs (qualitatively and quantitatively) in terms of HS image classification. Due to the construction of the adjacency matrix on all the data, traditional GCNs usually suffer from a huge computational cost, particularly in large-scale remote sensing (RS) problems. To this end, we develop a new mini-batch GCN (called miniGCN hereinafter) which allows to train large-scale GCNs in a mini-batch fashion. More significantly, our miniGCN is capable of inferring out-of-sample data without re-training networks and improving classification performance. Furthermore, as CNNs and GCNs can extract different types of HS features, an intuitive solution to break the performance bottleneck of a single model is to fuse them. Since miniGCNs can perform batch-wise network training (enabling the combination of CNNs and GCNs) we explore three fusion strategies: additive fusion, element-wise multiplicative fusion, and concatenation fusion to measure the obtained performance gain. Extensive experiments, conducted on three HS datasets, demonstrate the advantages of miniGCNs over GCNs and the superiority of the tested fusion strategies with regards to the single CNN or GCN models. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_GCN for the sake of reproducibility.

• The paper introduces miniGCNs, a mini-batch GCN variant that efficiently handles large-scale hyperspectral image classification.
• It demonstrates that fusing CNN and miniGCN features improves classification accuracy and produces smoother, detailed maps.
• Experimental results on datasets like Indian Pines and Pavia University confirm that miniGCNs outperform traditional GCNs and baseline classifiers.

Analyzing Graph Convolutional Networks for Hyperspectral Image Classification

Introduction

Hyperspectral (HS) image classification is a demanding task due to the intricate and high-dimensional nature of the data. Traditional convolutional neural networks (CNNs) have been widely applied for this purpose, leveraging their strong capability to capture spatial-spectral features within HS images. Nonetheless, CNNs are inadequate for capturing relational structures between data samples owing to their grid-based design. To address this limitation, the paper under discussion evaluates the potential of Graph Convolutional Networks (GCNs) in HS image classification. GCNs excel in modeling data with graph structures, thus holding promise for capturing non-local dependencies in HS data. The authors propose a novel mini-batch GCN approach (miniGCNs) and investigate its efficacy alongside CNNs, providing detailed experimental results and insights into their fusion.

Graph Convolutional Networks and their Limitations

The foundation of GCNs lies in their ability to operate on graph-structured data rather than grid-structured data like CNNs. This attribute is particularly vital for HS data characterized by complex spatial and spectral relationships. However, traditional GCNs are computationally prohibitive for large-scale HS data due to the construction of large adjacency matrices, which must capture relationships among all pixels. Additionally, GCNs necessitate full-batch updates, hindering their scalability and leading to substantial memory requirements. There is also the added problem that GCN models, once trained, cannot infer new samples directly without retraining.

Proposed Mini-Batch GCN (miniGCN)

To alleviate the inherent limitations of GCNs, the authors propose a mini-batch variant called miniGCN. The miniGCN approach introduces the ability to train GCNs in a mini-batch fashion similar to CNNs, efficiently managing computational resources while maintaining model accuracy. Crucially, miniGCNs retain the capacity to process new samples without necessitating re-training, thereby enhancing their practical utility. These features render miniGCNs a more robust and scalable solution for HS image classification.

Fusion of CNNs and GCNs

Recognizing that CNNs and miniGCNs extract complementary features, the authors explore their fusion to further improve classification performance. CNNs excel at capturing local spatial-spectral features, while miniGCNs effectively model long-range dependencies and relational structures within the data. The authors propose three fusion strategies:

• Additive Fusion (FuNet-A): Element-wise addition of features from CNNs and miniGCNs.
• Element-wise Multiplicative Fusion (FuNet-M): Element-wise multiplication of features.
• Concatenation Fusion (FuNet-C): Concatenation of features extracted from the two networks.

Experimental Results

The algorithms were evaluated on three well-known HS datasets: Indian Pines, Pavia University, and Houston2013. The experiments aimed to compare the standalone performance of traditional classifiers (KNN, RF, SVM), CNNs, GCNs, and the newly proposed miniGCNs, as well as the fusion networks (FuNet-A, FuNet-M, FuNet-C).

The experimental results indicate that:

• The miniGCNs consistently outperform traditional GCNs and other baseline classifiers.
• FuNet-C, the concatenation fusion strategy, often yields the highest overall performance, outperforming both standalone CNNs and miniGCNs.
• The fusion approach results in smoother and more detailed classification maps, highlighting the effectiveness of combining spatial-spectral features with relational context.

Implications and Future Work

The development of miniGCNs presents a significant advancement in HS image analysis, addressing critical issues in computational scalability and the ability to generalize to new data. The fusion strategies explored in the paper demonstrate the potential of combining different feature representation methods to enhance classification accuracy further.

This research opens various avenues for future work:

• Incorporating more sophisticated graph construction techniques and dynamic graph updates could yield further improvements.
• Investigating other deep learning architectures in conjunction with miniGCNs could identify even more potent combinations.
• Developing advanced fusion techniques, such as weighted or attention-based fusion, could leverage the complementary strengths of different networks more effectively.

Conclusion

This paper offers an in-depth examination of the use of GCNs in hyperspectral image classification, highlighting the shortcomings of traditional GCNs and proposing miniGCNs as a robust alternative. Furthermore, the research underscores the benefits of fusing CNNs and miniGCNs, presenting substantial performance gains in multiple HS datasets. The insights and methodologies presented in this work will contribute significantly to the field, prompting future developments in the domain of HS image analysis.