FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification (2011.05670v1)

Abstract: Deep learning techniques have provided significant improvements in hyperspectral image (HSI) classification. The current deep learning based HSI classifiers follow a patch-based learning framework by dividing the image into overlapping patches. As such, these methods are local learning methods, which have a high computational cost. In this paper, a fast patch-free global learning (FPGA) framework is proposed for HSI classification. In FPGA, an encoder-decoder based FCN is utilized to consider the global spatial information by processing the whole image, which results in fast inference. However, it is difficult to directly utilize the encoder-decoder based FCN for HSI classification as it always fails to converge due to the insufficiently diverse gradients caused by the limited training samples. To solve the divergence problem and maintain the abilities of FCN of fast inference and global spatial information mining, a global stochastic stratified sampling strategy is first proposed by transforming all the training samples into a stochastic sequence of stratified samples. This strategy can obtain diverse gradients to guarantee the convergence of the FCN in the FPGA framework. For a better design of FCN architecture, FreeNet, which is a fully end-to-end network for HSI classification, is proposed to maximize the exploitation of the global spatial information and boost the performance via a spectral attention based encoder and a lightweight decoder. A lateral connection module is also designed to connect the encoder and decoder, fusing the spatial details in the encoder and the semantic features in the decoder. The experimental results obtained using three public benchmark datasets suggest that the FPGA framework is superior to the patch-based framework in both speed and accuracy for HSI classification. Code has been made available at: https://github.com/Z-Zheng/FreeNet.

• The paper introduces a novel fast patch-free global learning framework to enhance hyperspectral image classification.
• It replaces patch-based methods with a fully convolutional encoder-decoder design and GS² sampling, achieving an overall accuracy of 99.81% on benchmark datasets.
• The lateral connection based semantic-spatial fusion effectively integrates global and local image information, refining feature maps for clearer classification results.

FPGA: Fast Patch-Free Global Learning Framework for Hyperspectral Image Classification

The paper "FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification" spearheads a novel approach for hyperspectral image (HSI) classification by introducing a fast, patch-free global learning (FPGA) framework. It tackles the existing limitations of patch-based methods prevalent in this domain, primarily focused on computational inefficiencies arising due to redundant calculations in overlapping patches. With a comprehensive design and inclusion of innovative mechanisms, this proposal enhances both efficiency and accuracy in processing HSIs.

Core Innovations and Contributions

The FPGA framework comprises three primary components:

1. Global Stochastic Stratified (GS²) Sampling Strategy: This sampling strategy ensures the diversity of gradients during FCN training by creating a stochastic sequence of stratified samples, thereby enabling the convergence of the deep learning models even when training samples are limited.
2. Encoder-Decoder Based Fully Convolutional Network (FCN): The FCN framework replaces the conventional patch-based methods by considering entire hyperspectral images, thus leveraging global spatial information for faster inference and improved classification performance.
3. Lateral Connection Based Semantic-Spatial Fusion (SSF): The introduction of lateral connections between the encoder and the decoder facilitates the fusion of spatial details with semantic information to refine feature maps, ultimately producing clearer classification maps.

Additionally, the paper introduces FreeNet as a dedicated FCN architecture tailored for hyperspectral classification. FreeNet incorporates spectral attention and lightweight decoder components to efficiently manage the spectral-spatial information inherent to hyperspectral data. Notably, the spectral attention module emphasizes global spatial context while preserving essential local detail features.

Experimental Validation

The authors validate the efficacy of the FPGA framework with extensive experiments on three benchmark datasets: Pavia University, Salinas, and University of Houston HSIs. Their results unequivocally demonstrate that FPGA not only accelerates processing times dramatically compared to traditional patch-based frameworks but also achieves superior overall accuracy (OA), average accuracy (AA), and Kappa coefficient across diverse datasets. Notably, the framework achieves an OA of 99.81% on the Pavia University dataset, setting a new performance standard.

Implications and Future Directions

The demonstrated success of FPGA underscores the integral role of model efficiency in real-world hyperspectral applications. By reducing computational overhead substantially, the proposed framework aligns with current trends in remote sensing, where data volume continues to rival computational resources.

Future research could primarily focus on two areas inspired by this work: first, exploring further optimizations for the GS² sampling strategy, potentially incorporating adaptive sampling techniques to dynamically adjust mini-batch sizes based on data complexity. Secondly, extending the FPGA framework’s applicability beyond HSI to other forms of high-dimensional remote sensing data may foster advancements in adjacent domains. The scalability and adaptability of FreeNet present an interesting research avenue in terms of generalizing hyperspectral processing capabilities.

In conclusion, the FPGA framework offers a pragmatic shift from localized to global learning methodologies in HSI classification. By enhancing computational performance and accuracy, it sets a promising precedent for both theoretical advancements and practical implementations in remote sensing technology.