Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging (2205.10102v3)

Abstract: In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement. Among these algorithms, deep unfolding methods demonstrate promising performance but suffer from two issues. Firstly, they do not estimate the degradation patterns and ill-posedness degree from the highly related CASSI to guide the iterative learning. Secondly, they are mainly CNN-based, showing limitations in capturing long-range dependencies. In this paper, we propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. Moreover, we customize a novel Half-Shuffle Transformer (HST) that simultaneously captures local contents and non-local dependencies. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST), for HSI reconstruction. Experiments show that DAUHST significantly surpasses state-of-the-art methods while requiring cheaper computational and memory costs. Code and models will be released at https://github.com/caiyuanhao1998/MST

• The paper presents a degradation-aware unfolding framework based on MAP theory that estimates key degradation parameters for improved HSI reconstruction.
• The paper develops a Half-Shuffle Transformer to capture both local and non-local dependencies, achieving significant performance gains.
• The paper demonstrates that combining degradation-aware estimation with transformer-based unfolding enhances computational efficiency and paves the way for real-time imaging applications.

Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

The paper presents an innovative approach to hyperspectral image (HSI) reconstruction in coded aperture snapshot spectral compressive imaging (CASSI) systems, introducing a Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST) framework. The focus is on addressing the limitations observed in existing deep unfolding methods, which primarily struggle with estimating degradation patterns and capturing long-range dependencies.

Core Contributions

1. Degradation-Aware Unfolding Framework (DAUF): The authors propose a novel degradation-aware unfolding framework rooted in the maximum a posteriori (MAP) theory. This framework distinguishes itself by estimating informative parameters from both the compressed image and the physical mask. These parameters, which capture critical aspects of degradation patterns and ill-posedness, are used to effectively direct the iterative learning process. DAUF aims to optimize computational efficiency and increase reconstruction quality by adaptively scaling linear projections and setting noise levels in the denoising network.
2. Half-Shuffle Transformer (HST): Building on the limitations of CNNs in capturing non-local dependencies, the paper introduces the HST. This architecture incorporates a Half-Shuffle Multi-head Self-Attention (HS-MSA) mechanism designed to excel in both local and non-local information extraction. By integrating this with DAUF, the paper marks the development of the first Transformer-based deep unfolding method for HSI reconstruction.

Experimental Insights

The DAUHST shows significant improvements over state-of-the-art HSI reconstruction methods. By leveraging both the DAUF strategy and the HST's enhanced attention mechanisms, the proposed approach achieves superior performance with over 4 dB improvement in PSNR, alongside competitive computational and memory efficiency.

Technical Implications

From a theoretical standpoint, this research highlights the effectiveness of integrating estimated degradation-aware parameters in the unfolding process. The MAP-based DAUF allows the iterative model to adaptively evolve with degradation degrees, optimizing the reconstruction process based on systematically learned cues.

On the application side, the implementation of DAUHST in real CASSI systems could expedite the process of HSI reconstruction in dynamic conditions. Furthermore, the demonstrated computational efficiency makes it a viable option for real-time applications, where resource constraints might limit the utility of more computationally expensive models.

Future Directions

This research opens several pathways for further exploration:

• Broader Transformer Architectures: The focus on leveraging Transformer architectures could be expanded to explore other modalities and domains in compressive imaging.
• Parameter Estimation Enhancement: Further research could investigate more advanced parameter estimation techniques to refine degradation modeling and adaptive response in unfolding methods.
• Real-World Applications: Integrating DAUHST into real-time imaging systems across various fields, such as medical imaging or remote sensing, could be invaluable in evaluating and expanding its practical applications.

Conclusion

The DAUHST framework represents a significant advancement in the field of spectral compressive imaging, addressing critical challenges of existing methodologies. Its unique integration of degradation-aware strategies with Transformer capabilities paves the way for more efficient and accurate hyperspectral image reconstruction, promising substantial impact in various applied domains of imaging technology.