Binarized Spectral Compressive Imaging (2305.10299v3)

Abstract: Existing deep learning models for hyperspectral image (HSI) reconstruction achieve good performance but require powerful hardwares with enormous memory and computational resources. Consequently, these methods can hardly be deployed on resource-limited mobile devices. In this paper, we propose a novel method, Binarized Spectral-Redistribution Network (BiSRNet), for efficient and practical HSI restoration from compressed measurement in snapshot compressive imaging (SCI) systems. Firstly, we redesign a compact and easy-to-deploy base model to be binarized. Then we present the basic unit, Binarized Spectral-Redistribution Convolution (BiSR-Conv). BiSR-Conv can adaptively redistribute the HSI representations before binarizing activation and uses a scalable hyperbolic tangent function to closer approximate the Sign function in backpropagation. Based on our BiSR-Conv, we customize four binarized convolutional modules to address the dimension mismatch and propagate full-precision information throughout the whole network. Finally, our BiSRNet is derived by using the proposed techniques to binarize the base model. Comprehensive quantitative and qualitative experiments manifest that our proposed BiSRNet outperforms state-of-the-art binarization methods and achieves comparable performance with full-precision algorithms. Code and models are publicly available at https://github.com/caiyuanhao1998/BiSCI and https://github.com/caiyuanhao1998/MST

• The paper presents BiSRNet, a novel model that supports efficient hyperspectral image reconstruction on mobile and edge devices.
• It leverages binarized spectral-redistribution convolution and optimized modules to minimize computational load while maintaining high precision.
• Experimental results show over 2.5 dB improvement and drastically reduced resource usage, enabling practical applications in fields like agriculture and medical imaging.

Binarized Spectral Compressive Imaging

The paper "Binarized Spectral Compressive Imaging" presents a novel approach targeted at improving hyperspectral image (HSI) reconstruction on resource-constrained devices. Existing methods, while effective, demand vast computational and memory resources which limits their deployment on mobile and edge devices. The authors introduce the Binarized Spectral-Redistribution Network (BiSRNet), designed specifically to address the challenges of efficiency and deployment without sacrificing performance.

Technical Contributions

The authors outline several key innovations:

1. Base Model Optimization: The design of a compact, easily deployable base model eschewing complex operations such as non-local self-attention and unfolding inference commonly used in CNN and Transformer architectures. This ensures compatibility with the bitwise operations required in binarized networks.
2. Binarized Spectral-Redistribution Convolution (BiSR-Conv): The core unit that acts to preprocess HSI representations across spectral channels before binarization. It uses a learnable redistribution mechanism, adapting the spectral characteristics prior to applying the Sign function. Moreover, a scalable hyperbolic tangent function is proposed, reducing the approximation error during backpropagation.
3. Binarized Convolutional Modules: Four binarized convolutional modules are developed to address issues like dimension mismatch during feature reshaping, ensuring that full-precision information propagates effectively through the layers. These include downsample, upsample, fusion up, and fusion down modules.

Experimental Results

Extensive experiments demonstrate that BiSRNet outperforms state-of-the-art binarized neural networks (BNNs) by notable margins, achieving more than 2.5 dB improvements over the best competitors on the simulation task. What's striking is that BiSRNet provides results comparable to full-precision CNNs, with significantly reduced memory and computational requirements. For instance, BiSRNet surpasses the performance of λ-Net by 1.23 dB while only utilizing 0.06% of its parameters and 1.0% of its operations.

Implications and Future Directions

BiSRNet delivers substantial benefits for deploying HSI reconstruction on devices with limited resources. This advancement has profound implications for a range of applications such as agricultural monitoring, medical imaging, and remote sensing, where mobile accessibility and platform limitations are significant considerations.

Looking forward, the integration of BiSRNet in more complex or hybrid systems suggests promising paths for further research. Exploring other quantization strategies or low-bit computations might yield models with even greater operational efficiency. As the hardware capabilities of edge devices continue to evolve, frameworks like BiSRNet could be pivotal in democratizing access to advanced imaging technologies.

In conclusion, "Binarized Spectral Compressive Imaging" provides a significant step towards efficient HSI restoration, balancing performance with the practical restrictions of edge computing. This paper underscores the potential of BNNs in pushing the boundaries of what's feasible within limited computational constraints, setting a foundation for future explorations in this domain.