Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging (2003.13654v2)

Abstract: Snapshot compressive imaging (SCI) aims to capture the high-dimensional (usually 3D) images using a 2D sensor (detector) in a single snapshot. Though enjoying the advantages of low-bandwidth, low-power and low-cost, applying SCI to large-scale problems (HD or UHD videos) in our daily life is still challenging. The bottleneck lies in the reconstruction algorithms; they are either too slow (iterative optimization algorithms) or not flexible to the encoding process (deep learning based end-to-end networks). In this paper, we develop fast and flexible algorithms for SCI based on the plug-and-play (PnP) framework. In addition to the widely used PnP-ADMM method, we further propose the PnP-GAP (generalized alternating projection) algorithm with a lower computational workload and prove the convergence of PnP-GAP under the SCI hardware constraints. By employing deep denoising priors, we first time show that PnP can recover a UHD color video ($3840\times 1644\times 48$ with PNSR above 30dB) from a snapshot 2D measurement. Extensive results on both simulation and real datasets verify the superiority of our proposed algorithm. The code is available at https://github.com/liuyang12/PnP-SCI.

• The paper introduces innovative plug-and-play algorithms, PnP-ADMM and PnP-GAP, to address computational challenges in large-scale snapshot compressive imaging.
• The methodology leverages CNN-based denoisers like FFDNet to efficiently reconstruct UHD video data with a PSNR above 30dB.
• The work establishes fixed-point and global convergence theories, paving the way for practical applications in surveillance, cinematography, and scientific imaging.

Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging

Snapshot compressive imaging (SCI) serves as a transformative approach to capturing high-dimensional (3D) images using a two-dimensional (2D) sensor, mitigating typical high-bandwidth and high-power requirements. Despite these advantages, applying SCI to large-scale applications such as high-definition (HD) and ultra-high-definition (UHD) videos remains a formidable challenge. The difficulty primarily stems from reconstruction algorithms that are either computationally intensive or insufficiently adaptable to varying encoding processes.

This publication introduces fast and adaptable algorithms within the plug-and-play (PnP) framework, specifically PnP-ADMM and a novel PnP-GAP algorithm, aimed at addressing the exigencies of large-scale SCI problems. PnP-GAP, with its reduced computational demands, emerges as a promising solution, supplemented by the incorporation of deep denoising priors. Notably, the paper achieves a milestone in showing the feasibility of reconstructing a UHD color video with a resolution of $3840 \times 1644 \times 48$ and a peak signal-to-noise ratio (PSNR) above 30dB from a single snapshot measurement.

Methodological Advancements

The developed PnP algorithms leverage convolutional neural network-based denoisers, particularly the FFDNet, which is known for its speed and adaptability across varying noise levels. The PnP framework, with its inherent flexibility, enables the integration of multiple types of denoisers to optimize the reconstruction process, thereby improving both speed and image quality. The theoretical underpinning that guarantees convergence of the PnP-GAP algorithm, contingent on hardware constraints specific to SCI, is also rigorously established.

Theoretical Implications

This work makes significant contributions to the domain of SCI, establishing a fixed-point convergence theory for PnP-ADMM and demonstrating global convergence for PnP-GAP. The framework’s flexibility allows for the seamless integration and substitution of different denoisers, thereby offering potential pathways to even more efficient and accurate reconstruction techniques in the future. The thorough analysis paves the way for extending the utility of SCI in practical applications, overcoming challenges associated with high-resolution video capturing.

Practical Implications

Through extensive experimentation across simulated and real datasets—ranging from benchmark 3D datasets to real-world data captured by SCI cameras—the proposed methods have been validated to deliver superior performance. The introduction of PnP-FFDNet as an efficient baseline for SCI reconstruction attests to its practical relevance, especially given its substantial acceleration compared to traditional methods like DeSCI. Importantly, the paper verifies that SCI can be efficaciously applied to real-world scenarios involving large-scale, high-resolution imagery.

Future Directions

The research suggests promising avenues for future exploration, notably the development of advanced video denoising networks that could be integrated into the PnP framework to push the boundaries of SCI further. Additionally, establishing large-scale video SCI systems in practical applications could catalyze advancements in numerous fields, including surveillance, cinematography, and scientific imaging.

This paper's comprehensive approach to developing scalable, adaptable reconstruction methods signals a significant progression in enabling practical applications of SCI technology in capturing dynamic scenes at high resolutions efficiently.