ISTA-Net++: Flexible Deep Unfolding Network for Compressive Sensing

Abstract: While deep neural networks have achieved impressive success in image compressive sensing (CS), most of them lack flexibility when dealing with multi-ratio tasks and multi-scene images in practical applications. To tackle these challenges, we propose a novel end-to-end flexible ISTA-unfolding deep network, dubbed ISTA-Net++, with superior performance and strong flexibility. Specifically, by developing a dynamic unfolding strategy, our model enjoys the adaptability of handling CS problems with different ratios, i.e., multi-ratio tasks, through a single model. A cross-block strategy is further utilized to reduce blocking artifacts and enhance the CS recovery quality. Furthermore, we adopt a balanced dataset for training, which brings more robustness when reconstructing images of multiple scenes. Extensive experiments on four datasets show that ISTA-Net++ achieves state-of-the-art results in terms of both quantitative metrics and visual quality. Considering its flexibility, effectiveness and practicability, our model is expected to serve as a suitable baseline in future CS research. The source code is available on https://github.com/jianzhangcs/ISTA-Netpp.

• The paper introduces ISTA-Net$^{++}$, a deep unfolding network enhancing flexibility for multi-ratio compressive sensing tasks using dynamic strategies.
• The Dynamic Unfolding Strategy (DUS) allows ISTA-Net$^{++}$ to adaptively handle various compression ratios with a single model.
• The Cross-Block Strategy (CBS) helps mitigate blocking artifacts and improves reconstruction quality by leveraging inter-block information.

Analysis of ISTA-Net$^{++}$: A Flexible Deep Unfolding Network for Compressive Sensing

The paper introduces ISTA-Net$^{++}$, a deep unfolding network specifically designed to address challenges in image compressive sensing (CS). It focuses on improving flexibility and performance across multi-ratio tasks and multi-scene images. This model builds on the existing ISTA-Net$^+$ framework by integrating a dynamic unfolding strategy (DUS) and a cross-block strategy (CBS). ISTA-Net$^{++}$ exhibits compatibility for varying CS ratios, enabling efficient reconstruction with a single network model rather than multiple, ratio-specific models.

Key Innovations

• Dynamic Unfolding Strategy (DUS): ISTA-Net$^{++}$ employs DUS to dynamically adjust network parameters based on different CS ratios using a condition module (CM). This method enhances adaptability and robustness by processing inputs corresponding to multiple ratio values, which proved challenging in prior models necessitating separate training for distinct ratios.
• Cross-Block Strategy (CBS): The introduction of CBS is crucial to mitigating blocking artifacts common in block-based CS processes. By employing cross-block sampling and reconstruction, ISTA-Net$^{++}$ leverages inter-block information to improve recovery performance and edge preservation.
• Balanced Training Dataset: The model takes advantage of a newly curated, balanced dataset for training, relied upon to increase robustness across various image scene reconstructions. This improves results over the traditional Train91 dataset, which lacks sufficient scene diversity.

Numerical Results and Analysis

ISTA-Net$^{++}$ demonstrates significant improvements in numerical metrics over baseline models in several benchmark datasets, such as BSD68 and Set11. It consistently surpasses ISTA-Net$^+$ and other state-of-the-art methods across multiple CS ratios, as evidenced by the provided PSNR values. The model's flexibility to manage multiple ratios without compromising on computational efficiency is a salient feature, saving resource allocation in practical applications.

Implications and Future Directions

• Practical Applications: The integration of a singular model capable of processing multi-ratio tasks promises enhanced feasibility in real-world applications, such as efficient storage and reduced complexity in deployment scenarios involving diverse CS requirements.
• Theoretical Contributions: ISTA-Net$$^{++} synthesizes model-based approaches with data-driven methods, demonstrating deep unfolding as a viable mechanism for balancing flexibility with high-performance reconstruction. This alignment with algorithmic rigor expands the theoretical discourse on inverse problem-solving in deep learning.</li> <li><strong>Speculation on AI Developments:</strong> Looking forward, methodologies akin to ISTA-Net$$^{++}$$may be vital in areas requiring <a href="https://www.emergentmind.com/topics/adaptive-learning-strategies" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">adaptive learning strategies</a> under constrained environments. This points to wider utility in domains like <a href="https://www.emergentmind.com/topics/sparse-signal-recovery" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">sparse signal recovery</a> and computational imaging, fostering innovation in dynamic modulation techniques.</li> </ul> <p>ISTA-Net$$^{++}$$stands out in its contribution to the evolution of CS technologies, setting good precedence for future research endeavors in advancing high-performance yet adaptable deep learning models. Researchers may focus on further exploration of dynamic conditions and cross-block processing to expand use cases and improve versatility. The findings establish ISTA-Net$$^{++}$ as a prominent baseline for upcoming compressive sensing challenges and innovations.