Unrolled Optimization with Deep Priors (1705.08041v2)

Abstract: A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model. Traditionally, hand-crafted priors along with iterative optimization methods have been used to solve such problems. In this paper we present unrolled optimization with deep priors, a principled framework for infusing knowledge of the image formation into deep networks that solve inverse problems in imaging, inspired by classical iterative methods. We show that instances of the framework outperform the state-of-the-art by a substantial margin for a wide variety of imaging problems, such as denoising, deblurring, and compressed sensing magnetic resonance imaging (MRI). Moreover, we conduct experiments that explain how the framework is best used and why it outperforms previous methods.

• The paper introduces a novel ODP framework that combines iterative optimization with deep networks to enhance image reconstruction.
• It demonstrates that the ODP approach outperforms classic techniques in tasks such as denoising, deblurring, and compressed sensing MRI.
• The study highlights the value of embedding image formation models into deep architectures for more efficient and robust inverse imaging.

Unrolled Optimization with Deep Priors

The paper "Unrolled Optimization with Deep Priors" introduces a novel framework for addressing inverse problems in imaging. These problems commonly arise in computational imaging, sensing, and low-level computer vision, and involve reconstructing latent images from measurements taken under a known image formation model. Traditional methods often utilize hand-crafted priors and iterative optimization techniques. However, the paper argues that a more systematic integration of image formation knowledge into deep networks could improve performance significantly.

Key Contributions

The paper proposes the use of Unrolled Optimization with Deep Priors (ODP), a principled framework that combines the strengths of classical iterative optimization methods with the learning capabilities of deep convolutional networks. This approach inherently incorporates prior information of the image formation model into the network architecture. The unrolling of optimization algorithms involves truncating traditional iterative methods and treating them as deep networks, allowing specific design choices to tailor network architectures.

Key contributions include:

1. Introduction of the ODP framework for incorporating prior knowledge into deep networks for inverse imaging problems.
2. Demonstration that ODP instances outperform state-of-the-art techniques across a variety of imaging tasks such as denoising, deblurring, and compressed sensing MRI.
3. Empirical insights into optimal use of prior information and suitable algorithms for unrolled optimization.

Experimental Insights

The experimental results reveal that ODP networks achieve superior performance compared to established methods in denoising, deblurring, and CS MRI tasks. Specifically, for denoising, the network exhibited higher PSNR values compared to methods such as BM3D and EPLL. In deblurring tasks, ODP networks generalized effectively across multiple image formation models, achieving robust results even when trained on diverse blur kernels. For CS MRI, the ODP framework demonstrated significant improvements, particularly in challenging scenarios with sparse sampling patterns, indicating the framework's capability to generalize across different sampling patterns without necessitating specialized models for each.

The ablation studies highlight that the framework combining CNN priors with data steps gives superior performance compared to pure residual networks. These studies suggest that incorporating the image formation model and iteratively optimizing the reconstruction task is crucial, particularly when the inversion step of the formation model is complex.

Furthermore, the choice between different optimization algorithms like proximal gradient, ADMM, and gradient descent when unrolled shows that algorithms approximating inverse steps each iteration perform best. The use of Lagrange multipliers showed marginal benefits, indicating that simpler primal algorithms are effective for unrolled optimization.

Implications and Future Directions

The implications of this research are profound both practically and theoretically. While the paper is centered on imaging, the framework could extend to other domains requiring integration of model priors into deep learning architectures, such as control systems where physical dynamics could be utilized. It opens pathways to explore blind inverse problems where image formation models aren't fully known and nonlinear imaging models, expanding applicability of deep learning in scientific realms.

The insights derived from the paper could inform future efforts to integrate deep networks more tightly with specialized knowledge domains, leveraging the predictive power of deep learning along with traditional model-based understanding.

In conclusion, the paper lays a solid foundation for future exploration of unrolled optimization within various fields in artificial intelligence and engineering, promising enhancements in efficiency and performance by systematically embedding prior knowledge within deep learning frameworks.