Plug-and-Play Image Restoration with Deep Denoiser Prior (2008.13751v2)

Abstract: Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems. Such a property induces considerable advantages for plug-and-play image restoration (e.g., integrating the flexibility of model-based method and effectiveness of learning-based methods) when the denoiser is discriminatively learned via deep convolutional neural network (CNN) with large modeling capacity. However, while deeper and larger CNN models are rapidly gaining popularity, existing plug-and-play image restoration hinders its performance due to the lack of suitable denoiser prior. In order to push the limits of plug-and-play image restoration, we set up a benchmark deep denoiser prior by training a highly flexible and effective CNN denoiser. We then plug the deep denoiser prior as a modular part into a half quadratic splitting based iterative algorithm to solve various image restoration problems. We, meanwhile, provide a thorough analysis of parameter setting, intermediate results and empirical convergence to better understand the working mechanism. Experimental results on three representative image restoration tasks, including deblurring, super-resolution and demosaicing, demonstrate that the proposed plug-and-play image restoration with deep denoiser prior not only significantly outperforms other state-of-the-art model-based methods but also achieves competitive or even superior performance against state-of-the-art learning-based methods. The source code is available at https://github.com/cszn/DPIR.

• The paper presents a novel image restoration method that integrates a deep CNN denoiser as a learned prior within the HQS optimization framework.
• The proposed approach efficiently marries model-based and learning-based techniques, enabling fast convergence and significant PSNR improvements in tasks like deblurring and super-resolution.
• The study demonstrates that a single robust CNN denoiser handles diverse noise levels, paving the way for adaptable and effective image restoration solutions.

An Overview of "Plug-and-Play Image Restoration with Deep Denoiser Prior"

The paper, "Plug-and-Play Image Restoration with Deep Denoiser Prior," presents an innovative approach to image restoration (IR) by leveraging a deep convolutional neural network (CNN) denoiser as a flexible prior within a plug-and-play framework. Traditional IR problems, characterized as ill-posed inverse problems, require prior information to provide regularization. This research extends conventional methods by utilizing a denoiser to implicitly define the image prior, thereby enabling the integration of both model-based and learning-based techniques in solving various IR tasks.

Background and Methodology

The authors address the interplay between model-based methods, which solve IR problems through direct optimization, and learning-based methods that use optimization to train models from data. They highlight the rigidity of these traditional methods—model-based methods offer flexibility across tasks but can be computationally intensive, while learning-based methods must be trained for specific tasks but offer rapid execution during testing.

To bridge these paradigms, the paper proposes a deep plug-and-play framework using a CNN denoiser. Implemented via Half Quadratic Splitting (HQS) algorithm, the system decouples the data fidelity and prior terms, employing the CNN-based denoiser in an iterative scheme. This method effectively handles diverse IR tasks, such as deblurring, super-resolution, and demosaicing, by iteratively refining solutions only distinguishable via noise variations introduced in each step.

Key Contributions and Experimental Results

1. Development of a Deep CNN Denoiser: The authors present a powerful CNN model that handles a wide range of noise levels with a single architecture. This capability contrasts with earlier models that trained multiple denoisers across different noise scenarios. The proposed model is robust, incorporating components from successful architectures like the ResNet and U-Net.
2. Integration with HQS: The approach embeds the denoiser within HQS, offering a hybrid solution balancing computational efficiency and task flexibility. This integration ensures fast convergence to optimal solutions, effectively addressing the degradation constraints posed by noise, blur, or downsampling.
3. Performance Evaluation: Extensive experiments illustrate the proposed method's superiority over existing state-of-the-art models, including IRCNN and traditional methods like BM3D. Across tasks, DPIR not only demonstrates significant PSNR improvements but also retains competitive performance with deep models tailored specifically for single tasks.

The experimental setup emphasizes the method's adaptability through superior results on synthetic benchmarks and real-world data, achieving PSNR gains exceeding those of traditional and some modern models. Notably, the paper reveals the intricate dependence of restoration quality on meticulous parameter selection within the HQS framework.

Implications and Future Directions

This research underscores the potential of incorporating deep learning within iterative optimization frameworks to flexibly and effectively address IR challenges. By minimizing coupling between data and prior terms using trained denoisers, new pathways open for efficient, adaptable restoration techniques applicable across various domains without retraining.

The theoretical significance lies in demonstrating how deep networks can serve as generalized priors, potentially influencing future developments in unsupervised or semi-supervised learning for IR tasks. Moreover, practical implications include deploying these systems in environments where diverse degradation scenarios are prevalent.

Looking forward, further exploration into integrating diverse deep prior architectures or refining convergence properties could enhance adaptability and performance. The paper also highlights the need to investigate deeper theoretical convergence guarantees within broader plug-and-play frameworks as complexity scales.

In conclusion, this paper exemplifies substantial progress in combining model-based versatility with deep learning effectiveness, providing a robust methodology to tackle an array of challenging IR problems.