Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser (2007.13640v3)

Abstract: Prior probability models are a fundamental component of many image processing problems, but density estimation is notoriously difficult for high-dimensional signals such as photographic images. Deep neural networks have provided state-of-the-art solutions for problems such as denoising, which implicitly rely on a prior probability model of natural images. Here, we develop a robust and general methodology for making use of this implicit prior. We rely on a statistical result due to Miyasawa (1961), who showed that the least-squares solution for removing additive Gaussian noise can be written directly in terms of the gradient of the log of the noisy signal density. We use this fact to develop a stochastic coarse-to-fine gradient ascent procedure for drawing high-probability samples from the implicit prior embedded within a CNN trained to perform blind (i.e., with unknown noise level) least-squares denoising. A generalization of this algorithm to constrained sampling provides a method for using the implicit prior to solve any linear inverse problem, with no additional training. We demonstrate this general form of transfer learning in multiple applications, using the same algorithm to produce state-of-the-art levels of unsupervised performance for deblurring, super-resolution, inpainting, and compressive sensing.

• The paper introduces a novel method that leverages implicit priors from pre-trained CNN denoisers to solve linear inverse problems without extra training.
• It employs a coarse-to-fine stochastic gradient ascent approach that iteratively refines images by balancing noise reduction with structural fidelity.
• The approach generalizes across applications such as inpainting, super-resolution, and compressive sensing, often outperforming traditional parametric models.

Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser

The paper "Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser" by Kadkhodaie and Simoncelli focuses on leveraging the implicit prior embedded within convolutional neural networks (CNNs) trained for denoising tasks to solve linear inverse problems in image processing, without additional training. Recognizing the complexity of explicitly modeling the full probability density of high-dimensional image spaces, this paper harnesses the sophisticated implicit priors from CNNs, which often outperform traditional parametric models.

Key to this approach is a classical statistic result from Miyasawa (1961), connecting the optimal denoiser's residual to the gradient of the log-likelihood of the observed noisy data. This insight allows for a stochastic gradient ascent method that generates high-probability samples from the implicit prior of an image denoiser. The algorithm adapts efficiently across multiple applications, including deblurring, super-resolution, inpainting, and compressive sensing, achieving unsupervised state-of-the-art levels.

Methodology

The paper develops a coarse-to-fine stochastic gradient ascent approach leveraging the denoiser's residual to estimate the likelihood's gradient. This method starts from a random initialization and iteratively improves the image by balancing the reduction of noise and the retention of structure, ensuring convergence towards high-probability samples consistent with the learned image manifold.

This strategy is further generalized to incorporate constraints from linear measurements, thus solving arbitrary linear inverse problems. The method partitions the gradient into two components—one aligning with data fidelity constraints and the other adjusting towards high-likelihood areas defined by the implicit prior.

Applications

The methodology is validated across various image restoration tasks:

1. Inpainting: The algorithm effectively fills in missing image sections by extrapolating structure from the surrounding context.
2. Super-resolution: It reconstructs high-resolution images from lower-resolution inputs, outperforming comparable techniques in terms of perceptual quality, albeit with slightly lower PSNR and SSIM scores.
3. Deblurring: The method restores images blurred by retaining only low-frequency components, producing results that maintain high edge definition.
4. Compressive Sensing: Capitalizing on a denoiser's manifold prior, the approach surpasses traditional sparse coding techniques, yielding high-quality reconstructions from significantly compressed measurements.

Implications and Future Directions

By effectively utilizing the implicit priors of pre-trained CNN denoisers, this research proposes a paradigm where task-agnostic neural networks extend their utility to broader inverse problem contexts. This capability could inspire more efficient imaging systems in medical imaging, satellite image reconstruction, and other domains requiring high-fidelity image recovery from corrupted or partial data inputs.

Looking forward, the work prompts inquiries into further refining this approach by integrating more advanced CNN architectures or extending the methodology to tackle non-linear or more complex inverse problem scenarios, potentially making impactful strides in both theoretical AI research and practical image processing applications. The adaptability and stochastic nature of this algorithm also present potential for examining its performance in dynamic or real-time settings where conditions change rapidly, such as video data processing.