From Denoising to Compressed Sensing (1406.4175v5)

Abstract: A denoising algorithm seeks to remove noise, errors, or perturbations from a signal. Extensive research has been devoted to this arena over the last several decades, and as a result, today's denoisers can effectively remove large amounts of additive white Gaussian noise. A compressed sensing (CS) reconstruction algorithm seeks to recover a structured signal acquired using a small number of randomized measurements. Typical CS reconstruction algorithms can be cast as iteratively estimating a signal from a perturbed observation. This paper answers a natural question: How can one effectively employ a generic denoiser in a CS reconstruction algorithm? In response, we develop an extension of the approximate message passing (AMP) framework, called Denoising-based AMP (D-AMP), that can integrate a wide class of denoisers within its iterations. We demonstrate that, when used with a high performance denoiser for natural images, D-AMP offers state-of-the-art CS recovery performance while operating tens of times faster than competing methods. We explain the exceptional performance of D-AMP by analyzing some of its theoretical features. A key element in D-AMP is the use of an appropriate Onsager correction term in its iterations, which coerces the signal perturbation at each iteration to be very close to the white Gaussian noise that denoisers are typically designed to remove.

• The paper presents D-AMP, an extension of AMP that embeds sophisticated denoising algorithms to enhance CS reconstruction accuracy and speed.
• The methodology leverages a deterministic state evolution framework to accurately predict the mean square error and maintain white Gaussian noise conditions.
• Empirical results demonstrate that high-performance denoisers like BM3D significantly reduce computation time while delivering superior reconstruction quality.

From Denoising to Compressed Sensing: An Overview

This paper presents an innovative approach to integrating denoising algorithms within compressed sensing (CS) frameworks, notably through a framework named Denoising-based Approximate Message Passing (D-AMP). The authors, Metzler, Maleki, and Baraniuk, propose extending the well-known approximate message passing (AMP) algorithms to incorporate sophisticated denoising methods, thus bridging the gap between denoising techniques and CS reconstruction.

Theoretical Insights and Methodology

The paper begins by acknowledging the challenges faced by traditional CS techniques in reconstructing high-dimensional signals from limited measurements. These challenges arise due to the inherent under-determinacy of problems, exacerbated in settings involving non-sparse signals, such as natural images.

The cornerstone of their methodology is the AMP framework, enhanced by integrating a wide range of denoisers. This extended AMP, termed D-AMP, introduces the concept of the Onsager correction term, which ensures the effective noise behaves as white Gaussian noise—an assumption that denoisers exploit effectively.

Numerical Evidence and Comparisons

A significant contribution of this work is the empirical validation of D-AMP's superior performance compared to traditional CS methods. When employing high-performance denoisers like BM3D, D-AMP not only outperforms existing algorithms but also operates substantially faster, demonstrating an order of magnitude reduction in computation time.

Theoretical Implications

The paper explores the theoretical underpinnings of D-AMP, presenting a deterministic state evolution framework that predicts the mean square error (MSE) of the algorithm accurately. This framework offers insights into the algorithm's convergence and noise sensitivity, marking a notable departure from traditional stochastic approaches.

Denoising Algorithms

The adaptability of D-AMP is examined via several denoising methods, including Non-Local Means, Bilateral Filters, and BM3D. The results indicate that better-performing denoisers lead directly to better CS reconstruction outcomes, aligning with theoretical expectations.

Impact and Future Directions

This integration of denoising within a CS framework opens avenues for enhanced signal recovery, particularly in imaging applications where non-sparse representations are prevalent. The proposed D-AMP framework showcases robustness to noise and efficient parameter tuning, promising broad applicability across diverse domains.

Given the empirical success and theoretical rigor of the paper, future developments likely will explore further refinements of the state evolution framework and adaptations to non-Gaussian measurement matrices. Additionally, extending the framework to other domains beyond imaging holds potential for expanding the impact of D-AMP.

In summary, this paper provides a comprehensive approach to leveraging advanced denoising techniques for CS problems, promising significant advancements in signal processing and related fields. The integration of these denoisers marks a meaningful evolution in the application of AMP algorithms, translating sophisticated noise removal capabilities directly into more effective CS reconstructions.