Bayesian Compressive Sensing via Belief Propagation (0812.4627v2)

Abstract: Compressive sensing (CS) is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable, sub-Nyquist signal acquisition. When a statistical characterization of the signal is available, Bayesian inference can complement conventional CS methods based on linear programming or greedy algorithms. We perform approximate Bayesian inference using belief propagation (BP) decoding, which represents the CS encoding matrix as a graphical model. Fast computation is obtained by reducing the size of the graphical model with sparse encoding matrices. To decode a length-N signal containing K large coefficients, our CS-BP decoding algorithm uses O(Klog(N)) measurements and O(Nlog^2(N)) computation. Finally, although we focus on a two-state mixture Gaussian model, CS-BP is easily adapted to other signal models.

• The paper proposes a Bayesian compressive sensing framework that employs belief propagation for efficient sparse signal reconstruction.
• The authors model the sparse encoding using a two-state mixture Gaussian, achieving fast O(Nlog²(N)) computation and reducing measurements by 20-30%.
• The method leverages sparse graphical models inspired by LDPC codes, highlighting its potential for scalable, low-cost signal processing applications.

Bayesian Compressive Sensing via Belief Propagation

The paper "Bayesian Compressive Sensing via Belief Propagation," authored by Dror Baron, Shriram Sarvotham, and Richard G. Baraniuk, provides a detailed exploration of employing Bayesian methods in compressive sensing (CS) using belief propagation (BP). The authors aim to enhance CS by leveraging statistical characterizations of sparse signals, thereby complementing traditional CS approaches that primarily utilize linear programming or greedy algorithms.

Overview of Compressive Sensing (CS)

Compressive sensing is a technique for acquiring sparse signals with a small number of linear projections. It contrasts traditional methods, which fully sample data before discarding insignificant coefficients. CS theory demonstrates that a sparse signal can be reconstructed from fewer measurements than conventionally required by Nyquist sampling.

Bayesian Approach and Belief Propagation

In scenarios where a statistical characterization of a signal is available, Bayesian inference offers enhanced estimation and reduced measurement requirements. The authors introduce a Bayesian framework using belief propagation (BP) to perform approximate inference. The encoding matrix in CS is depicted as a graphical model, enabling fast computation through the reduction of the graphical model's size, achieved by exploiting sparse encoding matrices.

The authors focus on a two-state mixture Gaussian model, enabling the Bayesian framework to adeptly model approximate sparsity. This approach is characterized by fast computation requirements of $O(N\log^2(N))$ and uses $O(K\log(N))$ measurements, where $N$ is the signal length and $K$ is the number of large coefficients.

Sparse Encoding and Decoding

The encoding matrix, inspired by LDPC codes, is sparse to facilitate rapid measurement processes. This structure accelerates both encoding and decoding, despite some limitations in information capacity compared to dense matrices. The decoding problem is framed via belief propagation algorithms on a bipartite factor graph, enabling efficient inference from sparse models.

Numerical Results and Comparisons

Extensive simulations demonstrated favorable performance of the proposed CS-BP algorithm, particularly in scenarios with noise and model mismatches. CS-BP required approximately 20-30% fewer measurements than traditional linear programming methods to achieve the same mean square error (MSE). Moreover, while its computational speed depends on the signal length, its scalability is more favorable than some contemporary approaches.

Implications and Future Work

The work provides significant implications for both theory and practice, highlighting potential applications in areas requiring efficient signal processing with reduced data acquisition costs. Moreover, the Bayesian framework's adaptability suggests further explorations in applying these concepts within other sparsifying bases, enhancing model robustness, and experimenting with irregular CS-LDPC matrices. The framework could also benefit applications where feedback processes play a role in dynamic measurement environments.

In conclusion, this paper contributes substantially to the intersection of Bayesian methods and compressive sensing. It opens avenues for further research in integrating more complex statistical models, supporting broader applications with structured sparsity, and optimizing performance across a variety of signal processing tasks.