Subspace Pursuit for Compressive Sensing Signal Reconstruction (0803.0811v3)

Abstract: We propose a new method for reconstruction of sparse signals with and without noisy perturbations, termed the subspace pursuit algorithm. The algorithm has two important characteristics: low computational complexity, comparable to that of orthogonal matching pursuit techniques when applied to very sparse signals, and reconstruction accuracy of the same order as that of LP optimization methods. The presented analysis shows that in the noiseless setting, the proposed algorithm can exactly reconstruct arbitrary sparse signals provided that the sensing matrix satisfies the restricted isometry property with a constant parameter. In the noisy setting and in the case that the signal is not exactly sparse, it can be shown that the mean squared error of the reconstruction is upper bounded by constant multiples of the measurement and signal perturbation energies.

• The paper introduces an innovative iterative algorithm that balances LP accuracy with the speed of greedy methods for sparse signal recovery.
• It provides rigorous theoretical guarantees for exact recovery under RIP conditions in both noiseless and noisy measurement scenarios.
• The algorithm achieves lower computational complexity than traditional LP approaches, making it practical for real-world signal processing applications.

Subspace Pursuit for Compressive Sensing Signal Reconstruction

The paper "Subspace Pursuit for Compressive Sensing Signal Reconstruction" by Wei Dai and Olgica Milenkovic presents a novel iterative algorithm, termed the Subspace Pursuit (SP) algorithm, for reconstructing sparse signals from a limited number of linear measurements, a key problem in Compressive Sensing (CS). The algorithm leverages the Restricted Isometry Property (RIP) and aims to offer performance on par with Linear Programming (LP) methods but with significantly lower computational complexity.

Core Contributions

The primary contributions of the paper are:

1. Algorithm Design: The introduction of the SP algorithm, which aims to balance the accuracy of LP methods with the speed of greedy algorithms like Orthogonal Matching Pursuit (OMP).
2. Theoretical Analysis:
   • Noiseless Case: A rigorous proof that the SP algorithm can exactly recover K-sparse signals from noiseless measurements provided the sampling matrix satisfies RIP with a suitably small constant.
   • Noisy Case: An extension of results to show that even in the presence of noise, the reconstruction error remains bounded.
3. Complexity Analysis: Theoretical bounds on the computational complexity and the number of iterations needed for the SP algorithm to converge.

Algorithmic Framework

The SP algorithm is inspired by concepts from coding theory, particularly the A∗ order-statistic decoding algorithm for additive white Gaussian noise channels. The procedure involves iterative estimation and refinement of the support set of the sparse signal. Each iteration consists of:

1. Initialization: Selection of an initial support set based on the largest correlations between the measurement vector and columns of the sampling matrix.
2. Iteration Steps:
   • Selection of additional candidate indices beyond the current estimate.
   • Projection onto the span of the selected support.
   • Refinement of the support set by retaining the most reliable indices.

The algorithm terminates when the reconstruction error stops improving significantly or when a predefined number of iterations is reached.

Theoretical Insights

Noiseless Reconstruction

The paper establishes that the SP algorithm can exactly recover any K-sparse signal as long as the sampling matrix Φ satisfies the RIP with $\delta_{3K} < 0.165$. The proof hinges on demonstrating that in each iteration, the algorithm incrementally refines the support set such that the error in estimating the residual vector consistently decreases.

Recovery from Noisy Measurements

In scenarios where measurements are contaminated with noise, the reconstruction error is controlled by the energy of the noise. Specifically, the paper shows that for any K-sparse signal $x$ and measurement $y = Φx + e$ with noise vector $e$, the reconstruction distortion satisfies:

$\|x - \hat{x}\|_2 \leq c_K \|e\|_2$

where $c_K$ is a constant dependent on $\delta_{3K}$.

Approximately Sparse Signals

For signals that are not exactly sparse but can be approximated by a sparse signal, the paper establishes bounds on the reconstruction error both in terms of the noise and the approximation error of the signal. If $x$ is approximately K-sparse, the reconstruction distortion can be bounded by:

$$\|x - \hat{x}\|_2 \leq c_{2K} \left( \|x - x_{2K}\|_1 + \|e\|_2 \right)$$

provided that the RIP constant $\delta_{6K} < 0.083$.

Computational Complexity

The SP algorithm excels in computational efficiency. For sparse signals where $K \leq O(N)$, the complexity of one iteration is $O(mNK)$, and the overall complexity for compressible signals with coefficients decaying slowly can be reduced to $O(mN \log K)$. This makes the SP algorithm significantly faster compared to LP methods, which have a complexity of $O(m^2 N^{3/2})$ using interior-point methods.

Empirical Evaluation

Numerical simulations demonstrate the empirical performance of the SP algorithm. It consistently outperforms OMP and ROMP in terms of the sparsity level at which exact recovery is guaranteed. For highly sparse signals, the SP algorithm's reconstruction capabilities closely match those of LP methods but with much lower computational overhead.

Practical Implications and Future Directions

The SP algorithm offers a feasible and efficient solution for real-world applications involving sparse signal reconstruction, such as medical imaging, sensor networks, and telecommunications, where both accuracy and speed are paramount. Future research could explore adaptive variants of the SP algorithm, better strategies for handling highly noisy measurements, and extensions to structured sparsity models.

Conclusion

The Subspace Pursuit algorithm represents a significant development in the domain of compressive sensing, providing a robust, theoretically sound, and computationally efficient method for sparse signal reconstruction. The combination of provable guarantees and practical efficiency ensures its relevance for a wide range of applications where rapid and reliable signal recovery is essential.