Accelerated Alternating Projections for Robust Principal Component Analysis (1711.05519v4)

Abstract: We study robust PCA for the fully observed setting, which is about separating a low rank matrix $\boldsymbol{L}$ and a sparse matrix $\boldsymbol{S}$ from their sum $\boldsymbol{D}=\boldsymbol{L}+\boldsymbol{S}$. In this paper, a new algorithm, dubbed accelerated alternating projections, is introduced for robust PCA which significantly improves the computational efficiency of the existing alternating projections proposed in [Netrapalli, Praneeth, et al., 2014] when updating the low rank factor. The acceleration is achieved by first projecting a matrix onto some low dimensional subspace before obtaining a new estimate of the low rank matrix via truncated SVD. Exact recovery guarantee has been established which shows linear convergence of the proposed algorithm. Empirical performance evaluations establish the advantage of our algorithm over other state-of-the-art algorithms for robust PCA.

• The paper presents the AccAltProj algorithm that accelerates robust PCA by projecting differences onto low-rank tangent spaces, reducing SVD complexity.
• It establishes exact recovery with linear convergence under incoherence and sparsity conditions, supported by strong theoretical guarantees.
• Comparative experiments show AccAltProj outperforms traditional methods in speed and scalability for applications like video background subtraction.

Accelerated Alternating Projections for Robust Principal Component Analysis

The paper "Accelerated Alternating Projections for Robust Principal Component Analysis" introduces a novel algorithm, termed Accelerated Alternating Projections (AccAltProj), aimed at improving computational efficiency in solving the robust principal component analysis (RPCA). The RPCA problem involves separating a low-rank matrix and a sparse matrix from their sum, which is a critical task in numerous applications such as video and voice background subtraction, graph clustering, and fault isolation.

Overview and Algorithmic Contributions

The proposed AccAltProj algorithm centers around enhancing the existing Alternating Projections (AltProj) method by introducing a subspace projection step to reduce the computational demand of truncated Singular Value Decompositions (SVD). Specifically, instead of performing a full SVD on matrix differences directly, the algorithm projects the difference onto a tangent space of low rank matrices defined by the trimmed estimates from previous iterations. This approach significantly lowers per-iteration complexity from $O(r^2n^2)$ to $O(rn^2)$, while preserving the linear convergence rate.

AccAltProj encompasses two main phases:

1. Initialization through two steps of AltProj: It ensures a good starting point within the basin of attraction for local convergence.
2. Iterative updates: Each iteration updates the low-rank matrix estimate by projecting onto a tangent space followed by an SVD truncation. The sparse matrix estimate is updated via hard thresholding based on singular value-related parameters.

Theoretical Results and Assumptions

The authors demonstrate that AccAltProj achieves exact recovery with linear convergence, under certain conditions involving the incoherence of the low-rank matrix and sparsity of the sparse matrix. Theorem 1 establishes local convergence provided the initial estimates are close to the ground truth, and the sparsity ratio of non-zero entries in the sparse matrix does not exceed a specified threshold. Theorem 2 assures that the initial estimates derived from the initialization phase are within these bounds.

Comparative Analysis

In comparison with existing RPCA methods such as AltProj and a gradient descent approach, AccAltProj is shown empirically to be faster and capable of handling larger matrix sizes. The numerical experiments demonstrate AccAltProj's robustness and computational efficiency across synthetic data and real-world video background subtraction scenarios.

Potential Impact and Future Directions

The proposed algorithm effectively addresses computational bottlenecks in large-scale RPCA tasks, promising practical utility in areas requiring real-time matrix decomposition. Despite conservative theoretical guarantees on the sparsity threshold, empirical results suggest broader applicability, prompting future exploration for tightening theoretical bounds. Other areas for future work include extending AccAltProj to handle noisy data and adapting the approach for partially observed settings.

In summary, the AccAltProj algorithm represents an advancement in efficient RPCA computation, unlocking potential for deployment in high-dimensional data environments where timely processing and robustness to outliers are paramount.