Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset (1511.01245v3)

Abstract: Recent research on problem formulations based on decomposition into low-rank plus sparse matrices shows a suitable framework to separate moving objects from the background. The most representative problem formulation is the Robust Principal Component Analysis (RPCA) solved via Principal Component Pursuit (PCP) which decomposes a data matrix in a low-rank matrix and a sparse matrix. However, similar robust implicit or explicit decompositions can be made in the following problem formulations: Robust Non-negative Matrix Factorization (RNMF), Robust Matrix Completion (RMC), Robust Subspace Recovery (RSR), Robust Subspace Tracking (RST) and Robust Low-Rank Minimization (RLRM). The main goal of these similar problem formulations is to obtain explicitly or implicitly a decomposition into low-rank matrix plus additive matrices. In this context, this work aims to initiate a rigorous and comprehensive review of the similar problem formulations in robust subspace learning and tracking based on decomposition into low-rank plus additive matrices for testing and ranking existing algorithms for background/foreground separation. For this, we first provide a preliminary review of the recent developments in the different problem formulations which allows us to define a unified view that we called Decomposition into Low-rank plus Additive Matrices (DLAM). Then, we examine carefully each method in each robust subspace learning/tracking frameworks with their decomposition, their loss functions, their optimization problem and their solvers. Furthermore, we investigate if incremental algorithms and real-time implementations can be achieved for background/foreground separation. Finally, experimental results on a large-scale dataset called Background Models Challenge (BMC 2012) show the comparative performance of 32 different robust subspace learning/tracking methods.

• The paper introduces a comprehensive evaluation of low-rank and additive matrix decompositions for segregating static backgrounds from dynamic foregrounds in videos.
• It details several robust methods, including RPCA, RNMF, and RMC, and compares the performance of 32 algorithms using extensive benchmarking.
• The findings reveal the potential for scalable, noise-robust, real-time systems and point to future research directions in computer vision surveillance.

An Overview of Background/Foreground Separation via Low-rank plus Additive Matrices

The paper "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset" by Bouwmans et al. provides a comprehensive examination of various methods for background and foreground separation in video surveillance. This is achieved through the decomposition of data into low-rank and additive components, a cornerstone methodology in robust computer vision applications.

Key Contributions and Insights

The focus of the paper is on a technique that involves the separation of video frames into static (background) and dynamic (foreground) components using low-rank and sparse matrices. This approach is especially relevant in video surveillance where identifying moving objects is crucial. The authors categorize the methodologies into several formulations including Robust Principal Component Analysis (RPCA), Robust Non-negative Matrix Factorization (RNMF), Robust Matrix Completion (RMC), and others. Each formulation presents a unique decomposition strategy, loss function, and solver approach.

Methodologies and Formulations

1. Robust Principal Component Analysis (RPCA): This method involves Principal Component Pursuit (PCP) where the data matrix is decomposed into a low-rank matrix representing the background and a sparse matrix representing the moving objects. Variants such as Stable PCP (SPCP) incorporate robust measures against noise.
2. Robust Non-negative Matrix Factorization (RNMF): This method approximates the non-negative data matrix using two non-negative factor matrices. This approach is particularly useful when the noise distribution in the data is not Gaussian.
3. Robust Matrix Completion (RMC): This technique focuses on completing a low-rank matrix from partial observations, important in applications where some data points are missing or corrupted.
4. Subspace Tracking and Recovery: These methods aim for real-time processing of video data streams where the foreground-background separation must be updated incrementally.

Each of these methodologies has been evaluated using a wide array of numerical experiments and datasets, including the extensive Background Models Challenge (BMC 2012) dataset, which provides a benchmark for such evaluations.

Numerical Results and Claims

The paper presents comprehensive numerical evaluations comparing 32 different algorithms across the various robust subspace learning frameworks. The results underscore the competitive performance of these approaches, particularly in their ability to deal with dynamic and real-world video sequences.

Implications and Future Directions

The paper highlights that while many approaches deliver strong performance in controlled experiments, challenges remain in achieving consistent results across diverse real-world scenarios. The need for scalable, real-time, and noise-robust algorithms is emphasized, and the authors suggest that future research could focus on improving incremental algorithms and real-time implementations. Moreover, these methods could be enhanced by integrating advanced machine learning frameworks, potentially offering greater adaptability to complex scenarios encountered in video surveillance.

Conclusion

In summarizing the extensive coverage of low-rank plus additive matrix decomposition methods, this paper serves as both a foundational text and a forward-looking review. It sets the stage for future research aimed at overcoming existing challenges, while simultaneously providing a structured evaluation framework to guide such endeavors. Researchers and practitioners in computer vision, particularly those involved in video surveillance, will find this paper a valuable resource for advancing their understanding of robust background and foreground separation techniques.