Correlation Adaptive Subspace Segmentation by Trace Lasso (1501.04276v1)

Abstract: This paper studies the subspace segmentation problem. Given a set of data points drawn from a union of subspaces, the goal is to partition them into their underlying subspaces they were drawn from. The spectral clustering method is used as the framework. It requires to find an affinity matrix which is close to block diagonal, with nonzero entries corresponding to the data point pairs from the same subspace. In this work, we argue that both sparsity and the grouping effect are important for subspace segmentation. A sparse affinity matrix tends to be block diagonal, with less connections between data points from different subspaces. The grouping effect ensures that the highly corrected data which are usually from the same subspace can be grouped together. Sparse Subspace Clustering (SSC), by using $\ell^1$-minimization, encourages sparsity for data selection, but it lacks of the grouping effect. On the contrary, Low-Rank Representation (LRR), by rank minimization, and Least Squares Regression (LSR), by $\ell^2$-regularization, exhibit strong grouping effect, but they are short in subset selection. Thus the obtained affinity matrix is usually very sparse by SSC, yet very dense by LRR and LSR. In this work, we propose the Correlation Adaptive Subspace Segmentation (CASS) method by using trace Lasso. CASS is a data correlation dependent method which simultaneously performs automatic data selection and groups correlated data together. It can be regarded as a method which adaptively balances SSC and LSR. Both theoretical and experimental results show the effectiveness of CASS.

• The paper presents a novel CASS method that uses trace Lasso to dynamically balance sparsity and grouping effects for adaptive subspace segmentation.
• It establishes rigorous theoretical conditions, including Enforced Block Sparse (EBS), to ensure block sparse solutions when data are drawn from independent subspaces.
• Experimental results on datasets like Hopkins 155 and Extended Yale B verify that CASS outperforms traditional methods in segmentation accuracy and affinity matrix approximation.

Correlation Adaptive Subspace Segmentation by Trace Lasso

The paper "Correlation Adaptive Subspace Segmentation by Trace Lasso," authored by Canyi Lu, Jiashi Feng, Zhouchen Lin, and Shuicheng Yan, presents a novel approach to subspace segmentation, the Correlation Adaptive Subspace Segmentation (CASS) method. This approach addresses the problem of segmenting a set of data points into clusters, where each cluster corresponds to a subspace in which the data points approximately lie. This is a critical task in computer vision and machine learning with numerous applications, such as motion segmentation and face clustering.

Key Concepts and Proposed Method

CASS is a method designed to address issues inherent in existing subspace segmentation techniques, such as Sparse Subspace Clustering (SSC), Low-Rank Representation (LRR), and Least Squares Regression (LSR). These existing methods exhibit either strong sparsity or grouping effect, but not both. SSC, utilizing $\ell^1$-minimization, emphasizes sparsity but lacks in adequately grouping correlated data, whereas LRR and LSR, through rank minimization and $\ell^2$-regularization respectively, excel in grouping but fall short in achieving sparsity.

CASS introduces trace Lasso as a novel regularization approach that adapts to data correlation. It adeptly balances the sparse solution quality of SSC and the grouping ability of LSR by interpolating between the $\ell^1$-norm and the $\ell^2$-norm based on data correlation. When data are highly correlated, trace Lasso tends towards $\ell^2$-norm, promoting grouping, and towards $\ell^1$-norm for less correlated data, promoting sparsity.

Theoretical Contributions

The theoretical underpinning of CASS is solidified through several contributions:

1. Enforced Block Sparse (EBS) Conditions: The paper extends the concept of Enforced Block Diagonal (EBD) conditions to EBS conditions, proving that if the function governing the representation solution satisfies these conditions, a block sparse solution is attainable. Trace Lasso is rigorously shown to satisfy these conditions.
2. Grouping Effect: The paper formally substantiates the grouping effect of trace Lasso, demonstrating that it can equivalently represent a group of highly correlated data points, a desirable property not inherently addressed by SSC.

A significant theoretical result demonstrated in the paper is the block sparse solution of CASS when data are drawn from independent subspaces, akin to the results of SSC, LRR, and LSR when similar conditions are met.

Experimental Validation

The efficacy of CASS is validated through experiments on well-known datasets, including the Hopkins 155 motion database, the Extended Yale B face database, and the MNIST handwritten digits database. The experiments distinctly show that CASS achieves superior segmentation accuracy, especially in complex scenarios with higher numbers of subjects or clusters.

The results established that CASS produces an affinity matrix that is closer to the ideal block diagonal structure compared to SSC, LRR, and LSR. Furthermore, semi-supervised learning applications using affinity matrices produced by CASS demonstrated improved performance, corroborating its utility beyond pure segmentation tasks.

Implications and Future Work

The introduction of trace Lasso in the context of subspace segmentation sets a precedent for future work concerning the adaptability of regularizers based on data properties. CASS's methodological contributions propose a promising direction for subspace learning, particularly in scenarios with noisy or highly correlated data.

Future work could explore the learning of compact and discriminative dictionaries for subspace representation to enhance CASS's scalability and broad applicability. Moreover, the extension of trace Lasso to other domains like classification and dimensionality reduction presents an intriguing prospect. Developing efficient optimization strategies for large-scale subspace segmentation will also be crucial for practical, real-world applications. Therefore, further exploration into these areas could significantly enhance the current understanding and application range of CASS.