Structured Sparse Subspace Clustering: A Joint Affinity Learning and Subspace Clustering Framework (1610.05211v2)

Abstract: Subspace clustering refers to the problem of segmenting data drawn from a union of subspaces. State-of-the-art approaches for solving this problem follow a two-stage approach. In the first step, an affinity matrix is learned from the data using sparse or low-rank minimization techniques. In the second step, the segmentation is found by applying spectral clustering to this affinity. While this approach has led to state-of-the-art results in many applications, it is sub-optimal because it does not exploit the fact that the affinity and the segmentation depend on each other. In this paper, we propose a joint optimization framework --- Structured Sparse Subspace Clustering (S$^3$C) --- for learning both the affinity and the segmentation. The proposed S$^3$C framework is based on expressing each data point as a structured sparse linear combination of all other data points, where the structure is induced by a norm that depends on the unknown segmentation. Moreover, we extend the proposed S$^3$C framework into Constrained Structured Sparse Subspace Clustering (CS$^3$C) in which available partial side-information is incorporated into the stage of learning the affinity. We show that both the structured sparse representation and the segmentation can be found via a combination of an alternating direction method of multipliers with spectral clustering. Experiments on a synthetic data set, the Extended Yale B data set, the Hopkins 155 motion segmentation database, and three cancer data sets demonstrate the effectiveness of our approach.

• The paper presents a joint optimization framework that unifies affinity learning and subspace segmentation to enhance clustering accuracy.
• It employs a subspace structured norm with alternating minimization to robustly capture inter-subspace relationships and handle noise.
• The framework outperforms traditional methods like SSC and LRR across synthetic, face, motion, and gene expression datasets, also incorporating side-information effectively.

Insight into Structured Sparse Subspace Clustering Framework

The paper presents a unified framework for Structured Sparse Subspace Clustering (S), which integrates the processes of affinity learning and data segmentation prevalent in traditional subspace clustering methods. The research focuses on overcoming the inherent disconnect between these two stages in previous methodologies, such as Sparse Subspace Clustering (SSC) and Low-Rank Representation (LRR), by simultaneously optimizing the affinity matrix and the subspace memberships.

Key Contributions and Methodology

1. Joint Optimization Framework: The S framework proposes a simultaneous optimization of data affinity and segmentation, allowing for a feedback loop that iteratively refines both. This contrasts with the two-stage approach where these are handled independently, potentially leading to suboptimal solutions.
2. Subspace Structured Norm: The introduction of a subspace structured norm integrates prior segmentation knowledge into sparse representation, promoting a representation matrix that is also segmentation-aware. The structured sparse representation is obtained using a combination of $\ell_1$ norm with a supplementary penalty dependent on the inter-subspace relationships.
3. Incorporation of Side Information: By extending the framework to Constrained Structured Sparse Subspace Clustering (S), prior knowledge or partial side-information can be seamlessly incorporated into the affinity learning process, yielding improved clustering performance, particularly in data with some known constraints.
4. Alternating Minimization Approach: The framework employs an alternating direction method of multipliers (ADMM) combined with spectral clustering to iteratively optimize the affinity and segmentation. This method captures the interdependencies between affinity and segmentation, attending to noise and outliers robustly.

Experimental Evaluation

Experiments conducted on synthetic datasets, the Extended Yale B face dataset, the Hopkins 155 motion segmentation database, and cancer gene expression datasets validate the effectiveness of the proposed method. Notably, the S framework outperforms existing subspace clustering methods, including SSC and LRR, in terms of clustering accuracy and robustness to noise.

• Synthetic Data: In scenarios with varying degrees of corruption, S consistently achieves lower clustering errors compared to SSC.
• Face and Motion Datasets: On real-world datasets like Extended Yale B and Hopkins 155, structured approaches yield substantial improvements in connectivity (CONN) and subspace-preserving rates (SPR).
• Cancer Gene Clustering: The S framework demonstrates flexibility by incorporating side-information, leading to enhanced accuracy, especially when partial constraints are known.

Theoretical and Practical Implications

Theoretically, the joint framework advances the understanding of segmentation-aware affinity learning, suggesting a path to more nuanced subspace clustering methods that can dynamically adjust based on evolving segmentation insights. Practically, the adaptability to include side constraints broadens its applicability, particularly in bioinformatics and other fields where expert knowledge can guide clustering.

Future Directions

While the paper lays a strong foundation, future work could explore scalability enhancements, particularly for large-scale datasets. Moreover, extending this framework to multiview clustering and semi-supervised learning contexts offers intriguing possibilities. Additionally, theoretical analysis on convergence guarantees and the impact of parameter tuning could further strengthen the framework's application in diverse settings.

In summary, the Structured Sparse Subspace Clustering framework skillfully integrates affinity learning with data segmentation, enabling a symbiotic optimization process that addresses the shortcomings of prior methodologies. This innovative approach marks a significant stride in handling high-dimensional clustering challenges with precision and adaptability.