Product of Projections in Multi-View Subspace Clustering

• Product of projections in multi-view subspace clustering is a technique that replaces full affinity matrices with low-rank, landmark-based approximations to capture intrinsic subspace structures.
• SDSNet leverages encoder-decoder architectures and scalable self-expressiveness modules to efficiently process high-dimensional data while preserving subspace integrity.
• Empirical results show that these projection methods significantly reduce computational complexity and memory requirements, enabling rapid clustering of tens of thousands of samples.

Deep Scalable Subspace Clustering (SDSNet) refers to a class of deep learning frameworks that address the scalability bottlenecks inherent in traditional and deep subspace clustering by replacing full affinity or self-expressive matrices with scalable, often low-rank, approximations within end-to-end architectures. Unlike classical approaches that require full-batch optimization of $n \times n$ affinity graphs for $n$ samples, SDSNet designs leverage mini-batch training, factorization schemes, and landmark-based approximations, enabling efficient clustering of tens of thousands of high-dimensional samples while retaining fidelity to subspace structure (Zhang et al., 2018, Mrabah et al., 24 Dec 2025).

1. Problem Formulation and Motivation

The objective of subspace clustering is to partition a dataset $X \in \mathbb{R}^{D \times n}$, whose $n$ columns are presumed to lie close to the union of $k$ (possibly non-linear) subspaces $\{\mathcal{S}_i\}_{i=1}^k$, into groups corresponding to those subspaces. Classical self-expressiveness models, notably Sparse Subspace Clustering (SSC) and Low-Rank Representation (LRR), express each data point as a linear combination of the others:

$X = X C + E,$

with $C \in \mathbb{R}^{n \times n}$ as the coefficient (affinity) matrix and $E$ as the residual. Deep Subspace Clustering Networks (DSC-Net) (Ji et al., 2017) extend this model into a non-linear latent space via an auto-encoder, then employ a self-expressive layer with $C$ as the learnable $n$0 matrix.

However, these approaches entail $n$1 memory (for $n$2) and $n$3 or higher computational costs (for spectral clustering on $n$4), making them impractical for large $n$5 (Ji et al., 2017, Zhang et al., 2018, Mrabah et al., 24 Dec 2025). SDSNet was developed to eliminate this scaling bottleneck while preserving or improving clustering accuracy.

2. Architectural Innovations

SDSNet architectures share three critical design choices:

1. Encoder-decoder backbone: A convolutional auto-encoder parametrizes the mapping $n$6 and $n$7. The encoder produces a latent embedding $n$8, with $n$9.
2. Scalable self-expressiveness module:
   • In (Zhang et al., 2018), each point $X \in \mathbb{R}^{D \times n}$0 is assigned to one of $X \in \mathbb{R}^{D \times n}$1 learnable subspaces $X \in \mathbb{R}^{D \times n}$2 (with $X \in \mathbb{R}^{D \times n}$3), avoiding the need for a global $X \in \mathbb{R}^{D \times n}$4 affinity.
   • In (Mrabah et al., 24 Dec 2025), the coefficient matrix $X \in \mathbb{R}^{D \times n}$5 is replaced by a low-rank factorization $X \in \mathbb{R}^{D \times n}$6 with $X \in \mathbb{R}^{D \times n}$7 and $X \in \mathbb{R}^{D \times n}$8. Instead of $X \in \mathbb{R}^{D \times n}$9, SDSNet enforces $n$0 for a selected landmark matrix $n$1.
3. Efficient clustering head: Spectral clustering is performed in the low-dimensional space spanned by the factor $n$2 or subspace assignment matrix $n$3, with all operations costing $n$4 when $n$5 are fixed.

A comparison of key self-expressive strategies appears below:

Model	Self-Expressive Representation	Parameter Count
DSC-Net	$n$6	$n$7
SDSNet (2018)	$n$8, $n$9	$k$0
SDSNet (2025)	$k$1, $k$2	$k$3

3. Optimization Objectives and Training Algorithms

SDSNet jointly optimizes auto-encoder reconstruction and subspace-preserving self-expressiveness losses:

• (Zhang et al., 2018):

$k$4

$k$5

$k$6

Alternating minimization updates the assignment $k$7, subspaces $k$8 (via SVD or Grassmannian gradient), and network parameters by minibatch stochastic gradient descent (Adam).

• (Mrabah et al., 24 Dec 2025):

$k$9

Optimizing over $\{\mathcal{S}_i\}_{i=1}^k$0 and $\{\mathcal{S}_i\}_{i=1}^k$1 is achieved via Procrustes SVD or least squares, alternating with back-propagation for encoder/decoder parameters. Affinity $\{\mathcal{S}_i\}_{i=1}^k$2 is block-diagonalized by optimizing representations during training.

4. Landmark-Based and Subspace-Assignment Approximations

A defining feature of SDSNet (Mrabah et al., 24 Dec 2025) is its replacement of the full self-expressiveness matrix $\{\mathcal{S}_i\}_{i=1}^k$3 with a landmark-based factorization. $\{\mathcal{S}_i\}_{i=1}^k$4 anchors are selected via $\{\mathcal{S}_i\}_{i=1}^k$5-means++ or random sampling in latent space, leading to $\{\mathcal{S}_i\}_{i=1}^k$6. The resulting $\{\mathcal{S}_i\}_{i=1}^k$7 encodes how each data point relates to the landmarks, with $\{\mathcal{S}_i\}_{i=1}^k$8. All affinity computations and subsequent spectral clustering occur in the $\{\mathcal{S}_i\}_{i=1}^k$9-dimensional anchor space, yielding $X = X C + E,$0 computation and $X = X C + E,$1 memory.

In (Zhang et al., 2018), explicit assignment variables $X = X C + E,$2 simplify clustering to an assignment to one of $X = X C + E,$3 subspaces per point; subspaces are updated via SVD of assigned embeddings.

5. Computational Complexity and Scalability

SDSNet substantially alleviates the cubic and quadratic scaling issues of prior methods:

• Classical affinity-based deep subspace clustering: $X = X C + E,$4 memory and $X = X C + E,$5 time (building, storing, and decomposing $X = X C + E,$6) (Ji et al., 2017).
• SDSNet (Zhang et al., 2018): Each epoch comprises encoding $X = X C + E,$7 samples ($X = X C + E,$8), assignments ($X = X C + E,$9), and SVD/Grassmann updates ($C \in \mathbb{R}^{n \times n}$0), all linear when $C \in \mathbb{R}^{n \times n}$1 and $C \in \mathbb{R}^{n \times n}$2 are constant.
• SDSNet (Mrabah et al., 24 Dec 2025): Encoder/decoder updates ($C \in \mathbb{R}^{n \times n}$3), $C \in \mathbb{R}^{n \times n}$4/$C \in \mathbb{R}^{n \times n}$5 updates ($C \in \mathbb{R}^{n \times n}$6), and final spectral clustering ($C \in \mathbb{R}^{n \times n}$7).

Empirically, SDSNet clusters 70,000 MNIST images in under 8 minutes using $C \in \mathbb{R}^{n \times n}$82 GB GPU memory; equivalent affinity-based approaches require $C \in \mathbb{R}^{n \times n}$937 GB just for storing the affinity matrix (Zhang et al., 2018). On synthetic datasets with $E$0 up to 10,000, computational time is observed to grow linearly (Mrabah et al., 24 Dec 2025).

6. Empirical Performance and Benchmarking

Extensive experiments demonstrate that SDSNet maintains or surpasses the clustering accuracy of both affinity-based and scalable state-of-the-art methods. On Fashion-MNIST (60,000 samples), SDSNet achieves ACC $E$1 63.8% and NMI $E$2 62.0% in under 10 minutes (Zhang et al., 2018). On YaleB, ORL, Coil100, UMIST, and Fashion-MNIST, (Mrabah et al., 24 Dec 2025) reports that SDSNet matches or exceeds alternatives (SSC-OMP, EnSC, SGL, S$E$3C, LMVSC, SSSC, A-DSSC) by margins of 5–35% in ACC and NMI, with subspace-preserving error (SPE) comparable to dense affinity approaches.

Convergence is achieved rapidly, with the relative change in $E$4 or $E$5 dropping below $E$6 in fewer than 10 outer iterations (Mrabah et al., 24 Dec 2025); affinity matrices display clear block-diagonal structure early in training.

7. Implementation Strategies and Hyperparameters

Robust performance requires careful tuning and implementation choices:

• Hyperparameters: Number of subspaces $E$7 is set to the true number of clusters (e.g., $E$8 for MNIST), embedding dimension $E$9 typically 20, subspace dimension $C$0 in [7, 11], anchor number $C$1 (proportional to $C$2), loss weightings $C$3, batch size $C$4, optimizer Adam ($C$5 lr), AE pre-training $C$6 epochs, joint training $C$7–$C$8 epochs (Zhang et al., 2018).
• Subspace updates: Either top-$C$9 SVD on assignments (discarding 10% outliers) or Riemannian updates (Grassmann retraction).
• Landmark selection: $n$00-means++ improves anchor robustness over random, especially as $n$01 approaches $n$02.
• Initialization and batch scheduling: Pre-training the auto-encoder facilitates stable convergence; initializing subspaces/anchors via $n$03-means or shallow DSC improves accuracy and speed.

SDSNet’s design establishes a paradigm shift for deep subspace clustering, yielding highly scalable, accurate solutions across large-scale, high-dimensional datasets by circumventing the full affinity bottleneck and optimizing computational paths through architectural and algorithmic innovations (Zhang et al., 2018, Mrabah et al., 24 Dec 2025).