Deep Scalable Subspace Clustering (SDSNet)

• Deep Scalable Subspace Clustering (SDSNet) is a deep learning-based method that improves subspace clustering by using landmark-based factorization and k-subspace assignments for scalability.
• It overcomes traditional methods by approximating large affinity matrices, reducing complexity from quadratic/cubic to linear or near-linear performance.
• SDSNet facilitates efficient clustering of high-dimensional data in large-scale applications, delivering competitive accuracy with significantly lower resource usage.

Deep Scalable Subspace Clustering (SDSNet) refers to a family of deep learning-based methods for subspace clustering that achieve computational scalability while leveraging deep representations. SDSNet frameworks address the prohibitive cost of the classical self-expressiveness approach and its deep counterparts, which require constructing and manipulating $n \times n$ affinity matrices, resulting in quadratic or cubic complexity with respect to the number of data points $n$. Recent advances, especially those categorized under the SDSNet moniker, achieve linear or near-linear computational complexity by factorizing affinity computations, approximating self-expression via landmarks, or employing direct subspace assignments in the latent space. These models are designed to cluster high-dimensional data that are assumed to lie near a union of low-dimensional subspaces, and they enable the efficient application of deep subspace clustering to large-scale datasets.

1. Subspace Clustering: Self-Expressiveness and Scalability Limitations

Subspace clustering seeks to partition a dataset $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$, whose columns are assumed to lie near a union of $k$ subspaces $\{S_i\}_{i=1}^k$, into groups corresponding to their generating subspaces. The self-expressiveness property, which underpins classical spectral subspace clustering methods such as Sparse Subspace Clustering (SSC) and Low-Rank Representation (LRR), posits that each data point can be linearly reconstructed from other points in its subspace. This is modeled as: $X = X C + E,$ where $C \in \mathbb{R}^{n \times n}$ is the coefficient (affinity) matrix and $E$ accounts for reconstruction errors. Minimizing the self-expressiveness loss, either with sparsity ($\ell_1$) or Frobenius norm regularization ($\ell_2$), yields the affinity $n$0 used in spectral clustering. However, for large $n$1, the memory and computational cost of forming and eigendecomposing $n$2 becomes prohibitive ($n$3 storage, $n$4 SVD), severely limiting the scalability of these approaches (Ji et al., 2017, Mrabah et al., 24 Dec 2025).

2. Deep Subspace Clustering Networks and Bottleneck Analysis

Deep subspace clustering networks (e.g., DSC-Net) (Ji et al., 2017) augment the self-expressiveness principle with deep encoding:

• An encoder $n$5 maps input data to latent codes $n$6.
• A self-expressive layer parameterized by $n$7 reconstructs the latent codes, $n$8.
• A decoder reconstructs the input from self-expressive codes.

The joint objective optimizes for code reconstruction, input reconstruction, and regularization of $n$9. In practice, these architectures require $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$0 parameters and memory for $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$1 and demand full-batch training. This bottleneck restricts deep subspace clustering to small datasets and precludes mini-batch optimization, hindering architectural depth and scalability. The necessity of loading the full dataset for C updates and spectral clustering further limits applicability.

3. SDSNet: Scalable Deep Subspace Clustering Methodologies

SDSNet approaches (Zhang et al., 2018, Mrabah et al., 24 Dec 2025) overcome the limitations of classical and early deep subspace clustering. Two major algorithmic families have been developed:

3.1 Low-Rank Landmark-Based Factorization

SDSNet (Mrabah et al., 24 Dec 2025) introduces scalable self-expression via a landmark-based factorization. Given encoder output $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$2, select $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$3 landmark codes $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$4 and factor the affinity as $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$5, with $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$6. The self-expression loss is formulated as: $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$7 where $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$8 is the decoder reconstruction. This yields $X = [x_1, ..., x_n] \in \mathbb{R}^{D \times n}$9 storage and $k$0-per-epoch scaling for learning $k$1, $k$2, encoder, and decoder. Spectral clustering is performed in the $k$3-dimensional factor space, circumventing the $k$4 affinity bottleneck.

3.2 Explicit k-Subspace Assignment in Latent Space

SDSNet (Zhang et al., 2018) jointly learns a non-linear embedding and $k$5 subspace bases $k$6 in latent space, enforcing that each embedded code $k$7 is well represented by its assigned subspace: $k$8 Assignments $k$9 and subspaces $\{S_i\}_{i=1}^k$0 are updated alternately with the encoder/decoder, using either closed-form SVD per cluster or Grassmannian manifold gradient updates. The memory cost is $\{S_i\}_{i=1}^k$1 and computation is amenable to mini-batch optimization.

4. Architecture, Training Algorithms, and Losses

SDSNet variants inherit the encoder-decoder paradigm of prior deep clustering approaches but replace dense self-expressive layers with scalable modules.

• Encoder: Typical SDSNet encoders stack convolutional layers, e.g., for $\{S_i\}_{i=1}^k$2 images, layers such as 20 filters of $\{S_i\}_{i=1}^k$3 (stride 2), followed by 10 and 5 filters of $\{S_i\}_{i=1}^k$4 (stride 2), outputting $\{S_i\}_{i=1}^k$5-dimensional latent codes.
• Self-Expression/Assignment Module: Either (1) a factorization $\{S_i\}_{i=1}^k$6 with alternating SVD updates for $\{S_i\}_{i=1}^k$7 and $\{S_i\}_{i=1}^k$8 (Mrabah et al., 24 Dec 2025); or (2) explicit nearest-subspace assignments with optimization over $\{S_i\}_{i=1}^k$9 and $X = X C + E,$0 (Zhang et al., 2018).
• Decoder: Symmetric to the encoder, reconstructing $X = X C + E,$1 from latent codes.
• Overall Loss: Weighted sum of reconstruction loss and subspace/self-expression loss; example:

$X = X C + E,$2

Training Algorithms include pre-training the auto-encoder, mini-batch alternating updates for embedding/SVDs, and periodic subspace or factor updates. Outlier rejection (dropping the 10% of points with highest projection residuals) is employed before basis updates. Standard optimizers, e.g., Adam with $X = X C + E,$3 learning rate, are used.

5. Complexity, Scalability, and Empirical Evaluation

SDSNet architectures dramatically reduce both memory and computational demands compared to non-scalable DSC methods.

• Complexity:
  • Landmark-based SDSNet: $X = X C + E,$4 memory, $X = X C + E,$5 computation per epoch when $X = X C + E,$6 fixed (Mrabah et al., 24 Dec 2025). Spectral clustering in $X = X C + E,$7 rather than $X = X C + E,$8.
  • k-Subspace SDSNet: $X = X C + E,$9 memory for codes, $C \in \mathbb{R}^{n \times n}$0 for subspaces (Zhang et al., 2018).
• Empirical Benchmarks:
  • On MNIST ($C \in \mathbb{R}^{n \times n}$1), SDSNet achieves ACC ≈ 87.1%, NMI ≈ 78.1%, ARI ≈ 75.8% in <8 minutes on a single GPU, with $C \in \mathbb{R}^{n \times n}$2GB memory. In comparison, classic SSC/LRR require $C \in \mathbb{R}^{n \times n}$3GB memory for affinity matrices even before clustering (Zhang et al., 2018).
  • On Fashion-MNIST, ACC ≈ 63.8%, NMI ≈ 62.0%, training in under 10 minutes.
  • Landmark-based SDSNet matches or outperforms scalable baselines (e.g., SSC-OMP, A-DSSC) by 5–35% in accuracy and NMI; versus non-scalable DSC methods, it is within 1–4% in accuracy despite linear-time complexity (Mrabah et al., 24 Dec 2025).
• Practical Notes: Pre-training convergence in ≈200 epochs; Grassmann manifold gradient updates can stabilize subspace learning; cluster initialization with k-means over latent codes accelerates convergence.

6. Spectral Clustering via Factorized Affinities

Rather than building the full $C \in \mathbb{R}^{n \times n}$4 affinity, SDSNet leverages the factor $C \in \mathbb{R}^{n \times n}$5.

• The graph Laplacian $C \in \mathbb{R}^{n \times n}$6 (with $C \in \mathbb{R}^{n \times n}$7 diagonal).
• The eigenproblem is reduced: if $C \in \mathbb{R}^{n \times n}$8, leading eigenvectors of $C \in \mathbb{R}^{n \times n}$9 lie in the range of $E$0. Solving the $E$1 eigenproblem $E$2 produces the embeddings $E$3 for k-means clustering (Mrabah et al., 24 Dec 2025).
• This eigen-decomposition costs $E$4, compared to $E$5 for dense methods.

7. Theoretical and Practical Implications

SDSNet has advanced subspace clustering by:

• Eliminating both affinity matrix bottlenecks and full-batch constraints, enabling deep architectures (e.g., deeper CNNs, large-scale self-supervised learning) for unsupervised clustering (Zhang et al., 2018, Mrabah et al., 24 Dec 2025).
• Showing convergence of learned affinities to block-diagonal structure after a few hundred epochs, matching clustering quality of traditional spectral methods with much reduced computational demands.
• Maintaining theoretical properties of self-expressive clustering (block-structured affinity, spectral guarantees) while being practical for tens or hundreds of thousands of samples.

A plausible implication is that SDSNet has paved the way for extensions with richer neural architectures, hybrid self-supervised objectives, and further innovations in scalable clustering. Careful initialization, mini-batch learning, and principled outlier rejection are critical for convergence and accuracy. The landmark-based and k-subspace assignment paradigms remain the foundation for future research in scalable deep subspace clustering.

Key References:

• "Scalable Deep Subspace Clustering Network" (Mrabah et al., 24 Dec 2025)
• "Scalable Deep $E$6-Subspace Clustering" (Zhang et al., 2018)
• "Deep Subspace Clustering Networks" (Ji et al., 2017)