Self-Expressive Layer in Subspace Clustering

• Self-Expressive Layer is a module that represents each sample as a linear combination of others, forming an affinity graph crucial for subspace clustering.
• Deep auto-encoder instantiations use the self-expressive layer between encoder and decoder, combining reconstruction with coefficient regularization to enforce latent linear relationships.
• Advanced methods refine the coefficient matrix through dual self-expression, entropy-based regularization, and scalable architectures tailored for temporal, graph, and federated data.

Searching arXiv for the cited work and closely related papers to ground the article. In subspace clustering literature, a self-expressive layer is a module that enforces the self-expressiveness property: each sample, or its latent representation, is reconstructed as a linear combination of other samples from the same union of subspaces. In its canonical form, the layer learns a coefficient matrix $C$ under constraints such as $\operatorname{diag}(C)=0$, and the learned coefficients are then converted into an affinity graph for spectral clustering. In deep variants, the module is usually inserted into an encoder-based architecture so that latent codes satisfy $Z \approx ZC$; more recent work augments this construction with second-order coefficient modeling, entropy-based connectivity regularization, contrastive or self-supervised training, and explicit mechanisms for scalability or out-of-sample inference (Zhao et al., 2023, Zhao et al., 2022, Haeffele et al., 2020).

1. Canonical formulation and algebraic role

The standard self-expressive model starts from data $X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$ and represents each point as

$x_j=\sum_{i\neq j} c_{ij}x_i,$

with the coefficient matrix $C\in\mathbb{R}^{N\times N}$ assembled from the coefficients $c_{ij}$. Equivalently, in matrix form,

$X = XC,\qquad \operatorname{diag}(C)=0,$

or, in relaxed form, $X \approx XC$. The diagonal constraint excludes the trivial identity solution in which each point represents itself (Zhang et al., 2021, Xu et al., 2019).

Optimization is typically written as a reconstruction term plus a coefficient regularizer. One classical template is

$$\min_C \frac{1}{2}\|X-XC\|_F^2+\|C\|_p \quad \text{s.t. } \operatorname{diag}(C)=0,$$

while deep and parametric variants replace $\operatorname{diag}(C)=0$0 by a latent representation $\operatorname{diag}(C)=0$1 or replace the explicit coefficients by a learned function $\operatorname{diag}(C)=0$2 (Zhao et al., 2022, Zhang et al., 2021). In the ideal regime, the coefficients are subspace-preserving, meaning that nonzero entries connect points from the same subspace. If the samples are reordered by cluster, the resulting matrix is expected to display a block-diagonal-like structure, and an affinity such as $\operatorname{diag}(C)=0$3 can be used by spectral clustering (Xu et al., 2019, Dyer et al., 2015).

This algebraic role distinguishes the self-expressive layer from a generic fully connected layer. Its purpose is not merely feature transformation, but the estimation of a sample-to-sample relation matrix that acts simultaneously as a reconstruction operator, a structural prior, and a graph generator.

2. Deep auto-encoder instantiations

In deep subspace clustering, the canonical insertion point is between encoder and decoder. A representative formulation maps the input $\operatorname{diag}(C)=0$4 to latent features $\operatorname{diag}(C)=0$5 and reconstructs through

$\operatorname{diag}(C)=0$6

with objective

$\operatorname{diag}(C)=0$7

Here the reconstruction term trains the auto-encoder, the self-expressive term enforces latent-space linear reconstructibility, and the regularizer shapes the coefficient structure (Zhao et al., 2023).

A common training strategy is two-stage. First, the encoder-decoder is pretrained with

$\operatorname{diag}(C)=0$8

so that optimization starts from a stable initialization. Second, the full model is optimized with the self-expressive terms active, after which the learned coefficients are symmetrized into an affinity graph and spectral clustering is followed by $\operatorname{diag}(C)=0$9-means on the spectral embedding (Zhao et al., 2023).

Not all deep variants retain the reconstruction branch during clustering. In S$Z \approx ZC$0CE, a ResNet-50 encoder is contrastively pretrained with a SimCLR/InfoNCE-style objective, then frozen, and the clustering stage optimizes only latent self-expression plus an entropy-norm term. The paper explicitly removes the decoder/reconstruction branch at this stage because reconstruction loss is argued to be unhelpful for relational structure; the self-expressive layer is still the latent-space linear reconstruction module $Z \approx ZC$1, but it is now supported by self-supervised pretraining rather than by end-to-end auto-encoder reconstruction (Zhao et al., 2022).

3. Structural refinement of the coefficient matrix

A major line of development treats the coefficient matrix not just as a by-product of reconstruction, but as a structured representation in its own right. In deep double self-expressive subspace clustering, the first coefficient matrix $Z \approx ZC$2 is itself treated as a feature representation of each sample, and a second self-expressive layer with weights $Z \approx ZC$3 is applied directly to $Z \approx ZC$4: $Z \approx ZC$5 The resulting affinity is formed not from $Z \approx ZC$6 alone, but from the fusion

$Z \approx ZC$7

This design is reported to reduce subspace-preserving representation error (SRE) and improve connectivity (CONN); in the ablation study, the fused $Z \approx ZC$8 graph has much lower SRE than either $Z \approx ZC$9 or $X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$0 alone (Zhao et al., 2023).

The same work adds a self-supervised contrastive term over pairs $X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$1, where $X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$2 is a latent feature and $X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$3 is its self-expressive reconstruction: $X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$4 The full objective

$X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$5

jointly optimizes reconstruction, first self-expression, second self-expression, and contrastive self-supervision (Zhao et al., 2023).

A different structural refinement is entropy-norm regularization. In S$X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$6CE, the clustering-stage objective is

$X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$7

The stated effect is to encourage dense and uniform within-subspace connectivity, reduce the chance that a sample is connected to only a few neighbors, and help form a well-spaced block-diagonal affinity matrix after symmetrization

$X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$8

This directly targets the limited connectivity that sparse or low-rank regularizers may induce (Zhao et al., 2022).

These formulations share the same premise: the quality of clustering depends not only on whether latent points can reconstruct one another, but also on the topology of the coefficient-induced graph.

4. Scalability, subset methods, and out-of-sample inference

A persistent difficulty is that the self-expressive coefficient matrix is $X=[x_1,\dots,x_N]\in\mathbb{R}^{D\times N}$9, so its memory and optimization costs scale quadratically with the sample count. Several methods retain the self-expressive principle while altering how the coefficients are produced (Zhang et al., 2021, Dyer et al., 2015, Hotta et al., 2021).

Method	Core construction	Reported consequence
SEED	Select $x_j=\sum_{i\neq j} c_{ij}x_i,$0 representative columns $x_j=\sum_{i\neq j} c_{ij}x_i,$1 from the data, then compute sparse codes in $x_j=\sum_{i\neq j} c_{ij}x_i,$2	Rectangular self-expressive decomposition and complexity $x_j=\sum_{i\neq j} c_{ij}x_i,$3
PMSSC	Solve subset-wise OMP self-expression on $x_j=\sum_{i\neq j} c_{ij}x_i,$4 sampled subsets, then combine coefficients as $x_j=\sum_{i\neq j} c_{ij}x_i,$5	Decomposed optimization that can be solved efficiently in parallel
SENet	Learn coefficients by a parametric function $x_j=\sum_{i\neq j} c_{ij}x_i,$6	Parameters do not need to scale with $x_j=\sum_{i\neq j} c_{ij}x_i,$7, and the network handles out-of-sample data

SEED replaces full $x_j=\sum_{i\neq j} c_{ij}x_i,$8 self-expression by a two-stage sparse self-expressive decomposition. First, oASIS greedily selects incoherent columns from the dataset. Second, OMP or batch OMP computes sparse representations with respect to the selected columns. The selected subset can exactly recover a rank-$x_j=\sum_{i\neq j} c_{ij}x_i,$9 matrix once $C\in\mathbb{R}^{N\times N}$0 linearly independent columns have been chosen, and the stated overall complexity is approximately $C\in\mathbb{R}^{N\times N}$1 (Dyer et al., 2015).

PMSSC replaces the single global dictionary by $C\in\mathbb{R}^{N\times N}$2 subsets of size $C\in\mathbb{R}^{N\times N}$3. For each subset, it solves an OMP-style sparse approximation and obtains subset reconstructions $C\in\mathbb{R}^{N\times N}$4; then it estimates combination weights $C\in\mathbb{R}^{N\times N}$5 and forms a final coefficient vector by weighted aggregation. The paper argues that combining multiple subset coefficient vectors improves self-expressiveness and graph connectivity while permitting parallel execution of the dominant subset problems (Hotta et al., 2021).

SENet moves further away from dataset-specific optimization. It learns two MLP embeddings $C\in\mathbb{R}^{N\times N}$6 and $C\in\mathbb{R}^{N\times N}$7, forms pairwise scores through inner products, and applies a learnable soft-thresholding operator

$C\in\mathbb{R}^{N\times N}$8

The objective remains unsupervised and reconstruction-based, but the learned function can be reused on unseen data. The paper also derives a two-pass SGD algorithm that keeps training memory essentially constant with respect to dataset size (Zhang et al., 2021).

5. Temporal, graph, and federated adaptations

The self-expressive layer has also been adapted to settings in which the data are temporal, graph-structured, or distributed across views and clients (Xu et al., 2019, Bandyopadhyay et al., 2020, Liu et al., 2024, Li et al., 12 Mar 2025).

In LSTM-ESCM, the coefficient matrix becomes a time-indexed hidden state: $C\in\mathbb{R}^{N\times N}$9 The LSTM hidden state is used as the self-expressive representation matrix, trained by a reconstruction error plus optional $c_{ij}$0 sparsity,

$c_{ij}$1

The affinity at each time step is $c_{ij}$2, so the layer explicitly learns how pairwise affinities evolve over time (Xu et al., 2019).

In SEComm, a 2-layer GCN first produces node embeddings $c_{ij}$3, after which a self-expressive module learns a batch-wise coefficient matrix $c_{ij}$4 through

$c_{ij}$5

The resulting similarities are converted into soft must-link and no-link constraints that supervise a community-assignment MLP. The module is explicitly post-encoder and pre-clustering rather than part of the message-passing stack, and the batch-wise design reduces the cubic cost from $c_{ij}$6 to about $c_{ij}$7 per batch (Bandyopadhyay et al., 2020).

In GCSR, self-expression reconstructs synthetic graph structure during graph condensation. Given condensed features $c_{ij}$8, a probabilistic adjacency prior $c_{ij}$9, and a historical matrix $X = XC,\qquad \operatorname{diag}(C)=0,$0, the method solves

$X = XC,\qquad \operatorname{diag}(C)=0,$1

with closed-form solution

$X = XC,\qquad \operatorname{diag}(C)=0,$2

and synthetic adjacency

$X = XC,\qquad \operatorname{diag}(C)=0,$3

Here the self-expressive layer is not a neural layer with learnable weights, but a closed-form reconstruction module that injects original graph structure and stabilizes topology across condensation epochs (Liu et al., 2024).

In FedMSGL, the self-expressive module is the local subspace-learning engine at each federated node. Each client learns a relation space by reconstructing

$X = XC,\qquad \operatorname{diag}(C)=0,$4

where $X = XC,\qquad \operatorname{diag}(C)=0,$5 is the consistent part and $X = XC,\qquad \operatorname{diag}(C)=0,$6 is the view-specific part, under constraints including zero diagonals, affine normalization, and nonnegativity. These local relation matrices have uniform $X = XC,\qquad \operatorname{diag}(C)=0,$7 shape regardless of the raw feature dimension $X = XC,\qquad \operatorname{diag}(C)=0,$8, after which the server fuses the consistent parts into a global subspace and constructs a hypergraph (Li et al., 12 Mar 2025).

6. Critique, misconceptions, and broader reuse of the term

The principal critique of deep self-expressive clustering is that the formulation can be ill-posed. If encoder and decoder are sufficiently expressive and include positively homogeneous nonlinearities such as ReLU, the embedding can be scaled toward zero while preserving reconstruction, so the self-expressive term can be driven arbitrarily small. Even when scale is constrained by dataset, batch, channel, or instance normalization, the optimized embeddings may still be degenerate: all points may collapse to one direction, only two points may remain nontrivial while others approach the origin, or points may become duplicated or sign-flipped copies. The same critique also argues that a significant portion of previously claimed empirical gains is attributable to ad hoc post-processing of $X = XC,\qquad \operatorname{diag}(C)=0,$9, including hard thresholding, the shape interaction matrix method, and entrywise powering, rather than to the deep self-expressive model itself (Haeffele et al., 2020).

This critique addresses a common misconception: a self-expressive layer is not guaranteed to induce a meaningful union-of-subspaces geometry merely because it minimizes a latent reconstruction penalty. The coefficient matrix may still require strong structural regularization, careful normalization, or downstream heuristics before it becomes a reliable affinity for clustering (Haeffele et al., 2020).

A second misconception is that any mechanism described as “self-expressive” is equivalent to the classical coefficient-learning layer. Outside subspace clustering, the term has been reused more broadly. One-layer self-attention with softmax and rank-1 $X \approx XC$0 is proved to be a contextual mapping that captures full sequence context in a single layer; the hidden state in TTT layers is itself a learnable model $X \approx XC$1 updated online by self-supervised gradient descent; SA-SLM explicitly states that its “self-expressive layer” is not a single literal layer name but a coupled mechanism consisting of Intent-Aware Bridging and Realization-Aware Alignment; and the handbook on self-aware systems defines self-expression as the capability of a node to assert behavior upon itself or other nodes based on state, context, goals, values, objectives, and constraints (Kajitsuka et al., 2023, Sun et al., 2024, Wang et al., 13 Apr 2026, Chen et al., 2014). This suggests that, beyond subspace clustering, “self-expressive layer” increasingly functions as a broader architectural metaphor for mechanisms that reuse a model’s own internal structure, state, or outputs as an explicit object of conditioning, regulation, or action.