Rank-1 Subspace in Matrix and Tensor Theory

• Rank-1 subspace is a linear subspace whose basis elements are rank-1 matrices expressed as outer products, forming the foundation for low-rank approximations.
• It is characterized through algebraic definitions and CP decomposition methods that enable unique recovery via simultaneous diagonalization or thresholded projections.
• Applications span signal processing, massive MIMO, deep learning optimization, and subspace clustering, where exploiting rank-1 structures enhances computational efficiency and model simplicity.

A rank-1 subspace is a linear subspace of matrices or tensors whose basis elements are of rank 1, i.e., each basis element can be written as the outer product of vectors. This notion plays a central role in matrix and tensor theory, low-rank approximation, signal processing, learning theory, and the analysis of optimization landscapes in modern deep learning. Rank-1 subspaces are algebraically rigid, computationally tractable, and their structural properties are fundamental for both theoretical classification and algorithmic exploitation.

1. Algebraic Definition and Characterization

Let $M \subset \mathbb{R}^{m \times n}$ be a $d$-dimensional linear subspace. $M$ is a rank-1 subspace if there exists a basis $\{X_1, ..., X_d\} \subset M$ such that each $X_\ell$ has $\mathrm{rank}(X_\ell) = 1$. Equivalently, each basis element can be expressed as $X_\ell = a_\ell b_\ell^T$ for some $a_\ell \in \mathbb{R}^m$, $b_\ell \in \mathbb{R}^n$; the entire subspace can be written as

$$M = \mathrm{span}\{a_1 b_1^T, a_2 b_2^T, ..., a_d b_d^T\}.$$

This is in striking contrast to general low-rank subspaces, where only the sum of basis ranks is small, or all ranks are bounded above by $d$0 (Nakatsukasa et al., 2015).

The dimension of a rank-1 subspace of $d$1 is at most $d$2, with equality if and only if all matrices in the subspace share a common image (span of columns) or a common kernel (null space), as established by the Atkinson–Lloyd theorem (Pazzis, 2010):

• Common-image case: $d$3 fixed $d$4, $d$5 with $d$6.
• Common-kernel case: $d$7 fixed $d$8, $d$9 with $M$0.

Rank-1 subspaces can also be formalized over arbitrary fields and occur in canonical classification results for spaces of matrices with bounded rank (Pazzis, 2010).

2. Tensor and CP Decomposition Formulation

Rank-1 subspaces have a distinguished characterization via tensor representation. For a basis $M$1, the slices form a tensor $M$2. The existence of a rank-1 basis is strictly equivalent to the tensor $M$3 having canonical polyadic (CP) rank $M$4:

$M$5

where each $M$6 encodes linear mixing coefficients. This representation underpins both existence and uniqueness:

• The subspace $M$7 admits a rank-1 basis if and only if $M$8.
• Under genericity conditions (full rank $M$9, no repeated diagonal ratios in $\{X_1, ..., X_d\} \subset M$0), the CP decomposition is unique up to scaling and permutation (Nakatsukasa et al., 2015).

This connection provides a direct algebraic pathway for both theoretical analysis and computational recovery via simultaneous diagonalization of coupled matrix pencils or higher-order tensor factorizations.

3. Algorithmic Recovery of Rank-1 Bases

Two main algorithmic paradigms are available for extracting a rank-1 basis from a given subspace:

• CP/Simultaneous Diagonalization (exact, $\{X_1, ..., X_d\} \subset M$1):

1. Given a basis $\{X_1, ..., X_d\} \subset M$2, for $\{X_1, ..., X_d\} \subset M$3 form $\{X_1, ..., X_d\} \subset M$4.
2. Perform eigendecomposition $\{X_1, ..., X_d\} \subset M$5; columns of $\{X_1, ..., X_d\} \subset M$6 recover $\{X_1, ..., X_d\} \subset M$7.
3. Recover $\{X_1, ..., X_d\} \subset M$8, $\{X_1, ..., X_d\} \subset M$9 from related pencils and Khatri–Rao products.
4. Yields global recovery in $X_\ell$0 arithmetic (Nakatsukasa et al., 2015).

• Soft-then-Hard Thresholding with Projections:
  • Phase I: Repeated projected soft-threshold SVD (nuclear norm shrinkage) to drive iterates toward rank-$X_\ell$1 ($X_\ell$2 in the rank-1 setting).
  • Phase II: Projected hard thresholding onto rank-$X_\ell$3 and subspace, converging linearly to the rank-1 element.
  • Greedy or simultaneous restart strategies allow extraction of a full rank-1 basis.
  • Empirically, 10–20 SVDs suffice per rank-1 element for subspaces up to $X_\ell$4 (Nakatsukasa et al., 2015).

The CP-based method fails when $X_\ell$5 due to singular pencils, whereas thresholded projection approaches remain robust, at the cost of additional iterations.

Additionally, Krylov subspace and Wedderburn rank-1 deflation strategies deliver efficient, iterative extraction of rank-1 (or higher) components given only matrix-vector or tensor-by-vector operations (Goreinov et al., 2010).

4. Computational Thresholds and Complexity

Recent work on the planted rank-1 subspace problem—finding $X_\ell$6 rank-1 matrices in a generic $X_\ell$7-dimensional subspace of $X_\ell$8—has produced sharp polynomial-time recovery thresholds. The algorithm of Johnston–Lovitz–Vijayaraghavan succeeds for $X_\ell$9, with failure occurring for $\mathrm{rank}(X_\ell) = 1$0, nearly doubling prior guarantees. The procedure relies on intersecting symmetric squares $\mathrm{rank}(X_\ell) = 1$1 with the variety of vanishing $\mathrm{rank}(X_\ell) = 1$2 minors, followed by simultaneous diagonalization of resulting symmetric tensors (Dastidar et al., 24 Apr 2025). These results directly inform the generic decomposability of order-4 tensors via flattening.

The complexity per element in practical algorithms varies from $\mathrm{rank}(X_\ell) = 1$3 for projection methods to $\mathrm{rank}(X_\ell) = 1$4 in simultaneous diagonalization, with polynomial-time scaling as long as $\mathrm{rank}(X_\ell) = 1$5. For very large $\mathrm{rank}(X_\ell) = 1$6 in massive MIMO, randomized sketches accelerate SVD-based rank-1 subspace estimation to $\mathrm{rank}(X_\ell) = 1$7, breaking the accuracy-complexity tradeoff in high-dimensional channel estimation (Li et al., 2024).

5. Applications Across Domains

Signal Processing and Massive MIMO: Rank-1 subspace methods underlie efficient maximum-likelihood channel estimation in massive MIMO systems. Construction of spatial Hankel matrices reveals a low-rank Vandermonde structure; for single-path channels (true rank-1), the parameter estimation reduces to subspace identification via SVD and subsequent beamforming, achieving NMSE decaying as $\mathrm{rank}(X_\ell) = 1$8 at scalable computational cost (Li et al., 2024).

Optimization Landscapes and Deep Learning: In LLM pre-training, the rank-1 subspace phenomenon manifests as late-stage model checkpoints (after averaging) collapsing onto a one-dimensional linear manifold. This structure is exploited for “Extra-Merge” extrapolation, wherein projected line-search steps along the principal direction provide loss improvements at no additional training cost—a direct consequence of geometric low-pass filtering of SGD noise (Zhou et al., 26 May 2026).

Low-Rank Neural Networks: In convolutional neural networks, imposing rank-1 structure on 3D filters (by factorization into the outer product of vectors) enforces that all layers’ feature maps live on low-dimensional subspaces, compresses parameters, and enables fast inference via separable convolution. The training process alternates between unconstrained updates and projections onto the rank-1 manifold, guaranteeing exact maintenance of the subspace constraint (Kim et al., 2018).

Subspace Clustering and Best Rank-1 Approximation: The generalized Eckart–Young–Mirsky theorem ensures that for any unitarily-invariant norm (Frobenius, nuclear, operator), the best rank-1 approximation is the truncation to the top singular component. This is exploited in subspace clustering algorithms, where partitioning data into unions of lines translates directly into extracting rank-1 approximations as primitives (Yu et al., 2012).

Grassmannian Geometry and Subspace Updates: Rank-1 modifications to low-dimensional subspaces admit an explicit geometric interpretation as geodesics on the Grassmann manifold, with closed-form updating formulas for basis and factor matrices, subspace angles, and distance metrics. These provide $\mathrm{rank}(X_\ell) = 1$9 efficient updates for sequential or online subspace tracking applications (Zimmermann, 2017).

6. Classification, Uniqueness, and Limitations

The structure of rank-1 subspaces is rigid, with the Atkinson–Lloyd theorem classifying all maximal-dimension rank-1 subspaces as those with a common image or kernel (no exceptional cases for $X_\ell = a_\ell b_\ell^T$0 over any field). Uniqueness of basis can fail only in the presence of repeated singular values or algebraic degeneracy; generically, the CP decomposition and the extracted basis are unique up to scaling and permutation (Pazzis, 2010, Nakatsukasa et al., 2015). The limitation of the rank-1 model is an inability to capture non-multilinear dependencies, necessitating the shift to higher-rank subspaces or relaxation methods when modeling full-complexity phenomena (Kim et al., 2018).

In sum, the rank-1 subspace—characterized by its basis of outer products—is a paradigmatic object with deep implications for matrix theory, tensor decomposition, optimization, signal processing, and modern machine learning, encompassing both classical algebraic characterizations and emergent computational phenomena.