Fixed-Rank PSD Matrix Identification

• Fixed-rank PSD matrices are symmetric, non-negative matrices with prescribed rank, admitting a factorization as UU^T and a quotient manifold structure.
• Methodologies include streaming and sketching pipelines like the Nyström approximation, as well as manifold optimization techniques ensuring efficient update and convergence.
• Algebraic and convex relaxation methods address rank-constrained semidefinite programs, with applications ranging from high-dimensional statistics to signal processing.

A fixed-rank positive semi-definite (PSD) matrix is a symmetric matrix $M\in \mathbb{R}^{n\times n}$ satisfying $M\succeq 0$ and $\mathrm{rank}(M)=r\le n$, with $r$ prescribed. Identification of such matrices arises in streaming PCA, semidefinite programming, high-dimensional statistics, signal processing, and matrix-variate data modeling. This domain synthesizes low-rank convex optimization, algebraic geometry, quotient manifold theory, and stochastic approximation, and poses challenging mathematical and computational problems with numerous applications in modern data science.

1. Geometric and Algebraic Structure of Fixed-Rank PSD Matrices

The set of $n\times n$ PSG matrices of rank $r$,

$$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$

forms a real algebraic variety and, modulo measure-zero subsets, can be endowed with a smooth manifold structure. Every $M\in S^+_r(n)$ admits a factorization

$$M = UU^T,\quad U\in \mathbb{R}^{n\times r},\, \mathrm{rank}(U)=r,$$

unique up to $O(r)$ action ($M\succeq 0$0, $M\succeq 0$1). Thus, $M\succeq 0$2 as a quotient manifold, where $M\succeq 0$3 is the set of full-rank $M\succeq 0$4 matrices. The Riemannian structure is inherited from the ambient Euclidean metric, with the horizontal space at $M\succeq 0$5 given by $M\succeq 0$6 (Mishra et al., 2012, Meyer et al., 2010, Szczapa et al., 2019). This geometric description underpins most algorithmic approaches for fixed-rank PSD matrix identification, guaranteeing invariance and efficient computation.

2. Identification From Streaming and Sketching: The Nyström Fixed-Rank Pipeline

High-dimensional streaming settings preclude direct storage of $M\succeq 0$7. The fixed-rank PSD identification pipeline developed in (Tropp et al., 2017) solves

• Given: linear update stream $M\succeq 0$8 ($M\succeq 0$9), sketch size $\mathrm{rank}(M)=r\le n$0
• Maintain: $\mathrm{rank}(M)=r\le n$1 for random $\mathrm{rank}(M)=r\le n$2 (Gaussian or orthonormal), updated as $\mathrm{rank}(M)=r\le n$3.
• At any time, approximate $\mathrm{rank}(M)=r\le n$4 as follows:
  1. Compute the full-rank Nyström approximation:

  $\mathrm{rank}(M)=r\le n$5

2. Obtain the best rank-$\mathrm{rank}(M)=r\le n$6 PSD approximation by truncating the spectrum:

$\mathrm{rank}(M)=r\le n$7

3. For numerical stability, employ a "shift-and-truncate" procedure: shift $\mathrm{rank}(M)=r\le n$8 slightly, apply Cholesky to $\mathrm{rank}(M)=r\le n$9, orthogonally solve for $r$0, SVD of $r$1, and truncate.

This method achieves for Schatten-1 (trace) norm

$r$2

with $r$3 (real field), and any prescribed relative error by selecting $r$4. Storage and time scale as $r$5 and $r$6, making it suitable for large-scale streaming PSD matrix identification (Tropp et al., 2017).

3. Manifold Optimization Approaches

Low-rank PSD identification from data/measurements is formulated as manifold optimization:

• Search space: $r$7
• Objective: e.g., least-squares of measurements $r$8, or regression losses in Mahalanobis metric estimation.
• Algorithms: Riemannian gradient descent and trust-region variants employ projected gradients (horizontal-space projection), retractions (linear, polar), and vector transport (Mishra et al., 2012, Meyer et al., 2010). The key steps are:
  1. Compute Euclidean gradient, then project onto the horizontal space:

  $r$9

2. Retraction employs $n\times n$0 (if full rank), or polar QR. 3. Trust-region subproblems solve quadratic models in the horizontal space. These methods guarantee local convergence to stationary points; in regression settings, convergence rates match standard Riemannian optimization theory (Mishra et al., 2012, Meyer et al., 2010).

In stochastic and online versions, mini-batch and decaying step-size rules extend scalability (Meyer et al., 2010).

4. Algebraic and Convex Relaxation Methods for Rank-Constrained Semidefinite Programs

Fixed-rank constraints render SDPs non-convex. For certain algebraically-structured problems (e.g., with null-shaping constraints), the reduction developed in (Morency et al., 2016) replaces the non-convex feasible set

$n\times n$1

by a convex program in a low-dimensional subspace, constructed via polynomial ideals:

• Null-shaping constraints define a subspace via a polynomial $n\times n$2 vanishing at prescribed points.
• The feasible set becomes all PSD matrices with column space in $n\times n$3, where $n\times n$4 encodes the ideal basis.
• Reformulate $n\times n$5 with $n\times n$6, $n\times n$7, and optimize $n\times n$8 subject to linear trace constraints in $n\times n$9.
• The dimension reduces from $r$0 to $r$1, and the resulting convex SDP is efficiently solvable (Morency et al., 2016). This guarantees that the solution $r$2 has rank $r$3.

This approach is exact whenever the structural constraints force the desired rank, and enables applications in array beamforming, phase retrieval, and signal/noise subspace identification.

5. Statistical and Riemannian Models for Structured Fixed-Rank PSD Cores

In multivariate and matrix-variate statistics, separable covariance models decompose a covariance as

$r$4

with $r$5 Kronecker-separable, $r$6 a fixed-rank core, and identifiability up to measure-zero sets (Sung, 30 Nov 2025). The core manifold

$r$7

is a compact, embedded submanifold of $r$8, with local coordinates given by $r$9, $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$0 in a Stiefel-type manifold. Partial isotropy estimators (PICSE) shrink $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$1 towards the identity via convex combinations, solved by Riemannian Newton methods on this submanifold. Empirically, PICSE outperforms core shrinkage (CSE) and Kronecker MLE for $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$2 (Sung, 30 Nov 2025). The diffeomorphism $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$3 provides geometric structure for full-rank covariance estimation.

6. Algorithmic and Complexity Theory: PSD Matrix Rank and Factorizations

PSD rank of a nonnegative matrix $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$4 is the smallest $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$5 such that $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$6 with $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$7, $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$8 (Shitov, 2016). Determining if $$S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}$$9 is ETR-complete (equivalent to the existential theory of the reals). For fixed $M\in S^+_r(n)$0, the problem reduces to feasibility of quadratic inequalities in $M\in S^+_r(n)$1 variables and is polynomial-time solvable; for general $M\in S^+_r(n)$2, the computational barrier is the same as ETR.

The identification problem for fixed-rank PSD matrices therefore subsumes complexity-theoretic barriers and admits efficiently computable relaxations only under fixed parameter or algebraic structure.

7. Applications and Practical Considerations

Fixed-rank PSD matrix identification algorithms are central in the following domains:

• Streaming PCA and kernel methods requiring online, space-efficient updates (Tropp et al., 2017)
• High-dimensional covariance estimation with low-rank signal plus isotropic noise (Sung, 30 Nov 2025)
• Rank-constrained semidefinite programming for array design, communications, and phase retrieval, where subspace structure is induced by algebraic or geometric constraints (Morency et al., 2016)
• Riemannian regression and distance metric learning, where Mahalanobis metrics are restricted to low-rank PSD forms to enhance generalization and scalability (Meyer et al., 2010)
• Action recognition and manifold trajectories, leveraging the quotient geometry of $M\in S^+_r(n)$3 for curve fitting and sequence alignment (Szczapa et al., 2019)

Practical implementation requires careful choice of rank $M\in S^+_r(n)$4, sketch size $M\in S^+_r(n)$5, and numerical stabilization (e.g., shift-and-truncate or polar QR retraction). Algorithmic complexity is governed by $M\in S^+_r(n)$6 for sketching/reconstruction and $M\in S^+_r(n)$7 per iteration for manifold optimization; the streaming methods and quotient geometry retain scalability for large $M\in S^+_r(n)$8 and moderate $M\in S^+_r(n)$9.

In summary, fixed-rank PSD matrix identification is a multifaceted domain, integrating streaming sketching, Riemannian optimization, algebraic SDP reformulation, and high-dimensional statistical modeling, with a unifying emphasis on the geometric and computational structure of the constraint set.