---
title: Fixed-Rank PSD Matrix Identification
url: https://www.emergentmind.com/topics/fixed-rank-positive-semi-definite-matrix-identification
type: topic
---

# Fixed-Rank PSD Matrix Identification

A fixed-rank positive semi-definite (PSD) matrix is a symmetric matrix $M\in \mathbb{R}^{n\times n}$ satisfying $M\succeq 0$ and $\mathrm{rank}(M)=r\le n$, with $r$ prescribed. Identification of such matrices arises in streaming PCA, semidefinite programming, high-dimensional statistics, signal processing, and matrix-variate data modeling. This domain synthesizes low-rank convex optimization, algebraic geometry, quotient manifold theory, and stochastic approximation, and poses challenging mathematical and computational problems with numerous applications in modern data science.

## 1. Geometric and Algebraic Structure of Fixed-Rank PSD Matrices

The set of $n\times n$ PSG matrices of rank $r$,
$$
S^+_{r}(n) = \{ M \in \mathbb{R}^{n\times n} : M = M^T \succeq 0,\, \mathrm{rank}(M)=r \}
$$
forms a real algebraic variety and, modulo measure-zero subsets, can be endowed with a smooth manifold structure. Every $M\in S^+_r(n)$ admits a factorization
$$
M = UU^T,\quad U\in \mathbb{R}^{n\times r},\, \mathrm{rank}(U)=r,
$$
unique up to $O(r)$ action ($U\mapsto UQ$, $Q\in O(r)$). Thus, $S^+_r(n) \cong \mathbb{R}_*^{n\times r}/O(r)$ as a quotient manifold, where $\mathbb{R}_*^{n\times r}$ is the set of full-rank $n\times r$ matrices. The Riemannian structure is inherited from the ambient Euclidean metric, with the horizontal space at $U$ given by $\{ \Delta \in \mathbb{R}^{n\times r} : U^T \Delta \text{ is symmetric} \}$ [1209.0430, 1006.1288, 1908.00646]. This geometric description underpins most algorithmic approaches for fixed-rank PSD matrix identification, guaranteeing invariance and efficient computation.

## 2. Identification From Streaming and Sketching: The Nyström Fixed-Rank Pipeline

High-dimensional streaming settings preclude direct storage of $M$. The fixed-rank PSD identification pipeline developed in [1706.05736] solves
- Given: linear update stream $M \gets \theta_1 M + \theta_2 H$ ($H=H^*$), sketch size $k \ge r$
- Maintain: $Y=M\Omega$ for random $\Omega \in \mathbb{R}^{n\times k}$ (Gaussian or orthonormal), updated as $Y\gets \theta_1 Y+\theta_2 H\Omega$.
- At any time, approximate $M$ as follows:
    1. Compute the full-rank Nyström approximation:
       $$
       \widehat M^{\mathrm{nys}} = Y (\Omega^*Y)^\dagger Y^T
       $$
    2. Obtain the best rank-$r$ PSD approximation by truncating the spectrum:
       $$
       \widehat M_r = [\widehat M^{\mathrm{nys}}]_r
       $$
    3. For numerical stability, employ a "shift-and-truncate" procedure: shift $Y$ slightly, apply Cholesky to $\Omega^* Y_\nu$, orthogonally solve for $E = Y_\nu C^{-1}$, SVD of $E$, and truncate.

This method achieves for Schatten-1 (trace) norm
$$
\mathbb{E}\|M-\widehat M_r\|_1 \le \left(1+\frac{r}{k-r-\alpha}\right)\|M-[M]_r\|_1
$$
with $\alpha=1$ (real field), and any prescribed relative error by selecting $k \sim (1+1/\varepsilon)r$. Storage and time scale as $O(nk)$ and $O(nk^2)$, making it suitable for large-scale streaming PSD matrix identification [1706.05736].

## 3. Manifold Optimization Approaches

Low-rank PSD identification from data/measurements is formulated as manifold optimization:
- Search space: $S^+_r(n) \cong \mathbb{R}_*^{n\times r} / O(r)$
- Objective: e.g., least-squares of measurements $y_i \approx \mathrm{tr}(A_i M)$, or regression losses in Mahalanobis metric estimation.
- Algorithms: Riemannian gradient descent and trust-region variants employ projected gradients (horizontal-space projection), retractions (linear, polar), and vector transport [1209.0430, 1006.1288]. The key steps are:
    1. Compute Euclidean gradient, then project onto the horizontal space:
       $$
       \nabla_{U} f \to \nabla_{U} f - U \mathrm{Sym}(U^T \nabla_{U} f)
       $$
    2. Retraction employs $U\mapsto U+\xi$ (if full rank), or polar QR.
    3. Trust-region subproblems solve quadratic models in the horizontal space.
These methods guarantee local convergence to stationary points; in regression settings, convergence rates match standard Riemannian optimization theory [1209.0430, 1006.1288].

In stochastic and online versions, mini-batch and decaying step-size rules extend scalability [1006.1288].

## 4. Algebraic and Convex Relaxation Methods for Rank-Constrained Semidefinite Programs

Fixed-rank constraints render SDPs non-convex. For certain algebraically-structured problems (e.g., with null-shaping constraints), the reduction developed in [1610.02181] replaces the non-convex feasible set
$$
\{X\succeq 0 : \mathrm{rank}(X)=r,\, a(\theta_\ell)^H X a(\theta_\ell)=0\}
$$
by a convex program in a low-dimensional subspace, constructed via polynomial ideals:
- Null-shaping constraints define a subspace via a polynomial $Q(x)$ vanishing at prescribed points.
- The feasible set becomes all PSD matrices with column space in $\mathrm{span}(V)$, where $V$ encodes the ideal basis.
- Reformulate $X=V P V^H$ with $P\succeq 0$, $P\succ \gamma I$, and optimize $f(VPV^H)$ subject to linear trace constraints in $P$.
- The dimension reduces from $n$ to $K=n-L$, and the resulting convex SDP is efficiently solvable [1610.02181]. This guarantees that the solution $X^*$ has rank $K$.

This approach is exact whenever the structural constraints force the desired rank, and enables applications in array beamforming, phase retrieval, and signal/noise subspace identification.

## 5. Statistical and Riemannian Models for Structured Fixed-Rank PSD Cores

In multivariate and matrix-variate statistics, separable covariance models decompose a covariance as
$$
\Sigma = K^{1/2} C K^{1/2, T}
$$
with $K$ Kronecker-separable, $C$ a fixed-rank core, and identifiability up to measure-zero sets [2512.01070]. The core manifold
$$
\mathcal{C}_{p_1,p_2,r}^+ = \{ C \in S_{p,r}^+ : \mathrm{tr}_1(C) = p_2 I_{p_1},\, \mathrm{tr}_2(C) = p_1 I_{p_2},\, \text{not canonically decomposable} \}
$$
is a compact, embedded submanifold of $S_{p,r}^+$, with local coordinates given by $C=AA^T$, $A$ in a Stiefel-type manifold. Partial isotropy estimators (PICSE) shrink $C$ towards the identity via convex combinations, solved by Riemannian Newton methods on this submanifold. Empirically, PICSE outperforms core shrinkage (CSE) and Kronecker MLE for $n<p$ [2512.01070]. The diffeomorphism $\mathcal{S}^{++}_p\cong \mathcal{S}^{++}_{p_1,p_2}\times \mathcal{C}^{++}_{p_1,p_2}$ provides geometric structure for full-rank covariance estimation.

## 6. Algorithmic and Complexity Theory: PSD Matrix Rank and Factorizations

PSD rank of a nonnegative matrix $A$ is the smallest $k$ such that $A_{ij} = \mathrm{tr}(B_i C_j)$ with $B_i, C_j \succeq 0$, $k\times k$ [1606.09065]. Determining if $\mathrm{psd\_rank}(A)\le k$ is ETR-complete (equivalent to the existential theory of the reals). For fixed $k$, the problem reduces to feasibility of quadratic inequalities in $O((m+n)k^2)$ variables and is polynomial-time solvable; for general $k$, the computational barrier is the same as ETR.

The identification problem for fixed-rank PSD matrices therefore subsumes complexity-theoretic barriers and admits efficiently computable relaxations only under fixed parameter or algebraic structure.

## 7. Applications and Practical Considerations

Fixed-rank PSD matrix identification algorithms are central in the following domains:
- Streaming PCA and kernel methods requiring online, space-efficient updates [1706.05736]
- High-dimensional covariance estimation with low-rank signal plus isotropic noise [2512.01070]
- Rank-constrained semidefinite programming for array design, communications, and phase retrieval, where subspace structure is induced by algebraic or geometric constraints [1610.02181]
- Riemannian regression and distance metric learning, where Mahalanobis metrics are restricted to low-rank PSD forms to enhance generalization and scalability [1006.1288]
- Action recognition and manifold trajectories, leveraging the quotient geometry of $S_r^+(n)$ for curve fitting and sequence alignment [1908.00646]

Practical implementation requires careful choice of rank $r$, sketch size $k$, and numerical stabilization (e.g., shift-and-truncate or polar QR retraction). Algorithmic complexity is governed by $O(nk^2)$ for sketching/reconstruction and $O(nr^2)$ per iteration for manifold optimization; the streaming methods and quotient geometry retain scalability for large $n$ and moderate $r$.

In summary, fixed-rank PSD matrix identification is a multifaceted domain, integrating streaming sketching, Riemannian optimization, algebraic SDP reformulation, and high-dimensional statistical modeling, with a unifying emphasis on the geometric and computational structure of the constraint set.

Source: https://www.emergentmind.com/topics/fixed-rank-positive-semi-definite-matrix-identification