---
title: Gram/Wishart/Stiefel Formulation
url: https://www.emergentmind.com/topics/gram-wishart-stiefel-formulation
type: topic
---

# Gram/Wishart/Stiefel Formulation

The Gram/Wishart/Stiefel formulation is a family of matrix representations in which the same mathematical object is viewed through three tightly linked lenses: orthonormal frames on the Stiefel manifold, Gram or covariance matrices of the form \(X^{T}X\), and Wishart-type random matrices. In the modern matrix-manifold literature, this formulation is used both as a geometric model for constrained matrix optimization and as a probabilistic factorization principle for Gaussian, spherical, and structured random matrices. A central theme is that orthogonally equivariant representations preserve symmetry, reduce the ambient dimension to the minimum possible in the Stiefel case, and yield closed-form expressions for metrics, geodesics, gradients, Hessians, and sampling procedures [2407.13482, 1201.6634].

## 1. Algebraic core of the formulation

The real Stiefel manifold is
\[
V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},
\]
the set of ordered \(p\)-frames in \(\mathbb R^n\). It is a compact manifold of dimension \(np-\frac{p(p+1)}{2}\). In the matrix-model viewpoint, a manifold \(M=G/H\) is embedded into a linear space so that the \(G\)-action is realized by a linear or conjugation action; for the Stiefel manifold, orthogonal equivariance under \(O(n)\) or \(SO(n)\) is the relevant symmetry [2407.13482].

A basic reparameterization replaces the standard constraint \(X^{T}X=I_p\) by a fixed positive-definite Gram matrix. For any \(A\in S_{++}^{p}\),
\[
V_A(p,n)=\{Y\in\mathbb R^{n\times p}:Y^{T}Y=A\}.
\]
Each \(V_A(p,n)\) is a smooth submanifold of \(\mathbb R^{n\times p}\) of the same dimension as \(V_{p,n}\). The Cholesky factorization \(A=R^{T}R\) gives the canonical form \(Y=Q[R;0]\) with \(Q\in O(n)\). In this sense, the family \(\{V_A\}_{A\in S_{++}^{p}}\) exhausts all lowest-dimensional \(O(n)\)-equivariant models of the Stiefel manifold, up to the stated exceptions, and is naturally parameterized by the Cartan manifold, namely the positive-definite cone with its natural Riemannian metric [2407.13482].

The Gram/Wishart side of the formulation begins with the observation that \(A=X^{T}X\) is positive-definite whenever \(X\) has full column rank. In statistics, such a matrix is a Gram matrix, and when \(X\) has i.i.d. Gaussian rows it is a central Wishart matrix. The same relation underlies the classical derivation of the Wishart law: if \(X\) is an \(n\times p\) Gaussian matrix and \(W=X^{T}X\), then the polar decomposition
\[
X=U\,W^{1/2},\qquad U\in V_p(\mathbb R^n),
\]
separates a Stiefel factor from a positive-semidefinite radial factor, and integrating out the uniform Stiefel variable yields the Wishart density on \(S_p^{+}\) [1201.6634].

## 2. Orthogonal equivariance and differential geometry

Orthogonal equivariance is not merely a representation-theoretic convenience. In the matrix-model framework it is explicitly tied to numerical stability, because orthogonal transformations are well-conditioned and because equivariance forces natural constructions—metrics, retractions, gradients, Hessians, and related operations—to respect the same symmetry. In the Stiefel case, this leads to closed-form formulas expressed with standard numerical linear algebra rather than iterative solves in the ambient \(n\times p\) space [2407.13482].

For the Cholesky model \(V_A(p,n)\), the tangent space at \(X\) is
\[
T_XV_A=\{U\in\mathbb R^{n\times p}:X^{T}U+U^{T}X=0\},
\]
and the ambient Euclidean metric restricts to
\[
g_X(U,V)=\operatorname{Tr}(U^{T}V).
\]
A commonly used equivalent pull-back form is
\[
g_X(U,V)=\operatorname{Tr}(A^{-1}U^{T}V),
\]
up to constant weights. On the Gram side, the positive-definite cone \(S_{++}^{p}\) carries the affine-invariant metric
\[
g_A(U,V)=\operatorname{Tr}(A^{-1}UA^{-1}V).
\]
With this metric, the unique geodesic from \(A\) to \(B\) is
\[
\gamma(t)=A^{1/2}(A^{-1/2}BA^{-1/2})^{t}A^{1/2},\qquad t\in[0,1],
\]
and the geodesic distance is
\[
d(A,B)=\|\log(A^{-1/2}BA^{-1/2})\|_F.
\]
The same geometry yields closed-form gradient and Hessian expressions:
\[
\nabla f(A)=A\Bigl(\frac{\partial f}{\partial A}\Bigr)A,
\]
with the Hessian involving directional derivatives of the matrix exponential and logarithm [2407.13482].

The computational consequences are explicit. Every \(O(n)\)-equivariant embedding of \(V_{p,n}\) into a Euclidean space must have ambient dimension at least \(np\), and the Cholesky models attain this bound. The standard model \(V_I(p,n)\) has \(X^{T}X=I\) and therefore condition number \(\kappa(X)=1\), whereas in the general model \(V_A\),
\[
\kappa(X)=\sqrt{\kappa(A)}.
\]
This makes conditioning a transparent design variable in the Gram parameter \(A\) [2407.13482].

## 3. Optimization on Stiefel and Gram domains

One major use of the formulation is to convert orthogonality-constrained optimization into algorithms that alternate between ambient linear algebra and manifold projections. In the quadratic problem on the Stiefel manifold,
\[
\min_{W^{T}W=I_k}\operatorname{Tr}(W^{T}AW-2W^{T}B),
\]
the data matrices often arise from Gram constructions such as \(A=XHX^{T}\) and \(B=XHY\) in orthogonal least-squares regression, or \(A=E^{T}E\) and \(B=E^{T}G\) in the unbalanced orthogonal Procrustes problem. The generalized power iteration method introduces a shift \(\widetilde A=\alpha I-A\succ0\), forms
\[
M_t=2(\widetilde A\,W_t+B),
\]
computes a compact SVD \(M_t=U_tS_tV_t^{T}\), and updates by \(W_{t+1}=U_tV_t^{T}\). The resulting sequence monotonically decreases the original objective, and every limit point satisfies the KKT condition [1701.00381].

A different optimization instantiation appears in inference-time activation steering for language models. STARS constrains the steering matrix to the compact Stiefel manifold
\[
M=\{S\in\mathbb R^{d\times k}:S^{T}S=I_k\},
\]
and defines the Gram matrix of the steered activations,
\[
G(S)=(A+S)^{T}(A+S).
\]
The objective is to maximize \(\det G(S)\), equivalently to minimize
\[
\ell(S)=-\log\det[(A+S)^{T}(A+S)].
\]
Its Euclidean gradient is
\[
\nabla_S\ell(S)=-2(A+S)G^{-1},
\]
and the Riemannian gradient is obtained by tangent projection onto
\[
T_SM=\{U:S^{T}U+U^{T}S=0\}.
\]
A polar-decomposition retraction
\[
\operatorname{Retr}_S(U)=(S+U)(I_k+U^{T}U)^{-1/2}
\]
supports Riemannian gradient descent with
\[
\min_{0\le t<T}\|\operatorname{grad}\ell(S_t)\|_F^2=O(1/T),
\]
while the one-step update based on the thin SVD of \(A\) reduces inference-time cost to \(O(k^3+dk^2)\) [2601.22010].

These examples illustrate two complementary uses of the same geometry. In one direction, Gram matrices \(X^{T}X\) generate the quadratic coefficients in Stiefel optimization. In the other, the determinant of a Gram matrix becomes the objective itself, with Stiefel orthogonality preventing degenerate steering directions. This suggests that the formulation is best understood as a transfer principle between low-dimensional Gram geometry and high-dimensional orthogonality constraints.

## 4. Probabilistic factorization and exact distributional identities

The probabilistic content of the formulation is most transparent in matrix polar decomposition. If \(Z_0\sim N_{n\times p}(0,I_n\otimes I_p)\), then
\[
Z_0=H_{Z_0}\,T_{Z_0}^{1/2},
\qquad
H_{Z_0}=Z_0(Z_0^{T}Z_0)^{-1/2}\in O(n,p),
\qquad
T_{Z_0}=Z_0^{T}Z_0\sim W_p(n,I_p),
\]
with \(H_{Z_0}\) and \(T_{Z_0}\) independent, \(H_{Z_0}\sim\mathcal U_{n,p}\), and \(T_{Z_0}\) Wishart. Conversely, if \(U\sim\mathcal U_{n,p}\) is independent of \(A\sim W_p(n,I_p)\), then \(UA^{1/2}\sim N_{n\times p}(0,I_n\otimes I_p)\). This equivalence is the precise probabilistic version of the Gram/Wishart/Stiefel decomposition [2603.12672].

The same factorization yields an exact multivariate normality test. For centered and studentized data, one constructs a residual matrix \(Z\) satisfying
\[
Z^{T}Z=nI_p,
\qquad
U=n^{-1/2}Z\in O(n,p).
\]
Under \(H_0\) of i.i.d. \(N_p(\mu,\Sigma)\) data, \(U\sim\mathcal U_{n,p}\), \(A=nS\sim W_p(n,I_p)\), and they are independent. The test statistic
\[
T=UA^{1/2}=n^{-1/2}ZA^{1/2}
\]
then satisfies
\[
T\sim N_{n\times p}(0,I_n\otimes I_p)
\]
exactly, without asymptotics [2603.12672].

An analogous structure appears for uniform points on the sphere. If \(X_1,\dots,X_n\) are i.i.d. uniform on \(\mathbb S^{d-1}\) and \(X=[X_1|\cdots|X_n]\), then the spherical Wishart matrix
\[
W=X^{T}X
\]
has \(\operatorname{diag}(W)=\mathbf 1_n\). It is related to a classical Gaussian Wishart matrix by a radius-angle decomposition
\[
\widetilde W=(DX)^{T}(DX)=DWD,
\]
where \(D\) contains the \(\chi(d)\)-distributed column norms and is independent of \(W\). This yields exact characteristic-function formulas for \(W\), and in the regime \(d=\Omega(n\log n)\) with \(\|\Theta\|_{\mathrm{op}}=o(d)\),
\[
\log\Psi(\Theta)=-\frac{1}{4d}\operatorname{Tr}(\Theta^2)+O\!\Bigl(\frac{\|\Theta\|_F^3}{d^2}\Bigr),
\]
so the spherical Wishart matrix becomes asymptotically GOE-like at the level of fixed-size marginals [2110.10785].

## 5. Structured, indefinite, and nonclassical variants

The classical positive-definite Wishart model is only one instance of the formulation. In the rank-2 indefinite case,
\[
A=W^{T}W\,\Sigma,\qquad \Sigma=\operatorname{diag}(x_1,x_2),\qquad x_1x_2<0,
\]
a full-rank Gaussian matrix \(W\) admits a \(QR\) factorization with
\[
Q\in V_2(\mathbb F^L),\qquad Q^{*}Q=I_2,
\]
and the indefinite matrix can be analyzed through
\[
A\longleftrightarrow R\,\Sigma\,R^{*}.
\]
The Stiefel factor \(Q\) carries the angular degrees of freedom, while the triangular factor \(R\) determines the eigenvalue ratio and the condition number. For real, complex, quaternionic, and ghost \(\beta\), the paper derives closed-form densities for the eigenvalue ratio and the folded condition-number law [1207.3062].

A different loss of classical structure occurs when the Gram matrix is built from a \(2\times2\) complex Gaussian matrix with arbitrary variance profile,
\[
W=HH^{\dagger},\qquad H_{ij}\sim\mathcal{CN}(0,\phi_{ij}),
\]
because global unitary invariance no longer holds unless the variances are isotropic or Kronecker-factorizable. In this nonclassical setting, the distribution of \(W\) and of its eigenvalues is recovered by an explicit parameterization of \(U(2)\), together with LQ and eigen decompositions and exact angular integration. The resulting formulas reduce to the classical correlated or uncorrelated Wishart laws as special cases [1705.05214].

Graphical structure leads to another extension. For an undirected graph \(G=(V,E)\), the G-Wishart distribution restricts the concentration matrix to
\[
S^p_{\succ0}(G)=\{K\in S^p_{\succ0}:K_{ij}=0\ \text{for every }(i,j)\notin E\}.
\]
Its density is
\[
p(K\mid G,b,D)=\frac{1}{C_G(b,D)}|K|^{(b-2)/2}\exp\!\bigl(-\tfrac12\operatorname{tr}(DK)\bigr)\mathbf 1_{K\in S^p_{\succ0}(G)}.
\]
For chordal graphs, the normalizing constant factorizes over cliques and separators; for arbitrary graphs, an explicit finite-sum and differential-operator formula closes the long-standing question of whether the normalizing constant admits an explicit representation [1406.4901].

The manifold side also generalizes beyond the definite constraint \(X^{T}X=I\). On the indefinite Stiefel manifold
\[
M=\{X\in\mathbb R^{n\times k}:X^{T}AX=J\},
\]
with \(A\) nonsingular symmetric and \(J^2=I_k\), the tangent space is
\[
T_XM=\{Z:X^{T}AZ+Z^{T}AX=0\},
\]
the normal space under the metric \(g_X(\xi,\eta)=\operatorname{tr}(\xi^{T}M\eta)\) is
\[
T_XM^{\perp,g}=\{MAXS:S\in\operatorname{Sym}(k)\},
\]
and the orthogonal projector is obtained from a Lyapunov equation. A Cayley-transform retraction gives a second-order-accurate manifold update, and the resulting Riemannian gradient-descent method has global convergence guarantees [2410.22068].

## 6. Applications, limits, and interpretive significance

The formulation appears across several areas because it isolates radial information in a Gram or Wishart object and angular information in a Stiefel or orthogonal factor. In multivariate statistics this separation produces exact tests based on the independence of a uniform Stiefel factor and a Wishart factor [2603.12672]. In random matrix theory it supports exact and asymptotic analyses of spherical Wishart matrices and, through them, inclusion-divergence comparisons between random geometric graphs and Erdős–Rényi graphs in the sparse high-dimensional regime [2110.10785]. In communication theory it makes it possible to derive exact eigenvalue and outage-related formulas for \(2\times2\) Gram matrices with arbitrary variance profile, even when the classical unitary-invariant toolbox no longer applies [1705.05214].

In optimization and machine learning, the same decomposition supports both conventional orthogonality-constrained regression and inference-time activation steering. The common structural element is the Stiefel constraint, but the objectives differ: quadratic trace optimization in generalized power iteration, determinant maximization of a Gram matrix in STARS, and line-search Riemannian descent on the indefinite Stiefel manifold [1701.00381, 2601.22010, 2410.22068]. This suggests that the formulation is less a single algorithm than a reusable geometric template for problems whose natural variables are frames, covariances, or both.

A recurring misconception is that all Wishart-type analyses inherit classical orthogonal or unitary invariance. Several of the cited works show the contrary. Arbitrary variance profiles break global unitary invariance and require explicit group parameterizations [1705.05214]. Indefinite signatures replace positive-definite Gram geometry by an “indefinite-conjugation” form with one positive and one negative eigenvalue [1207.3062]. Graphical constraints replace the full cone \(S^p_{\succ0}\) by sparsity-pattern submanifolds and alter the normalizing constant problem fundamentally [1406.4901].

Recent theoretical physics work pushes the same language into large-\(d\) gauge theory. In the \(N=2\), large-\(d\) planar endpoint theory of BFSS/BMN matrix quantum mechanics, the endpoint degrees of freedom are reorganized into rank-two Wishart eigenvalues and relative Stiefel angular variables. The holonomy invariants are written in radial and angular Gram data, the exact \(O(2)\) angular integral produces a rank-two Bessel kernel, and the universal linear contribution \(-A\) shifts the mass parameter to \((\alpha_\Lambda)_{\mathrm{eff}}=\alpha_\Lambda-\tfrac12\). In that setting, finite polynomial truncations lead to an apparent large-\(d\) perturbativity bound incompatible with the continuum limit, and the bound is shown to be an artifact of truncation [2607.08481].

Taken together, these developments establish the Gram/Wishart/Stiefel formulation as a unifying framework rather than a single theorem. Its defining move is to trade constrained matrices, covariance-like objects, and orthogonal frames against one another without losing the underlying symmetry. When that trade is available, one gains exact factorization identities, low-dimensional geometric formulas, and computationally tractable algorithms; when symmetry is broken or generalized, the same framework clarifies exactly which parts of the classical theory survive and which must be replaced.

Source: https://www.emergentmind.com/topics/gram-wishart-stiefel-formulation