Gram/Wishart/Stiefel Formulation

• The Gram/Wishart/Stiefel formulation is a geometric and probabilistic framework that connects orthonormal frames, covariance matrices, and random matrix models.
• It leverages orthogonal equivariance and fixed positive-definite reparameterizations to derive closed-form expressions for metrics, gradients, geodesics, and Hessians.
• Its applications span constrained optimization, multivariate statistical inference, and machine learning, demonstrating both versatility and computational efficiency.

The Gram/Wishart/Stiefel formulation is a family of matrix representations in which the same mathematical object is viewed through three tightly linked lenses: orthonormal frames on the Stiefel manifold, Gram or covariance matrices of the form $X^{T}X$, and Wishart-type random matrices. In the modern matrix-manifold literature, this formulation is used both as a geometric model for constrained matrix optimization and as a probabilistic factorization principle for Gaussian, spherical, and structured random matrices. A central theme is that orthogonally equivariant representations preserve symmetry, reduce the ambient dimension to the minimum possible in the Stiefel case, and yield closed-form expressions for metrics, geodesics, gradients, Hessians, and sampling procedures (Lim et al., 2024, Mayerhofer, 2012).

1. Algebraic core of the formulation

The real Stiefel manifold is

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$

the set of ordered $p$-frames in $\mathbb R^n$. It is a compact manifold of dimension $np-\frac{p(p+1)}{2}$. In the matrix-model viewpoint, a manifold $M=G/H$ is embedded into a linear space so that the $G$-action is realized by a linear or conjugation action; for the Stiefel manifold, orthogonal equivariance under $O(n)$ or $SO(n)$ is the relevant symmetry (Lim et al., 2024).

A basic reparameterization replaces the standard constraint $X^{T}X=I_p$ by a fixed positive-definite Gram matrix. For any $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$0,

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$1

Each $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$2 is a smooth submanifold of $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$3 of the same dimension as $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$4. The Cholesky factorization $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$5 gives the canonical form $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$6 with $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$7. In this sense, the family $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$8 exhausts all lowest-dimensional $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$9-equivariant models of the Stiefel manifold, up to the stated exceptions, and is naturally parameterized by the Cartan manifold, namely the positive-definite cone with its natural Riemannian metric (Lim et al., 2024).

The Gram/Wishart side of the formulation begins with the observation that $p$0 is positive-definite whenever $p$1 has full column rank. In statistics, such a matrix is a Gram matrix, and when $p$2 has i.i.d. Gaussian rows it is a central Wishart matrix. The same relation underlies the classical derivation of the Wishart law: if $p$3 is an $p$4 Gaussian matrix and $p$5, then the polar decomposition

$p$6

separates a Stiefel factor from a positive-semidefinite radial factor, and integrating out the uniform Stiefel variable yields the Wishart density on $p$7 (Mayerhofer, 2012).

2. Orthogonal equivariance and differential geometry

Orthogonal equivariance is not merely a representation-theoretic convenience. In the matrix-model framework it is explicitly tied to numerical stability, because orthogonal transformations are well-conditioned and because equivariance forces natural constructions—metrics, retractions, gradients, Hessians, and related operations—to respect the same symmetry. In the Stiefel case, this leads to closed-form formulas expressed with standard numerical linear algebra rather than iterative solves in the ambient $p$8 space (Lim et al., 2024).

For the Cholesky model $p$9, the tangent space at $\mathbb R^n$0 is

$\mathbb R^n$1

and the ambient Euclidean metric restricts to

$\mathbb R^n$2

A commonly used equivalent pull-back form is

$\mathbb R^n$3

up to constant weights. On the Gram side, the positive-definite cone $\mathbb R^n$4 carries the affine-invariant metric

$\mathbb R^n$5

With this metric, the unique geodesic from $\mathbb R^n$6 to $\mathbb R^n$7 is

$\mathbb R^n$8

and the geodesic distance is

$\mathbb R^n$9

The same geometry yields closed-form gradient and Hessian expressions: $np-\frac{p(p+1)}{2}$0 with the Hessian involving directional derivatives of the matrix exponential and logarithm (Lim et al., 2024).

The computational consequences are explicit. Every $np-\frac{p(p+1)}{2}$1-equivariant embedding of $np-\frac{p(p+1)}{2}$2 into a Euclidean space must have ambient dimension at least $np-\frac{p(p+1)}{2}$3, and the Cholesky models attain this bound. The standard model $np-\frac{p(p+1)}{2}$4 has $np-\frac{p(p+1)}{2}$5 and therefore condition number $np-\frac{p(p+1)}{2}$6, whereas in the general model $np-\frac{p(p+1)}{2}$7,

$np-\frac{p(p+1)}{2}$8

This makes conditioning a transparent design variable in the Gram parameter $np-\frac{p(p+1)}{2}$9 (Lim et al., 2024).

3. Optimization on Stiefel and Gram domains

One major use of the formulation is to convert orthogonality-constrained optimization into algorithms that alternate between ambient linear algebra and manifold projections. In the quadratic problem on the Stiefel manifold,

$M=G/H$0

the data matrices often arise from Gram constructions such as $M=G/H$1 and $M=G/H$2 in orthogonal least-squares regression, or $M=G/H$3 and $M=G/H$4 in the unbalanced orthogonal Procrustes problem. The generalized power iteration method introduces a shift $M=G/H$5, forms

$M=G/H$6

computes a compact SVD $M=G/H$7, and updates by $M=G/H$8. The resulting sequence monotonically decreases the original objective, and every limit point satisfies the KKT condition (Nie et al., 2017).

A different optimization instantiation appears in inference-time activation steering for LLMs. STARS constrains the steering matrix to the compact Stiefel manifold

$M=G/H$9

and defines the Gram matrix of the steered activations,

$G$0

The objective is to maximize $G$1, equivalently to minimize

$G$2

Its Euclidean gradient is

$G$3

and the Riemannian gradient is obtained by tangent projection onto

$G$4

A polar-decomposition retraction

$G$5

supports Riemannian gradient descent with

$G$6

while the one-step update based on the thin SVD of $G$7 reduces inference-time cost to $G$8 (Zhu et al., 29 Jan 2026).

These examples illustrate two complementary uses of the same geometry. In one direction, Gram matrices $G$9 generate the quadratic coefficients in Stiefel optimization. In the other, the determinant of a Gram matrix becomes the objective itself, with Stiefel orthogonality preventing degenerate steering directions. This suggests that the formulation is best understood as a transfer principle between low-dimensional Gram geometry and high-dimensional orthogonality constraints.

4. Probabilistic factorization and exact distributional identities

The probabilistic content of the formulation is most transparent in matrix polar decomposition. If $O(n)$0, then

$O(n)$1

with $O(n)$2 and $O(n)$3 independent, $O(n)$4, and $O(n)$5 Wishart. Conversely, if $O(n)$6 is independent of $O(n)$7, then $O(n)$8. This equivalence is the precise probabilistic version of the Gram/Wishart/Stiefel decomposition (Shimizu et al., 13 Mar 2026).

The same factorization yields an exact multivariate normality test. For centered and studentized data, one constructs a residual matrix $O(n)$9 satisfying

$SO(n)$0

Under $SO(n)$1 of i.i.d. $SO(n)$2 data, $SO(n)$3, $SO(n)$4, and they are independent. The test statistic

$SO(n)$5

then satisfies

$SO(n)$6

exactly, without asymptotics (Shimizu et al., 13 Mar 2026).

An analogous structure appears for uniform points on the sphere. If $SO(n)$7 are i.i.d. uniform on $SO(n)$8 and $SO(n)$9, then the spherical Wishart matrix

$X^{T}X=I_p$0

has $X^{T}X=I_p$1. It is related to a classical Gaussian Wishart matrix by a radius-angle decomposition

$X^{T}X=I_p$2

where $X^{T}X=I_p$3 contains the $X^{T}X=I_p$4-distributed column norms and is independent of $X^{T}X=I_p$5. This yields exact characteristic-function formulas for $X^{T}X=I_p$6, and in the regime $X^{T}X=I_p$7 with $X^{T}X=I_p$8,

$X^{T}X=I_p$9

so the spherical Wishart matrix becomes asymptotically GOE-like at the level of fixed-size marginals (Paquette et al., 2021).

5. Structured, indefinite, and nonclassical variants

The classical positive-definite Wishart model is only one instance of the formulation. In the rank-2 indefinite case,

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$00

a full-rank Gaussian matrix $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$01 admits a $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$02 factorization with

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$03

and the indefinite matrix can be analyzed through

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$04

The Stiefel factor $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$05 carries the angular degrees of freedom, while the triangular factor $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$06 determines the eigenvalue ratio and the condition number. For real, complex, quaternionic, and ghost $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$07, the paper derives closed-form densities for the eigenvalue ratio and the folded condition-number law (Movassagh et al., 2012).

A different loss of classical structure occurs when the Gram matrix is built from a $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$08 complex Gaussian matrix with arbitrary variance profile,

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$09

because global unitary invariance no longer holds unless the variances are isotropic or Kronecker-factorizable. In this nonclassical setting, the distribution of $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$10 and of its eigenvalues is recovered by an explicit parameterization of $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$11, together with LQ and eigen decompositions and exact angular integration. The resulting formulas reduce to the classical correlated or uncorrelated Wishart laws as special cases (Auguin et al., 2017).

Graphical structure leads to another extension. For an undirected graph $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$12, the G-Wishart distribution restricts the concentration matrix to

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$13

Its density is

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$14

For chordal graphs, the normalizing constant factorizes over cliques and separators; for arbitrary graphs, an explicit finite-sum and differential-operator formula closes the long-standing question of whether the normalizing constant admits an explicit representation (Uhler et al., 2014).

The manifold side also generalizes beyond the definite constraint $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$15. On the indefinite Stiefel manifold

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$16

with $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$17 nonsingular symmetric and $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$18, the tangent space is

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$19

the normal space under the metric $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$20 is

$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$21

and the orthogonal projector is obtained from a Lyapunov equation. A Cayley-transform retraction gives a second-order-accurate manifold update, and the resulting Riemannian gradient-descent method has global convergence guarantees (Tiep et al., 2024).

6. Applications, limits, and interpretive significance

The formulation appears across several areas because it isolates radial information in a Gram or Wishart object and angular information in a Stiefel or orthogonal factor. In multivariate statistics this separation produces exact tests based on the independence of a uniform Stiefel factor and a Wishart factor (Shimizu et al., 13 Mar 2026). In random matrix theory it supports exact and asymptotic analyses of spherical Wishart matrices and, through them, inclusion-divergence comparisons between random geometric graphs and Erdős–Rényi graphs in the sparse high-dimensional regime (Paquette et al., 2021). In communication theory it makes it possible to derive exact eigenvalue and outage-related formulas for $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$22 Gram matrices with arbitrary variance profile, even when the classical unitary-invariant toolbox no longer applies (Auguin et al., 2017).

In optimization and machine learning, the same decomposition supports both conventional orthogonality-constrained regression and inference-time activation steering. The common structural element is the Stiefel constraint, but the objectives differ: quadratic trace optimization in generalized power iteration, determinant maximization of a Gram matrix in STARS, and line-search Riemannian descent on the indefinite Stiefel manifold (Nie et al., 2017, Zhu et al., 29 Jan 2026, Tiep et al., 2024). This suggests that the formulation is less a single algorithm than a reusable geometric template for problems whose natural variables are frames, covariances, or both.

A recurring misconception is that all Wishart-type analyses inherit classical orthogonal or unitary invariance. Several of the cited works show the contrary. Arbitrary variance profiles break global unitary invariance and require explicit group parameterizations (Auguin et al., 2017). Indefinite signatures replace positive-definite Gram geometry by an “indefinite-conjugation” form with one positive and one negative eigenvalue (Movassagh et al., 2012). Graphical constraints replace the full cone $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$23 by sparsity-pattern submanifolds and alter the normalizing constant problem fundamentally (Uhler et al., 2014).

Recent theoretical physics work pushes the same language into large-$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$24 gauge theory. In the $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$25, large-$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$26 planar endpoint theory of BFSS/BMN matrix quantum mechanics, the endpoint degrees of freedom are reorganized into rank-two Wishart eigenvalues and relative Stiefel angular variables. The holonomy invariants are written in radial and angular Gram data, the exact $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$27 angular integral produces a rank-two Bessel kernel, and the universal linear contribution $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$28 shifts the mass parameter to $V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$29. In that setting, finite polynomial truncations lead to an apparent large-$V_{p,n}=\{X\in\mathbb R^{n\times p}:X^{T}X=I_p\},$30 perturbativity bound incompatible with the continuum limit, and the bound is shown to be an artifact of truncation (Ydri, 9 Jul 2026).

Taken together, these developments establish the Gram/Wishart/Stiefel formulation as a unifying framework rather than a single theorem. Its defining move is to trade constrained matrices, covariance-like objects, and orthogonal frames against one another without losing the underlying symmetry. When that trade is available, one gains exact factorization identities, low-dimensional geometric formulas, and computationally tractable algorithms; when symmetry is broken or generalized, the same framework clarifies exactly which parts of the classical theory survive and which must be replaced.