Indefinite Stiefel Manifolds

• Indefinite Stiefel manifold is a generalized orthogonal frame space under an indefinite metric, unifying hyperbolic, Lorentzian, and J-orthogonal cases.
• Its structure is defined via the equation XᵀAX = J and leverages pseudo-orthogonal group actions to present a homogeneous space with rich geometric properties.
• It supports computational models and optimization techniques that exploit pseudo-Riemannian metrics, enabling efficient solutions in high-dimensional and trace minimization problems.

The indefinite Stiefel manifold is a fundamental geometric object arising from the paper of orthogonality under indefinite inner products. Whereas the classical Stiefel manifold parameterizes $k$-frames that are orthonormal with respect to a positive-definite quadratic form, the indefinite variant generalizes this to frames orthogonal with respect to an indefinite form, typically of signature $(p,q)$ with $p+q=n$. The practical and theoretical relevance spans pseudo-Riemannian geometry, representation theory, algebraic topology, and manifold optimization—particularly in contexts where indefinite metrics or noncompact symmetry groups appear.

1. Core Definitions and Structural Properties

Formally, the indefinite Stiefel manifold is the set

$$\operatorname{iSt}_{A, J}(k, n) = \left\{ X \in \mathbb{R}^{n \times k} : X^{T} A X = J \right\}$$

where $A \in \mathbb{R}^{n \times n}$ is a symmetric nonsingular matrix (possibly indefinite) and $J \in \mathbb{R}^{k \times k}$ is symmetric with $J^2 = I_k$ (Tiep et al., 29 Oct 2024, Tiep et al., 19 Sep 2025). This structure encompasses several canonical cases:

• If $A = I_n$ and $J = I_k$, this reduces to the classical (compact, orthogonal) Stiefel manifold.
• If $A$ is positive-definite and $J = I_k$, one obtains the generalized Stiefel manifold.
• If $A$ (or $J$) is diagonal with both $+1$ and $-1$ entries, the structure models hyperbolic, Lorentz, or $J$-orthogonal manifolds.

The manifold $\operatorname{iSt}_{A, J}(k, n)$ is embedded and closed; its dimension is $nk - \frac{1}{2}k(k+1)$ (Tiep et al., 29 Oct 2024).

2. Lie Group Action and Homogeneous Space Presentation

The indefinite Stiefel manifold is naturally a homogeneous space for a pseudo-orthogonal group. For signature $(p,q)$, the defining orthogonality condition is

$$X^{T} J X = I_k,\quad J = \operatorname{diag}(I_p, -I_q)$$

and the symmetry group is $O(p, q)$ or $SO(p, q)$. The manifold can be represented as a quotient $O(p, q)/O(p-k, q)$. For various nonstandard quotients (such as flip Stiefel manifolds under pairwise coordinate flipping), one obtains descriptions like $O(n)/(C_2 \times O(n-2k))$ and, more generally, with noncompact symmetry groups in the indefinite setting (Basu et al., 2023).

The isometry group in the indefinite setting is determined via the automorphism group of the associated non-associative algebra $\mathfrak{m}$, derived from the tangent space at the identity coset of the homogeneous space (Sedano-Mendoza, 2019). For Stiefel-type homogeneous spaces $L \backslash H$, isometries are described (up to finite index) as $(G\times H)/T$, where $T$ is a finite central subgroup.

3. Riemannian and Pseudo-Riemannian Geometry

Unlike their definite counterparts, indefinite Stiefel manifolds are generally noncompact. The geometry is dictated by a pseudo-Riemannian metric, typically induced from the Killing form or an invariant extension: $g(Z_1, Z_2) = \operatorname{tr}(Z_1^T M Z_2)$ where $M$ may be chosen as $A$ (possibly indefinite), or as a positive-definite preconditioner (Tiep et al., 29 Oct 2024, Tiep et al., 19 Sep 2025). The tangent space characterization at $X$ is given by

$$T_X \operatorname{iSt}_{A, J} = \left\{ Z \in \mathbb{R}^{n \times k}: Z^T A X + X^T A Z = 0 \right\}$$

A decomposition

$Z = XW + A^{-1} X_\perp K$

with $JW \in \mathrm{Skew}(k)$ and $X^T X_\perp = 0$ yields computationally tractable approaches to evaluating geometric operations.

Geodesic equations and retractions rely on the indefinite metric structure, with retractions based on the Cayley transform, as well as explicit quasi-geodesic curves defined by matrix exponentials involving the characteristic matrix $J$ (Tiep et al., 19 Sep 2025).

4. Algebraic and Topological Invariants

Degree theory, homotopy invariants, and characteristic classes for indefinite Stiefel manifolds exhibit significant differences from the definite case (Brysiewicz et al., 2019, Krzyżanowska et al., 2011). In the algebraic variety setting,

$$\operatorname{St}^{(p,q)}(k,n) = \left\{A \in \operatorname{Mat}_{k\times n} : A J A^T = I_k \right\}$$

leads to pseudo-orthogonal group actions, requiring generalized representation theory, modified Gelfand–Tsetlin patterns, and potentially virtual degree computations due to noncompactness.

Homotopy invariants, particularly for polynomial mappings, may be formulated using signatures of quadratic forms constructed from sets of minors of the defining matrix. For $a:S^{n-k}\to \widetilde{V}_k(\mathbb{R}^n)$, one builds quadratic forms (using trace forms on the finite-dimensional algebra derived from the ideal generated by minors), and computes invariants as half the sum of their signatures: $$A(a) = \frac{1}{2}(\operatorname{signature} O_\delta + \operatorname{signature} O_{f\cdot\delta})$$ This approach can be applied to polynomial immersions to compute intersection numbers, generalizing results of Whitney and Smale (Krzyżanowska et al., 2011).

Topological invariants such as cohomology rings and Stiefel–Whitney classes can also be calculated using spectral sequences associated to bundle fibrations (e.g., in flip Stiefel manifold quotients) and are adaptable via parity conditions and substitution of symmetry groups (Basu et al., 2023, Basu et al., 2021). The Fadell–Husseini index provides cohomological obstructions and can be generalized to the indefinite case by incorporating characteristic classes compatible with the ambient pseudo-Riemannian structure (Basu et al., 2021).

5. Metric-Measure Aspects and High-Dimensional Limits

Recent work demonstrates that, under rescaling and in the high-dimensional limit, the metric-measure structure of (projective) Stiefel manifolds converges weakly to the infinite-dimensional Gaussian space $\Gamma_\infty$, a prototypical “indefinite” structure in Gromov’s metric measure topology (Shioya et al., 2015). Quotienting by mm-isomorphic group actions (e.g., unitary or orthogonal groups), the limit object becomes a quotient of $\Gamma_\infty$, no longer a classical compact manifold but rather a metric-measure pyramid characterized by infinite-dimensional Gaussian fluctuations.

Observable diameter asymptotics quantify concentration-of-measure phenomena: for rescaled manifolds,

$$\text{ObsDiam}(V_{n,n}; -\kappa) \to 2\Phi^{-1}(1-\kappa)$$

with $\Phi^{-1}$ the inverse CDF of the standard normal. This underscores the fundamental metric-measure differences between definite and indefinite (limit) structures.

6. Computational Models, Optimization, and Applications

Orthogonally-equivariant matrix models (“Cholesky models”) parameterized by the cone of symmetric positive-definite matrices $A$ furnish minimal-dimension, numerically favorable representations (Lim et al., 18 Jul 2024). Indefinite generalizations replace the constraint $X^T X = I$ by $X^T \Sigma X = A$ for signature matrix $\Sigma$ and matrix $A$ in the cone of symmetric matrices with prescribed inertia. This aligns the indefinite Stiefel manifold with pseudo-Cartan symmetric spaces, thus enabling closed-form geometric means, projections, and efficient computation.

Optimization algorithms on indefinite Stiefel manifolds utilize tailored Riemannian metrics, explicit projections, and retractions that respect the pseudo-Riemannian structure (Tiep et al., 29 Oct 2024, Tiep et al., 19 Sep 2025). In large-scale problems (e.g., trace minimization, Procrustes problem, hyperbolic eigenvalue problems), employing generalized canonical metrics and quasi-geodesic based retractions leads to substantial computational gains, eliminating the need for Lyapunov equation solves, and yielding feasibility errors on the order of $10^{-12}$–$10^{-13}$ (cf. Table 1 (Tiep et al., 19 Sep 2025)).

7. Algebraic and Topological Implications

The adaptation of classical tools (Gelfand–Tsetlin patterns, representation decompositions, spectral sequences, cohomological index theory) to the indefinite Stiefel manifold framework is nontrivial due to loss of compactness and modified symmetry groups. Combinatorial models, branching rules, and characteristic class computations are all affected by indefinite metrics. Nonetheless, methods exist to complexify the quadratic form, employ signature-modified invariants, and derive criteria for parallelizability, stable span, and the existence of equivariant maps and balancing configurations in topological combinatorics (Basu et al., 2023, Basu et al., 2021). The response of these invariants to changes in signature, central group kernel structure, and noncompactness remains an area of active mathematical investigation.

---

In summary, the indefinite Stiefel manifold generalizes the canonical orthogonal frame space to indefinite metric settings, with ramifications for differential geometry, algebraic topology, representation theory, metric-measure analysis, and manifold optimization. Its paper necessitates the development and adaptation of geometric, algebraic, and computational techniques, all of which reveal deep connections among symmetry, noncompactness, and the analytical structure of high-dimensional and indefinite spaces.