Riemannian Optimization over Symmetric Positive Definite Matrices with the Alpha-Procrustes Geometry

Abstract: In Riemannian optimization, it is well known that the condition number of the Riemannian Hessian at an optimum strongly influences the asymptotic convergence behavior of optimization algorithms. On the manifold of symmetric positive definite (SPD) matrices, several commonly used metrics for optimization, such as the Affine-Invariant (AI) and Bures--Wasserstein (BW) metrics, tend to become ill-conditioned as the underlying SPD matrix becomes ill-conditioned. As a result, even when the Euclidean Hessian remains uniformly well-conditioned on the SPD manifold, optimization may still become difficult near an optimum associated with an ill-conditioned SPD matrix. In this paper, we address this issue through the Alpha-Procrustes (AP) geometry on the SPD manifold. This geometry generalizes several well-known metrics, including the Log-Euclidean (LE) metric for (α=0) and the BW metric for (α=1/2). We first show that, when (α=1), all eigenvalues of the Riemannian metric operator induced by the AP geometry are uniformly bounded independently of the underlying SPD matrix. Therefore, under the assumption that the Euclidean Hessian satisfies the uniform spectral bounds, all the eigenvalues of the corresponding Riemannian Hessian are uniformly bounded independently of the underlying SPD matrix. Consequently, the case (α=1) provides a robust geometric framework for several Riemannian optimization problems involving ill-conditioned SPD matrices. Finally, we validate our theoretical findings through extensive numerical experiments across a range of applications.