From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matrices (1407.1120v2)

Abstract: Representing images and videos with Symmetric Positive Definite (SPD) matrices and considering the Riemannian geometry of the resulting space has proven beneficial for many recognition tasks. Unfortunately, computation on the Riemannian manifold of SPD matrices --especially of high-dimensional ones-- comes at a high cost that limits the applicability of existing techniques. In this paper we introduce an approach that lets us handle high-dimensional SPD matrices by constructing a lower-dimensional, more discriminative SPD manifold. To this end, we model the mapping from the high-dimensional SPD manifold to the low-dimensional one with an orthonormal projection. In particular, we search for a projection that yields a low-dimensional manifold with maximum discriminative power encoded via an affinity-weighted similarity measure based on metrics on the manifold. Learning can then be expressed as an optimization problem on a Grassmann manifold. Our evaluation on several classification tasks shows that our approach leads to a significant accuracy gain over state-of-the-art methods.

• The paper introduces an orthonormal projection on Grassmann manifolds to map high-dimensional SPD matrices to a lower-dimensional space while preserving intrinsic Riemannian geometry.
• It formulates the dimensionality reduction as an optimization problem leveraging the Affine Invariant Riemannian Metric or Stein divergence for robust classification.
• Empirical evaluations demonstrate marked improvements in visual recognition tasks, achieving up to 66.6% accuracy on material categorization benchmarks.

Overview of Geometry-Aware Dimensionality Reduction for SPD Matrices

The paper "From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matrices," authored by Harandi et al., proposes a novel technique aimed at addressing the computational challenges associated with handling high-dimensional Symmetric Positive Definite (SPD) matrices, particularly in the context of visual recognition tasks. The authors present a method that strategically transforms a high-dimensional SPD manifold into a lower-dimensional counterpart while preserving and enhancing its discriminative properties. This approach leverages the inherent Riemannian geometry of SPD matrices more effectively compared to existing Euclidean space techniques, which often flatten the manifold, leading to distortion and suboptimal performance.

Key Contributions

The paper primarily contributes to the field of manifold-based learning with the following advancements:

• Orthogonal Projection on Grassmann Manifolds: The authors propose using an orthonormal projection to map high-dimensional SPD matrices to lower-dimensional ones. This approach circumvents the distortions common in techniques that rely on tangent space approximations or Euclidean embedding. The projection is optimized by maximizing discriminative power encoded through a similarity measure based on either the Affine Invariant Riemannian Metric (AIRM) or the Stein divergence.
• Optimization Framework: The dimensionality reduction is expressed as an optimization problem on a Grassmann manifold. This formulation allows the learning task to be performed on Riemannian manifolds, capitalizing on their geometric properties and ensuring affine invariance through optimizations utilizing conjugate gradient methods.

Numerical Results and Claims

The authors provide empirical evidence demonstrating that their approach substantially enhances classification accuracy for several benchmark tasks: material categorization, face recognition, and action recognition. In particular:

• The proposed dimensionality reduction technique, when paired with nearest neighbor and sparse coding classifiers, leads to significant accuracy improvements compared to state-of-the-art methods.
• Their approach yields an impressive accuracy of 66.6% on the UIUC material dataset, surpassing traditional covariance discriminant learning methods. Furthermore, improvements are consistently seen across classification tasks involving both images and motion capture data.

Theoretical and Practical Implications

Harandi et al.’s technique has both theoretical and practical implications:

• Theoretical Advantages: The methodology underscores the importance of preserving manifold geometry during dimensionality reduction. By handling SPD matrices without flattening the manifold, the approach maintains essential curvature properties which are otherwise neglected in Euclidean methods. The affine invariance property of the employed metrics further ensures robust performance under transformations typical in computer vision.
• Practical Benefits: On a practical level, the technique efficiently handles high-dimensional SPD matrices, previously constrained by computational cost, thus allowing the exploitation of richer descriptors in vision applications. This indicates potential extensions to real-time scenarios and larger datasets.

Future Directions

The authors suggest future exploration into unsupervised and semi-supervised settings, as well as extending the framework to different types of Riemannian manifolds. There is an apparent opportunity to generalize these findings to broader applications in machine learning and signal processing, where manifold structures play a crucial role.

This paper contributes significantly to the understanding and operationalization of SPD matrices within manifold-based learning contexts, paving the way for more advanced techniques in dimensionality reduction that faithfully respect and utilize manifold geometries.