Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods (1605.06182v1)

Abstract: Representing images and videos with Symmetric Positive Definite (SPD) matrices, and considering the Riemannian geometry of the resulting space, has been shown to yield high discriminative power in many visual recognition tasks. Unfortunately, computation on the Riemannian manifold of SPD matrices -especially of high-dimensional ones- comes at a high cost that limits the applicability of existing techniques. In this paper, we introduce algorithms able to handle high-dimensional SPD matrices by constructing a lower-dimensional SPD manifold. To this end, we propose to model the mapping from the high-dimensional SPD manifold to the low-dimensional one with an orthonormal projection. This lets us formulate dimensionality reduction as the problem of finding a projection that yields a low-dimensional manifold either with maximum discriminative power in the supervised scenario, or with maximum variance of the data in the unsupervised one. We show that learning can be expressed as an optimization problem on a Grassmann manifold and discuss fast solutions for special cases. Our evaluation on several classification tasks evidences that our approach leads to a significant accuracy gain over state-of-the-art methods.

• The paper introduces geometry-aware dimensionality reduction techniques that preserve the SPD manifold structure while enhancing classification accuracy.
• It outlines both supervised and unsupervised methods using Riemannian metrics and Grassmann manifold optimization.
• Experimental evaluations show improved performance in material categorization, action recognition, and video clustering compared to conventional methods.

Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods

The paper presents an in-depth analysis and development of dimensionality reduction (DR) techniques tailored for Symmetric Positive Definite (SPD) manifolds, emphasizing the importance of considering the unique geometric characteristics of these spaces. The authors address the unsuitability of traditional DR methods like Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) that are confined to flat Euclidean spaces, and propose a more geometrically coherent framework for SPD manifolds.

Theoretical Foundation

SPD manifolds provide a rich structure for visual data representation, yet computational complexity, particularly with high-dimensional SPD matrices, remains a significant obstacle. The authors leverage the Riemannian geometry of the SPD manifold, implementing dimension reduction through orthonormal projections onto a lower-dimensional SPD manifold. Two scenarios are investigated:

• Supervised Dimensionality Reduction: Here, the focus is on maximizing discriminative power, thus enhancing classification performance by minimizing intra-class distances and maximizing inter-class distances. The AIRM, Stein divergence, and Jeffrey divergence are utilized for measuring distances, each offering computational advantages and desirable metric properties, such as affine invariance.
• Unsupervised Dimensionality Reduction: Without labeled data, variance maximization becomes the objective, akin to PCA or Maximum Variance Unfolding (MVU). The authors propose methods for both scenarios and provide optimization strategies utilizing Grassmann manifold techniques.

Methodology

The approach starts with defining a mapping transformation from high-dimensional to low-dimensional SPD matrices and expressing this as an optimization problem on a Grassmann manifold. Solutions are formulated through Newton-type optimization methods on the Grassmannian, allowing efficient computation while preserving geometric fidelity.

Moreover, detailed derivations for gradient computation for each metric facilitate the fast execution of optimization tasks. The authors rigorously prove the equivalency of curve lengths under affine invariant Riemannian metrics and establish conditions for intrinsic metric equality. These theoretical contributions are vital for ensuring the proposed dimensionality reduction methods maintain the intuitive metric properties of SPD spaces.

Experimental Evaluation

The proposed DR methodologies are evaluated on several classification and clustering tasks, showing notable improvements over existing methods:

• Material Categorization: Utilizing high-dimensional region covariance matrices (RCMs), the techniques outperform state-of-the-art SIFT-based approaches.
• Action Recognition: Applying the methods to covariance descriptors from motion capture data results in enhanced recognition rates, demonstrating the utility of high-dimensional geometric-aware DR.
• Face Recognition and Video Clustering: Analyzed using high-dimensional covariance representations of video frames, the DR methods significantly boost performance. Kernalized variants further underscore the power of geometry-aware DR.

Implications and Future Directions

This work represents a significant advancement in handling high-dimensional data on Riemannian manifolds, chiefly SPD matrices. The geometry-aware dimensionality reduction techniques not only provide practical benefits in terms of accuracy and computational efficiency but also broaden the theoretical understanding of manifold-based machine learning methods. Future research could extend this approach to other non-Euclidean spaces or investigate unsupervised variants in more complex settings.

The methodologies presented here pave the way for developing more sophisticated algorithms, capable of leveraging the full spectrum of Riemannian geometry for machine learning tasks, and setting a precedent for subsequent investigations into manifold-centric DR techniques.