Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the compact case

Abstract: Gaussian processes are arguably the most important class of spatiotemporal models within machine learning. They encode prior information about the modeled function and can be used for exact or approximate Bayesian learning. In many applications, particularly in physical sciences and engineering, but also in areas such as geostatistics and neuroscience, invariance to symmetries is one of the most fundamental forms of prior information one can consider. The invariance of a Gaussian process' covariance to such symmetries gives rise to the most natural generalization of the concept of stationarity to such spaces. In this work, we develop constructive and practical techniques for building stationary Gaussian processes on a very large class of non-Euclidean spaces arising in the context of symmetries. Our techniques make it possible to (i) calculate covariance kernels and (ii) sample from prior and posterior Gaussian processes defined on such spaces, both in a practical manner. This work is split into two parts, each involving different technical considerations: part I studies compact spaces, while part II studies non-compact spaces possessing certain structure. Our contributions make the non-Euclidean Gaussian process models we study compatible with well-understood computational techniques available in standard Gaussian process software packages, thereby making them accessible to practitioners.

• The paper introduces stationary kernels on compact Lie groups using representation theory to derive computationally efficient covariance functions.
• It leverages the structure of homogeneous spaces to extend Gaussian processes beyond traditional Euclidean settings.
• Practical techniques for kernel evaluation and sampling demonstrate scalability and rapid truncation error decay in experiments.

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces

The paper "Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the compact case" presents a framework for constructing stationary Gaussian processes on compact Lie groups and their homogeneous spaces. This research addresses a significant challenge in machine learning and statistics: extending Gaussian process models to non-Euclidean spaces where traditional Euclidean assumptions are unsuitable.

Core Contributions

The authors introduce methods to develop Gaussian processes on a broad class of non-Euclidean spaces influenced by symmetries, aiming for computational efficiency and applicability within standard Gaussian process software. The focus is on the compact case, with a proposed extension to non-compact spaces in a subsequent paper.

Key contributions include:

1. Stationary Kernels: Constructive methods for calculating covariance kernels that remain invariant under symmetry operations characteristic of the space in question.
2. Gaussian Processes on Lie Groups: The authors leverage the foundation provided by representation theory to handle the challenges inherent in defining Gaussian processes on compact Lie groups. This involves transforming Yaglom's abstract descriptions into computationally feasible methods.
3. Homogeneous Spaces: Extending the Gaussian process models to include homogeneous spaces (formed when a group acts transitively on a space) and ensuring these models are well-suited to capture the symmetry properties of the data space.
4. Kernel Evaluation and Sampling: Practical techniques for pointwise kernel evaluation and efficient sampling from the defined processes are developed, essential for making these models applicable to real-world data.

Theoretical and Practical Implications

The theoretical underpinning of this research is the use of representation theory to define and compute kernels. Characters and zonal spherical functions are employed to evaluate kernels, while computational techniques are developed to traverse irreducible representations of Lie groups, pivotal for practical implementations.

Practical implications include:

• Broadened Application Scope: The developed methods make it feasible to apply Gaussian processes to spaces such as spheres, tori, and more complex geometric structures relevant to fields like robotics, neuroscience, and spatial statistics.
• Computational Feasibility: By aligning with existing computational frameworks, the techniques are accessible to practitioners, not only as theoretical constructs but as usable tools in their workflows.

Numerical Results and Future Directions

Numerical results suggest the proposed methods yield reliable kernel approximations with scalable complexity, making them both applicable and efficient. The analyses demonstrate the rapid decay of truncation error, especially in smooth kernel settings, underscoring the practicality of these models.

Looking forward, the continuation into non-compact spaces promises to further expand the applicability of these methods. Such developments would include spaces of symmetric positive definite matrices and hyperbolic spaces, which are of growing interest in modern machine learning applications.

Conclusion

The work by Azangulov et al. represents a significant step towards integrating deep mathematical principles into practical machine learning workflows, specifically in contexts where traditional Euclidean assumptions fail. It effectively bridges the gap between abstract mathematical theory and tangible computational techniques, positioning Gaussian processes as a versatile tool for a new class of applications in non-Euclidean spaces. The continuation of this work will potentially unlock even broader applications, reinforcing the relevance of stationary kernels in complex, symmetry-rich environments.