---
title: Self-supervised Riemannian GNNs
url: https://www.emergentmind.com/topics/self-supervised-riemannian-gnns-selfrgnn
type: topic
---

# Self-supervised Riemannian GNNs

A self-supervised Riemannian Graph Neural Network (SelfRGNN) is a class of GNN architectures designed to learn node representations on graphs by embedding them in Riemannian manifolds, particularly with learnable, possibly time-varying or heterogeneous curvature, and optimizing objectives without requiring external labels. These models generalize classical Euclidean GNNs and fixed-curvature hyperbolic/spherical GNNs, addressing limitations in geometric expressivity and data efficiency, especially for temporal or structurally diverse graphs. Several instantiations exist under this paradigm, including temporal curvature dynamics [2208.14073], mixed-curvature product spaces [2112.05393], motif-aware generative-contrastive approaches [2401.01232], and Ricci curvature–driven co-refinement techniques [2401.12780].

## 1. Riemannian Manifolds and Curvature in Graph Embedding

SelfRGNNs employ Riemannian manifolds as embedding spaces, parametrized by curvature $\kappa$. Specific instantiations include constant-curvature spaces (hyperbolic: $\kappa<0$, Euclidean: $\kappa=0$, spherical: $\kappa>0$) and, in some frameworks, product manifolds of varying curvature factors to model heterogeneous graph regions. Embedding nodes in these manifolds enables the capture of hierarchical, cyclical, or bottleneck patterns, which manifest as negative, positive, or mixed curvature at different scales or epochs.

Crucially, recent models allow the curvature $\kappa$ to be learned and adapted per time step (temporal graphs) [2208.14073], per component (mixed-curvature) [2112.05393], per motif [2401.01232], or per edge/region (heterogeneous curvature) [2401.12780]. This flexibility is essential for representing nonuniform or evolving graph geometries, in contrast to earlier models constrained to single, static $\kappa$.

## 2. Architectural Components of Self-supervised Riemannian GNNs

A prototypical SelfRGNN processes a graph $G=(V,E,X,T)$ as follows [2208.14073, 2112.05393, 2401.01232, 2401.12780]:

- **Time or Structural Encoding:** Time points or structural factors are encoded using translation-invariant schemes, often leveraging random Fourier features for temporal graphs (e.g., $\phi^0(t)$ using sinusoidal bases) [2208.14073] or gyrovector Fourier features for product-manifold factors [2401.01232, 2401.12780].
- **Curvature Module:** A neural network (e.g., "CurNN" or similar) maps encodings to functional curvature(s), e.g., $\kappa(t)=MLP(\phi^0(t))^T W MLP(\phi^0(t))$ [2208.14073], providing curvature values per time or graph region.
- **Graph Convolution in Riemannian Spaces:** Riemannian graph convolutions generalize Euclidean message passing to curved contexts using operation sets—exponential/logarithm maps for transporting between tangent and manifold, Riemannian attention for weighted aggregation, and gyrovector operations for addition and transformation [2208.14073, 2112.05393].
- **Numerical Stability:** Certain variants replace unstable $\exp/\log$ operations with gyrovector kernel mappings (e.g., using random Fourier–style features in each manifold factor, maintaining isometry invariance and preventing numerical blow-up for large $|\kappa|$) [2401.01232, 2401.12780].
- **Feature and Structure Co-Refinement:** Some architectures, such as DeepRicci, couple feature learning with structure learning via differentiable, Ricci curvature–aware updates and backward Ricci flow to alleviate over-squashing [2401.12780].

| Layer/Module                   | Core Mechanism                               | Curvature Adaptation         |
|-------------------------------|----------------------------------------------|-----------------------------|
| Time/Structural Encoder       | Random Fourier, gyrovector or Eucl. features | Per-time/factor             |
| Curvature Module              | MLP/Bilinear maps, GRUs                      | Time-varying or local       |
| Riemannian GNN Layer          | Manifold-aware conv/attention                | Contextual per layer/factor |
| Kernel Mapping (optional)     | Fourier/gyrovector kernels                   | Numerically stable          |
| Ricci/Curvature Co-Refinement | Differentiable Ricci est. + Ricci flow       | Heterogeneous (per edge)    |

## 3. Self-supervised Objectives and Contrastive Mechanisms

SelfRGNN frameworks eschew label supervision in favor of contrastive and generative objectives tailored to the Riemannian context:

- **Self-Contrastive Learning:** Embeddings from different time points or manifold curvatures are projected (via exp/log maps or Lorentz projections) into shared spaces, treating alternate curvatures or timestamps as positive pairs and reweighting negatives based on geometric similarity [2208.14073, 2211.17068]. Adversarial motif-aware contrastive losses regularize learning to emphasize hard positives/negatives [2401.01232].
- **Dual/Hierarchical Contrast:** Mixed-curvature models contrast representations across the canonical hyperbolic, spherical, and Euclidean "views," utilizing Riemannian discriminators and projectors [2112.05393].
- **Ricci-based Curvature Regularization:** Edge-based Ricci curvature (often Ollivier–Ricci) is computed in the embedding geometry, then empirically regularized to agree with the functional/global curvature(s) learned by the model—via sequence modeling (e.g., GRUs) and explicit curvature losses [2208.14073, 2401.12780].
- **Motif-aware Generative Games:** MotifRGC introduces a GAN-like generator-discriminator min-max game in Riemannian product space, generating fake motifs and regularizing both the manifold and node embeddings to respect observed motif structure [2401.01232].

In all these cases, the fundamental aim is to structure the embedding space (and the learned curvature) so as to maximize mutual information, structural motif consistency, or agreement with Ricci curvature—all without access to labels.

## 4. Riemannian Tools: Manifold Operations and Product Manifolds

SelfRGNNs extend classical GNN operations to Riemannian settings through a unified notation for:

- **Exponential and Logarithm Maps:** Moves between the manifold and tangent spaces at the origin or arbitrary points, allowing consistent transformations when mixing representations of different curvatures or time steps [2208.14073, 2211.17068].
- **Distance and Attention Calculations:** Riemannian/geodesic distances characterize similarity, attention, and negative sampling, with manifold-aware kernels ensuring invariance [2208.14073, 2401.01232].
- **Product Manifolds:** Multi-factor mixed-curvature product manifolds are constructed by Cartesian products of constant-curvature spaces; the squared product metric is the sum of per-factor squared distances [2112.05393, 2401.01232, 2401.12780].
- **Numerically Stable Kernelization:** Random gyrovector Fourier features are used to build isometry-invariant kernel representations of manifold points, eliminating the need for unstable $\exp/\log$ operations and facilitating stable message passing [2401.01232, 2401.12780].
- **Backward Ricci Flow:** By iteratively modifying the edge structure based on learned Ricci curvatures, bottlenecks are widened, mitigating over-squashing [2401.12780].

## 5. Empirical Validation and Theoretical Results

Empirical studies across multiple benchmarks (e.g., Wikipedia, MOOC, Cora, Citeseer, Chameleon, Squirrel, Airport) demonstrate that SelfRGNNs outperform both Euclidean and fixed-curvature Riemannian GNNs—frequently by 1–8 percentage points in classification or link prediction AUC [2208.14073, 2112.05393, 2401.01232, 2401.12780]. Key findings include:

- Time-varying curvature or mixed-curvature product spaces consistently outperform single constant-curvature models [2208.14073, 2112.05393].
- Numerically stable kernel-based architectures prevent NaN/overflow errors observed in high-curvature $\exp/\log$ approaches [2401.01232, 2401.12780].
- Ricci-based backward flow demonstrably alleviates over-squashing, with spectral and Cheeger constant analyses confirming increased graph conductance [2401.12780].
- Theoretical propositions establish closed-form exp/log maps under unified notation, translation invariance of time-encoding kernels, and equivalence of contrastive objectives to InfoNCE in suitable limits [2208.14073].
- Case studies (e.g., Physics citation network, ogbn-arXiv) reveal that learned curvatures track known temporal or task-driven geometric transitions (e.g., from spherical to hyperbolic) [2208.14073, 2211.17068].
- Ablations indicate that both geometry-aware curvature adaptation and curvature-driven contrastive mechanisms are necessary for top performance; fixed-curvature or Euclidean variants are consistently outperformed [2208.14073, 2211.17068, 2401.12780].

## 6. Relationship to Related Self-supervised and Riemannian GNN Approaches

The SelfRGNN paradigm encompasses, generalizes, or complements several lines of research:

- **SelfMGNN** [2112.05393]: Introduces mixed-curvature product manifolds and dual-view self-supervised learning, focusing on static graphs but handling diverse geometric regions via hierarchical attention and Riemannian projectors.
- **RieGrace** [2211.17068]: Targets continual graph learning with adaptive curvature, neural curvature adapters (CurvNet), and label-free Lorentzian distillation in Riemannian space; supports task-sequential graphs.
- **MotifRGC** [2401.01232]: Combines motif-level generative adversarial learning with motif-aware contrastive objectives in a diverse-curvature product manifold, employing numerically stable gyrovector kernels.
- **DeepRicci** [2401.12780]: Applies product manifolds and gyrovector mapping to self-supervised graph structure learning, operationalizing differentiable Ricci curvature and backward Ricci flow for structure-feature co-refinement and over-squashing mitigation.

These frameworks share the core principles of (i) learning or adapting curvature in the embedding space; (ii) using Riemannian geometry to better represent graph structure; and (iii) employing self-supervised, typically contrastive, objectives—often enhanced by curvature- or motif-aware regularization. They collectively mark a shift from static, Euclidean or uniformly curved models to a more expressive, geometry-adaptive, and label-efficient regime for graph neural networks.

Source: https://www.emergentmind.com/topics/self-supervised-riemannian-gnns-selfrgnn