---
title: Adaptive Riemannian Graph Neural Networks
url: https://www.emergentmind.com/topics/adaptive-riemannian-graph-neural-networks-argnn
type: topic
---

# Adaptive Riemannian Graph Neural Networks

Adaptive Riemannian Graph Neural Networks (ARGNNs) are graph neural architectures in which node or edge representations reside on one or more learnable Riemannian manifolds, and in which message passing, attention, normalization, and optimization are executed through manifold operations such as $\operatorname{Log}$, $\operatorname{Exp}$, parallel transport, and Fréchet means. Across the recent literature, the term encompasses several closely related programs: intrinsic-geometry-first graph modeling under the Riemannian Foundation Model agenda, continuous node-wise metric-tensor learning, mixed-curvature product or expert architectures, adaptive scale selection through Riemannian graph-of-graphs constructions, and continual or spiking variants that adapt geometry across graphs, layers, or tasks [2603.21601] [2508.02600].

## 1. Conceptual foundations

The modern ARGNN perspective is grounded in the claim that, for graphs, “Riemannian geometry speaks louder than words.” In that view, graph serialization for language-model-style processing fragments global topology and cannot reliably capture high-order structural dependencies, while classical GNNs remain constrained by fixed Euclidean inductive biases, local message passing, and poor memory retention under multi-domain pretraining. Riemannian geometry is proposed instead as a universal structural substrate: manifolds model structural diversity, curvature encodes structural complexity, moving frames align local coordinates across domains, and vector bundles integrate graph structure as a Riemannian base with attributes as Euclidean fibers [2603.21601].

Within this program, ARGNN does not merely switch from Euclidean space to a non-Euclidean embedding space. It treats intrinsic geometry—flatness, symmetry, connectivity, and anisotropy—as the primary object of learning, with geometric invariants expected to emerge endogenously from the learned manifold. This is the basis for the description of ARGNN as a “Riemannian engine” whose representations, attention, and optimization live on learned manifolds rather than on a fixed ambient space [2603.21601].

A complementary line of work frames the same problem through geometric heterogeneity inside a single graph. Real graphs may contain tree-like hierarchies and dense communities simultaneously, so a single fixed-curvature manifold or a discrete product of constant-curvature factors can be too coarse. In response, ARGNN has also been formalized as learning a continuous, anisotropic Riemannian metric tensor field over the graph, so that each node determines its own local geometry [2508.02600].

This conceptual shift also intersects with the theory of over-squashing. In Riemannian GNNs of variable curvature, the decay of sensitivity across graph bottlenecks can be separated into a topological term and a curvature-dependent geometric multiplier. Negative curvature can increase that multiplier, suggesting a mechanism by which adaptive geometry may alleviate over-squashing in tree-like regions, while positive curvature can instead decrease sensitivity [2311.15945].

## 2. Geometric and mathematical structure

The common mathematical backbone is a Riemannian manifold $(M,g)$ with metric tensor $g$. In the intrinsic formulation, geodesic distance is defined by
$$
d_{M}(x,y) = \inf_{\gamma \in \Gamma(x,y)} \int_{0}^{1} \sqrt{\langle \dot{\gamma}(t), \dot{\gamma}(t)\rangle_{g_{\gamma(t)}}}\,dt,
$$
where $\Gamma(x,y)$ is the set of smooth curves from $x$ to $y$. The exponential map $\operatorname{Exp}_x(v)$ and logarithmic map $\operatorname{Log}_x(y)$ bridge manifold space and tangent space, enabling tangent-space computation followed by manifold-valued updates. Parallel transport $\mathcal{P}_{x\to y}$, defined through the Levi-Civita connection, gives the canonical way to move vectors or gradients between tangent spaces before aggregation [2603.21601].

Optimization is similarly geometric. The Riemannian gradient satisfies
$$
g_x(\operatorname{grad}_M f(x), v)=df_x(v),
$$
and in coordinates with metric tensor $G(x)$,
$$
\operatorname{grad}_M f(x)=G(x)^{-1}\nabla f(x).
$$
Riemannian SGD and Riemannian Adam take tangent-space steps and retract or exponentiate back to the manifold, while momentum terms can be transported between tangent spaces by parallel transport [2603.21601].

A major branch of ARGNN adopts constant-curvature spaces. In the unified hyperspherical–hyperbolic formalism,
$$
d_M(x,y)=\frac{1}{\sqrt{|\kappa|}}\cdot \cos^{-1}_{\kappa}\!\big(\kappa\langle x,y\rangle_{\kappa}\big),
$$
with corresponding closed-form exponential and logarithmic maps
$$
\exp^{\kappa}_{x}(v)=\cos_{\kappa}(v)x+\frac{\sin_{\kappa}(v)}{v}v,\qquad
\log^{\kappa}_{x}(y)=\frac{\cos^{-1}_{\kappa}(\beta)}{\sin_{\kappa}(\cos^{-1}_{\kappa}(\beta))}(y-\beta x),
$$
where $\beta=\kappa\langle x,y\rangle_{\kappa}$. This unifies hyperspherical and Lorentz-hyperbolic geometry inside one algebraic scheme [2211.17068].

A second branch moves from manifold selection to metric-field learning. There the ambient feature space remains $\mathbb{R}^d$, but each node $i$ carries its own symmetric positive definite metric tensor, specialized to a learnable diagonal form
$$
G_i=\operatorname{diag}(g_i)=\operatorname{diag}(g_{i,1},\ldots,g_{i,d})\in S_{++}^{d},
$$
with
$$
g_i=\operatorname{softplus}(f^{(g)}_{\theta}([h_i;a_i]))+\epsilon.
$$
The node-local inner product is $\langle u,v\rangle_{G_i}=u^\top G_i v$, and the locally geodesic distance between neighboring embeddings is
$$
d_{G_i}(h_i,h_j)=\sqrt{(h_i-h_j)^\top G_i(h_i-h_j)}.
$$
This formulation replaces global curvature choice with node-wise anisotropic scaling and yields $O(d)$ learnable metric degrees of freedom per node under the diagonal constraint [2508.02600].

Curvature itself may also be learned explicitly. One proposal parameterizes curvature as
$$
\kappa=\mathrm{softplus}(\theta)-\mathrm{softplus}(\phi),
$$
allowing positive, negative, or near-zero curvature while regularizing smoothness across neighborhoods [2603.21601].

## 3. Architectural patterns

A canonical ARGNN layer begins by placing embeddings on manifolds through either a learned chart $\psi:\mathbb{R}^{d}\to M$ or a tangent-space projection at a learned base point $p_i$, such as
$$
h_i^{(0)}=\operatorname{Exp}_{p_i}(W x_i).
$$
Base points function as moving frames: local orthonormal coordinates aligned with the evolving metric, intended to support cross-domain alignment and comparison [2603.21601].

Message passing then takes place intrinsically. One blueprint aggregates neighbor information in tangent spaces after transport:
$$
h_{i}^{(k+1)}=
\operatorname{Exp}_{p_i}\!\Big(
\sum_{j\in\mathcal{N}(i)} \alpha_{ij}\,
\mathcal{P}_{p_j\to p_i}\big(\operatorname{Log}_{p_j}(h_j^{(k)})\big)
+\beta\,\operatorname{Log}_{p_i}(h_i^{(k)})
\Big),
$$
with attention weights derived from geodesic distances,
$$
\alpha_{ij}\propto \operatorname{softmax}_{j}\!\big(-d_M(h_i^{(k)},h_j^{(k)})^2/\tau\big).
$$
Pooling may be defined by Fréchet means, and normalization may be done in $T_{p_i}M$ before mapping back by $\operatorname{Exp}_{p_i}$ [2603.21601].

The continuous-metric formulation replaces explicit manifold-valued transport with receiver-specific local geometry. It defines a direction vector
$$
u_{ij}=h_j-h_i,\qquad d_{ij}=\frac{u_{ij}}{\|u_{ij}\|_2+\epsilon},
$$
a geometric modulation coefficient
$$
\tau_{ij}=\sum_{k=1}^{d} d_{ij,k}^{2}\cdot \tanh(-\log g_{i,k}),
$$
and a geometry-aware attention score
$$
\alpha_{ij}=\frac{\langle h_i,h_j\rangle_{G_i}}{\|h_i\|_{G_i}\cdot \|h_j\|_{G_j}+\epsilon}.
$$
Messages are then
$$
m_{ij}^{(l)}=\tau_{ij}^{(l)}\cdot \sigma(\alpha_{ij}^{(l)})\cdot W_m^{(l)} h_j^{(l)},
$$
followed by the update
$$
h_i^{(l+1)}=\sigma\!\left(W_s^{(l)}h_i^{(l)}+\sum_{j\in N(i)}m_{ij}^{(l)}\right).
$$
No parallel transport is required because the model uses local metrics at receivers and sender norms for normalization [2508.02600].

Another architectural family addresses scale adaptivity jointly with geometry. R-GFM constructs a Graph-of-Graphs (GoG) in which each node corresponds to a $k$-hop ego-subgraph, $k\in\{1,\ldots,K\}$, around the same center node. Hop-wise embeddings are stacked into $X_{\text{sub}}\in\mathbb{R}^{K\times d}$, and a similarity matrix
$$
S=X_{\text{sub}}X_{\text{sub}}^\top
$$
defines sparse GoG edges through budgeted similarity sampling. A router then assigns each GoG to a subset of Riemannian experts with different constant curvatures, using dynamic Top-$m$ routing and confidence-aware sparsification [2605.09993].

Closely related but more symmetry-focused is the manifold diffusion perspective. Manifold GCN defines a graph Laplacian
$$
\Delta f(v)=-\sum_{u\sim v} w(v,u)\log_{f(v)}f(u),
$$
and an explicit Euler diffusion step
$$
(\operatorname{step}_{t,\alpha}f)(v)=\exp_{f(v)}\!\left(-t\,\sigma_{f(v)}^{\alpha}(\Delta f(v))\right),
$$
together with a tangent multilayer perceptron (tMLP) that mixes channels in tangent spaces before reprojecting to the manifold. Both layers are permutation equivariant and isometry equivariant [2401.14381].

## 4. Objectives, regularization, and optimization

ARGNN training objectives usually combine standard supervised losses with geometry-aware terms. In the intrinsic manifold setting, supervised cross-entropy or regression can be computed in tangent space or via manifold linear maps. Contrastive learning is often geodesic:
$$
L=-\sum_i \log
\frac{\exp(-d_M(h_i,h_i^{+})^2/\tau)}
{\sum_l \exp(-d_M(h_i,h_l)^2/\tau)},
$$
and triplet constraints use the same metric to enforce relative separation. Fréchet regularization is added to stabilize cluster structure, pooling, and hierarchy [2603.21601].

Regularization of the geometry itself is central. In continuous ARGNN, the discrete Ricci approximation
$$
Ric_{kk}^{(i)}=\frac{1}{2|N(i)|}\sum_{j\in N(i)}(g_{i,k}-g_{j,k})
$$
induces a Ricci flow-inspired penalty
$$
L_{\text{Ricci}}=\sum_{i\in V}\sum_{k=1}^{d}\big(Ric_{kk}^{(i)}\big)^2,
$$
while edgewise smoothness is enforced through
$$
L_{\text{smooth}}=\sum_{(i,j)\in E}\|g_i-g_j\|_2^2.
$$
The total objective is
$$
L_{\text{total}}=L_{\text{task}}+\alpha L_{\text{Ricci}}+\beta L_{\text{smooth}}.
$$
These terms are explicitly motivated as curvature flattening and metric-field continuity [2508.02600].

Other regularizers target curvature smoothness, metric eigenvalue ranges, anisotropy penalties, or boundary safety. For hyperbolic balls, one proposal uses
$$
\lambda \sum_i \max(0,\|h_i\|-r_{\max})^2
$$
with $r_{\max}<1$, together with gradient clipping in tangent space [2603.21601].

Optimization schemes vary by parameter type. Euclidean parameters are typically updated with Adam, while geometry-aware parameters use RiemannianAdam, Riemannian SGD, or RAMSGrad. R-GFM explicitly uses Adam for Euclidean parts and RiemannianAdam for curvature-aware parameters, with `stabilize=100` [2605.09993]. Earlier hyperbolic GNN work used RAMSGrad for hyperbolic parameters and AMSGrad for Euclidean parameters, establishing the now-standard separation between Euclidean and Riemannian optimization paths [1910.12892].

Self-supervised and continual variants extend this training picture. RieGrace uses Label-free Lorentz Distillation with intra-distillation and inter-distillation, mediated by a Generalized Lorentz Projection and Lorentz Layer that preserve manifold constraints across changing curvatures and dimensions [2211.17068].

## 5. Representative systems and empirical record

The current ARGNN literature is not a single architecture but a family of realizations that differ in what is adapted: local metric, curvature sign, manifold expert, scale, motif structure, temporal dynamics, or energy profile.

| System | Adaptive mechanism | Reported setting/result |
|---|---|---|
| ARGNN [2508.02600] | Node-wise diagonal SPD metric tensor field; Ricci flow-inspired regularization | Cora F1 $86.83\pm0.84$; Cora AUROC $91.03$ |
| R-GFM [2605.09993] | Multi-scale GoG; dynamic Riemannian expert routing | Up to a $49\%$ relative improvement on downstream tasks; Cora AUC-ROC $89.27$ |
| RieGrace [2211.17068] | CurvNet + AdaRGCN + label-free Lorentz Distillation | Cora PM/FM $95.2/-1.2$ |
| CUSP [2502.00401] | Mixed-curvature spectral filtering; Cusp Laplacian; hierarchical attention | Up to $5.3\%$ improvement in node classification and up to $5.11\%$ in link prediction |
| MSG [2410.17941] | Manifold spiking neuron; differentiation via manifold | Physics node classification $95.93\pm0.07$; Physics energy $0.029$ mJ |
| GSG [2508.06793] | Mixed-curvature product space; curvature-aware attention and gating | Physics node classification $96.27\pm0.13$; Physics LP $97.46\pm0.73$ |

R-GFM is the clearest foundation-model-scale realization of adaptive scale and geometry. It reports that fixed-hop GFMs suffer scale mismatch, constructs multi-scale GoGs, and shows state-of-the-art downstream performance across ten benchmarks and four domains. Its link prediction AUC-ROC scores include $89.27$ on Cora, $88.66$ on PubMed, $81.53$ on Photos, and $87.94$ on Texas, while robustness experiments report $46.91\%$ accuracy under edge drop $p=0.5$ and $46.84\%$ under node masking $r=0.5$ [2605.09993].

The continuous metric-field ARGNN reports strong performance on both homophilic and heterophilic datasets, including Texas $92.28\pm1.59$, Cornell $90.85\pm0.33$, and Wisconsin $90.65\pm2.34$ for node classification, alongside link prediction gains such as Actor $76.40$ and CiteSeer $90.13$. Its ablations attribute notable drops to removing Ricci regularization or metric smoothness, especially on heterophilic graphs [2508.02600].

Several specialized variants extend the adaptive program. Manifold GCN applies diffusion and tangent MLP layers to manifold-valued node features and reports $0.765\pm0.075$ accuracy on Alzheimer’s classification with sphere-valued normals and only $2554$ parameters, compared with $0.752\pm0.085$ for a Euclidean GCN and $0.592\pm0.073$ for MeshCNN [2401.14381]. MotifRGC couples a diverse-curvature product manifold with a stable gyrovector kernel layer and motif-aware generative-contrastive learning, reporting link-prediction AUC values as high as $98.86$ on Cora and $99.09$ on PubMed, while also noting that removing the kernel layer can produce NaNs on Citeseer link prediction [2401.01232].

Spiking variants introduce a separate axis of adaptation. MSG frames manifold updates as geodesic state evolution driven by spiking currents and replaces BPTT with differentiation via manifold, reporting at least $20\times$ energy savings over ANN-R baselines across datasets [2410.17941]. GSG adds mixed-curvature product spaces with instance-wise manifold gating and curvature-aware attention, giving, for example, Computers node classification $90.11\pm0.27$ and link prediction $97.27\pm0.92$ [2508.06793].

## 6. Theory, interpretability, and unresolved issues

The theoretical literature around ARGNN has grown unusually quickly. For the continuous metric-field model, the convergence theorem states that under bounded features, Lipschitz task loss, SPD and bounded metrics, and connected graph assumptions,
$$
\mathbb{E}\big[\|\nabla L_{\text{total}}^{(t)}\|^2\big]
=
O\!\left(\frac{1}{\sqrt{t}}\exp\!\left(-\frac{\mu_{\text{eff}} t}{L}\right)\right),
$$
and a separate universality theorem shows that Euclidean GNNs, constant-curvature models, and product-manifold GNNs arise as constrained cases of the adaptive diagonal metric field [2508.02600].

R-GFM contributes complementary guarantees at the foundation-model level. Its Theorem 3.2 states that fused multi-hop noise satisfies $\|\sigma_{\nu}\|_2\le \|\sigma_F\|_2$ relative to fixed-hop noise. Theorem 3.3 states that the expected squared embedding error of the sparse similarity-based GoG is strictly lower than both the no-edge and fully connected constructions. Theorem 3.4 gives an excess-risk upper bound
$$
R(j)=A S_N+B\sqrt{S_N/n_N},
$$
with dynamic expert selection satisfying $R(U_D)\le R(V_F)$, and Theorem 3.5 states that the best achievable target bound of R-GFM is strictly smaller than that of MDGFM under mild target shift assumptions [2605.09993].

Over-squashing analysis provides a geometric interpretation of long-range information flow:
$$
\left\|
\frac{\partial \mathbf{x}_i^{(\ell)}}{\partial \mathbf{x}_j^{(0)}}
\right\|_2
\le
c_\sigma^{\ell} w^{\ell}\beta_i(k,K)^{\ell}(\tilde{\mathbf{A}}^{\ell})_{ij}.
$$
Here $(\tilde{\mathbf{A}}^{\ell})_{ij}$ captures graph topology, while $\beta_i(k,K)$ captures curvature through the differentials of $\operatorname{Exp}$ and $\operatorname{Log}$. Negative curvature can make $\beta_i(k,K)>1$, while positive curvature can make it smaller than $1$ [2311.15945].

Interpretability claims in ARGNN are usually geometric rather than token-based. Curvature fields act as intrinsic indicators of structure; geodesic attention makes relational focus explicit; Fréchet centers expose “structural hubs”; and large holonomy, interpreted as path-dependent transport, is proposed as a flag for inconsistency, whereas stable curvature fields indicate consistent structural logic [2603.21601]. In the continuous metric-field model, learned geometries are additionally reported to show flatter regions for separable clusters and higher curvature near decision boundaries, while Neighbor-Relative Metric Dispersion correlates with heterophily [2508.02600].

The limitations are equally consistent across papers. Reported failure modes include numerical instability near hyperbolic boundaries, curvature overfitting, ambiguous manifold selection in sparse neighborhoods, and the expense of exact parallel transport on general manifolds [2603.21601]. The diagonal metric assumption misses rotated anisotropy and off-diagonal coupling [2508.02600]. CV-based expert-count selection in R-GFM is explicitly described as effective but heuristic [2605.09993]. Continual contrastive distillation in RieGrace has $O(|V|^2)$ complexity [2211.17068]. CUSP inherits the overhead of Ollivier–Ricci estimation and Sinkhorn-based curvature-aware preprocessing [2502.00401]. MSG is presently focused on undirected, homophilous graphs [2410.17941].

The forward agenda is already visible. One branch seeks universal structural understanding through intrinsic geometry and a vector-bundle interface between graph structure and Euclidean semantic fibers, positioning ARGNN as the internal Riemannian engine of broader Graph Foundation Models and Riemannian Foundation Models [2603.21601]. Another branch pushes toward time-varying metrics, edge-dependent metrics, non-diagonal SPD parameterizations, multi-relational geometric fields, and joint geometry–rewiring schemes [2508.02600] [2311.15945]. A plausible implication is that future ARGNN systems will be judged less by whether they choose hyperbolic or spherical space, and more by how effectively they learn, regularize, and exploit intrinsic geometry as a transferable computational object.

Source: https://www.emergentmind.com/topics/adaptive-riemannian-graph-neural-networks-argnn