SPD-Valued Sheaves in Geometric Learning

• SPD-valued sheaves are a graph-based learning framework that replaces Euclidean fibers with SPD matrices, enabling the capture of second-order information such as covariance, anisotropy, and directional relationships.
• They leverage a Lie-group structure through matrix logarithm and exponential mappings to define well-posed sheaf operators and restriction maps that preserve the SPD manifold’s structure.
• Empirical studies show that combining SPD geometry with sheaf convolution yields state-of-the-art performance on molecular tasks and enhances resistance to oversmoothing in deep networks.

Searching arXiv for the cited paper and closely related sheaf/GDL context. {"query":"(Peng et al., 22 Apr 2026) Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning","max_results":5} I found the paper metadata and abstract on "Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning" (Peng et al., 22 Apr 2026), published 2026-04-22. {"query":"arXiv sheaf neural networks graph sheaf Laplacian cellular sheaf neural networks", "max_results": 10} Relevant related arXiv results include work on sheaf neural networks and sheaf Laplacians, which provide the broader graph-sheaf context for (Peng et al., 22 Apr 2026). SPD-valued sheaves are a graph-based geometric learning formalism in which the stalks attached to vertices and edges are taken to be points on the manifold of symmetric positive definite matrices, $\mathrm{SPD}_n$, rather than Euclidean vector spaces. In "Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning" (Peng et al., 22 Apr 2026), this construction is used to combine two previously separate ideas: sheaf neural networks, which permit edge-specific transformations through learned restriction maps, and SPD-valued representations, which encode second-order geometric structure such as covariance, anisotropy, and relationships between directions. The resulting framework defines sheaf operators natively on the SPD manifold via a Lie-group structure induced by matrix logarithm and exponential, and is presented as both a theoretical generalization of Euclidean sheaves and a practical message-passing mechanism for molecular representation learning.

1. Graph sheaves and the move from vector fibers to SPD fibers

In the graph-learning setting, a cellular sheaf on an undirected graph $G=(V,E)$ assigns a feature space to each vertex and each edge, together with restriction maps that compare vertex features with edge features. The standard definition recalled in (Peng et al., 22 Apr 2026) is a cellular sheaf $(G,\mathcal F)$ consisting of a vector space $\mathcal F(v)$ for each vertex, a vector space $\mathcal F(e)$ for each edge, and a linear restriction map $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$ for each incident pair $v\in e$. In classical sheaf neural networks, these restriction maps are learned so that different edges transform information differently, providing a mechanism for heterogeneity that is absent from shared message-passing operators.

The Euclidean coboundary operator measures inconsistency across an oriented edge $e=(u,v)$ by

$$\delta(\mathbf{x})_e = \mathcal{F}_{v \to e} \mathbf{x}_v - \mathcal{F}_{u \to e} \mathbf{x}_u,$$

and the associated sheaf Laplacian is $\mathbf L_\mathcal F=\delta^\top \delta$. Its kernel corresponds to global sections, namely configurations that are consistent across all edge incidences.

SPD-valued sheaves replace these vector-space stalks with $G=(V,E)$0. The construction described in (Peng et al., 22 Apr 2026) assigns

• $G=(V,E)$1 to each node,
• $G=(V,E)$2 to each edge.

This shift is motivated by the claim that many graph and molecular tasks require matrix-valued representations rather than purely vector-valued ones. Whereas vectors encode first-order information such as directions and gradients, SPD matrices encode second-order structure, including covariance-like relations, anisotropy, and local geometric context. A common limitation of prior sheaf neural network formulations, as stated in (Peng et al., 22 Apr 2026), is that they remain confined to vector spaces and therefore cannot propagate matrix-valued features.

2. Geometric basis: $G=(V,E)$3 as target space and Lie group

The central geometric motivation for SPD-valued sheaves is that many relevant observables are inherently second-order. The paper emphasizes molecular examples, including bond orientations, angles between directions, planarity of rings, local anisotropy, and covariance of local directional structure (Peng et al., 22 Apr 2026). In this setting, an outer product of a direction with itself yields an effectively rank-1 SPD-like object, while propagation can transform that object into a full-rank matrix carrying richer geometric information.

The construction depends on the fact that $G=(V,E)$4 admits a global logarithm/exponential parametrization at the identity, which induces an Abelian Lie group structure: $G=(V,E)$5 The identity is $G=(V,E)$6, and the inverse is

$G=(V,E)$7

This group law is the key step that makes sheaf operators well-posed on a nonlinear manifold. Operations are carried out additively in the Lie algebra $G=(V,E)$8 through $G=(V,E)$9, and the result is mapped back to $(G,\mathcal F)$0 through $(G,\mathcal F)$1. The paper states that this works under both the affine-invariant metric and the Log-Euclidean metric because at the identity the Riemannian exponential and logarithm coincide with the matrix exponential and logarithm (Peng et al., 22 Apr 2026).

The restriction maps in the SPD setting are not arbitrary linear maps. They are orthogonal congruence maps,

$(G,\mathcal F)$2

which preserve the SPD manifold and are proved in (Peng et al., 22 Apr 2026) to be isometries under both AIRM and Log-Euclidean metrics. This preserves manifold structure while retaining the edge-specific transformation principle that characterizes sheaf-based message passing.

3. Operators, consistency, and the SPD sheaf Laplacian

Because direct subtraction of manifold points is unavailable, the SPD coboundary operator is defined through logarithmic coordinates. For an oriented edge $(G,\mathcal F)$3,

$(G,\mathcal F)$4

This is the SPD analogue of the Euclidean coboundary, now valued in the Lie group defined by $(G,\mathcal F)$5 (Peng et al., 22 Apr 2026).

A structural property proved in the paper is the homomorphism-style identity

$(G,\mathcal F)$6

This gives the coboundary a form of linearity when the additive structure is understood in logarithmic coordinates rather than in ambient Euclidean space.

To define an adjoint, the framework introduces a pairing through the logarithm at the identity: $(G,\mathcal F)$7 Using a Green-identity argument and an incidence sign index $(G,\mathcal F)$8 depending on edge orientation, the adjoint is written as

$(G,\mathcal F)$9

The SPD sheaf Laplacian is then defined by composition,

$\mathcal F(v)$0

with explicit action

$\mathcal F(v)$1

The notion of global section is correspondingly multiplicative. A $\mathcal F(v)$2-cochain is globally consistent when its coboundary is the identity element $\mathcal F(v)$3 on every edge, equivalently when it lies in the kernel of the SPD Laplacian. The paper states

$\mathcal F(v)$4

Thus the Hodge-type interpretation survives, but “zero inconsistency” in Euclidean sheaves becomes “identity inconsistency” in SPD sheaves (Peng et al., 22 Apr 2026).

4. Expressive power and strict generalization of Euclidean sheaves

A principal theoretical claim of (Peng et al., 22 Apr 2026) is that SPD-valued sheaves are strictly more expressive than Euclidean sheaves. The bridge from Euclidean sections to SPD sections is given by the embedding

$\mathcal F(v)$5

This maps a vector to an effectively rank-1 SPD matrix.

The paper further states that Euclidean restriction maps lift canonically to SPD congruence maps,

$\mathcal F(v)$6

and that Euclidean consistency implies SPD consistency: $\mathcal F(v)$7

The formal strict generalization theorem is

$\mathcal F(v)$8

Every Euclidean global section embeds into an SPD global section, but the inclusion is proper. The proof outline reported in (Peng et al., 22 Apr 2026) has two components. The inclusion follows because congruence-transformed rank-1 matrices match edgewise when the original Euclidean section is consistent. Strictness follows because there exist SPD global sections not representable in the form $\mathcal F(v)$9. In particular, matrices with more than two distinct eigenvalues are outside the image of $\mathcal F(e)$0.

The significance of this theorem is conceptual as well as algebraic. It identifies a precise sense in which SPD sheaves enlarge the harmonic or globally consistent configuration space. This suggests that second-order structure is not merely a different parametrization of vector-valued information, but an extension that admits qualitatively new consistent configurations.

5. Neural architecture and message passing on the SPD manifold

The neural realization in (Peng et al., 22 Apr 2026) is a dual-stream architecture designed for molecular prediction. The geometric stream processes $\mathcal F(e)$1D coordinates using SPD sheaf convolution, while the semantic stream processes atom, bond, and categorical chemistry features with a standard GNN.

In the geometric stream, coordinates $\mathcal F(e)$2 are centered and normalized: $\mathcal F(e)$3 The initial SPD feature is

$\mathcal F(e)$4

This is effectively rank-1 and initially encodes directional information only.

The geometric update rule uses Lie-group multiplication with the SPD Laplacian response: $\mathcal F(e)$5 Here

$\mathcal F(e)$6

Restriction maps are learned from node features through an MLP followed by a Cayley transform so that the resulting matrices remain orthogonal: $\mathcal F(e)$7 A TgReEig-type nonlinearity is applied after each geometric update (Peng et al., 22 Apr 2026).

The semantic stream uses a GraphSAGE-like update,

$\mathcal F(e)$8

Cross-modal interaction is introduced by allowing geometry to modulate semantics: $\mathcal F(e)$9 where $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$0 is learned from both streams.

For graph-level prediction, the SPD features are pooled by

$\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$1

then vectorized in logarithmic coordinates,

$\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$2

The semantic stream yields $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$3, and the two are fused through factorized bilinear pooling: $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$4 For BACE, cross-attention fusion is used instead of bilinear fusion.

A common misconception is that sheaf neural networks are inherently restricted to Euclidean features. The construction in (Peng et al., 22 Apr 2026) directly contradicts that view by defining a sheaf convolution, Laplacian, and consistency notion natively on $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$5 rather than by projecting manifold-valued features to a Euclidean space for propagation.

6. Empirical results, depth robustness, and second-order emergence

The empirical evaluation reported in (Peng et al., 22 Apr 2026) uses MoleculeNet benchmarks. The model, termed SPD-Sheaf in the paper’s results discussion, is reported to achieve state-of-the-art on $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$6 out of $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$7 datasets and second on the remaining one.

Dataset	ROC-AUC
BBBP	77.4
BACE	89.0
ClinTox	99.4
SIDER	84.3
Tox21	80.1
HIV	80.9
MUV	82.3

The paper highlights that BBBP improves by about $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$8 absolute over the best listed baseline in its table, that ClinTox reaches $\mathcal F_{v\to e}:\mathcal F(v)\to \mathcal F(e)$9 and beats EGNN, and that the model is especially strong on geometry-sensitive tasks such as BBBP, BACE, and SIDER (Peng et al., 22 Apr 2026).

Ablations are used to separate the contributions of sheaf structure and SPD geometry. The comparison includes Euclidean SheafNN, representing sheaf structure without SPD geometry; SPD4GNN and HyperbolicGCN, representing geometry without sheaf structure; and SPD Sheaf, representing the combination. The reported conclusion is that both components matter and that the combination performs best. Additional architectural ablations remove the semantic stream, the geometric stream, or cross-modal interaction; all degrade performance, often substantially.

The depth study is used to assess robustness to oversmoothing. The paper states that sheaf methods resist oversmoothing better than standard GNNs, and that SPD sheaves are especially robust to depth. It reports that the SPD model retains about $v\in e$0 performance at $v\in e$1 layers and often performs best at shallow depth, especially around $v\in e$2 layers. This is interpreted in the paper as evidence that edge-specific sheaf propagation helps preserve distinctiveness across layers.

One of the most distinctive empirical observations is the transition from effectively rank-1 initialization to higher-rank representations after propagation. Initial node features take the form

$v\in e$3

with effective rank about $v\in e$4. After sheaf propagation, the effective rank rises substantially:

• BACE: $v\in e$5
• BBBP: $v\in e$6
• SIDER: $v\in e$7
• ClinTox: $v\in e$8
• HIV: $v\in e$9
• Tox21: $e=(u,v)$0
• MUV: $e=(u,v)$1

The paper presents this as evidence that SPD sheaf convolution transforms essentially first-order directional inputs into richer second-order, full-rank geometric representations (Peng et al., 22 Apr 2026). A plausible implication is that the formal enlargement of the global-section space has a practical correlate in the learned feature geometry: the network does not merely preserve SPD structure, but actively exploits it to construct representations unavailable to vector-valued sheaf models.