Message-Passing Topological Neural Networks

Updated 9 December 2025

Message-Passing Topological Neural Networks (TNNs) are deep learning models that generalize GNNs to higher-dimensional topological domains using multi-way message passing across nodes, edges, and cycles.
They employ specialized mechanisms like cycle-based, cellular, and simplicial attention to capture rich relational patterns and topological invariants in complex structures.
Applications in traffic forecasting, molecular analysis, and cosmology demonstrate TNNs achieve superior performance while mitigating oversquashing and enhancing network expressivity.

Message-Passing Topological Neural Networks (TNNs) constitute a class of deep learning architectures that generalize classical graph neural networks (GNNs) to operate on higher-order topological domains, such as hypergraphs, simplicial complexes, cell complexes, and combinatorial complexes. Unlike standard GNNs that propagate features solely along edges, TNNs pass messages across cells of arbitrary dimension (nodes, edges, faces, cycles), thereby capturing richer relational patterns and topological invariants. These models integrate multi-way message exchanges, topological attention, and cycle or group-based dependencies, enabling enhanced expressivity for tasks ranging from traffic forecasting to molecular property prediction and cosmological inference.

1. Foundational Principles and Mathematical Structures

Message-passing TNNs extend the graph-centric paradigm to topological domains defined over sets of cells endowed with boundary, coboundary, and adjacency relations. For a topological domain $X$ , let $X^{(r)}$ denote the set of $r$ -cells (e.g., nodes for $r=0$ , edges for $r=1$ , faces for $r=2$ , etc.). The incidence structure is encoded through boundary matrices $B_r$ (with $(B_r)_{ij} \neq 0$ if $i$ bounds $j$ ) and their transposes (coboundary operators). The interplay of these matrices yields generalized Hodge Laplacians $L_{↓, r}$ and $L_{↑, r}$ , controlling diffusion on rank- $r$ cells.

A TNN layer typically operates via:

Message computation: For each neighborhood type (boundary, coboundary, lower/upper adjacency) and target cell $x$ , messages $m_{y\to x}$ are computed using local features and possibly those of related cells.
Aggregation: Messages are combined (sum, mean, concatenation) within and across neighborhood types.
Update: The cell’s features are refreshed using learnable functions (MLP, gating) parameterized by cell type and dimension.

This structure enables information flow not just over graph edges ( $r=1$ ) but through faces, rings, cycles, and higher-order polytopes, thereby encompassing the full topology of the domain (Papillon et al., 2023).

2. Specialized Message-Passing Mechanisms and Topological Invariants

Certain TNN variants explicitly exploit cycle-based or higher-dimensional invariants for increased expressivity. The Cycle to Mixer (Cy2Mixer) model (Lee et al., 29 Jan 2024) demonstrates this approach:

Cycle basis extraction: For a graph $G$ , a 1-cycle is a closed walk with no repeated edges. A cycle basis $C_G$ spans all such cycles and corresponds to the first homology group $H_1(G;\mathbb{Q})$ .
Clique adjacency construction: A matrix $A_C$ connects node pairs $(i,j)$ lying in the same cycle, allowing message passing along cycles in addition to edges.
Three-block architecture: The Cy2Mixer layer comprises temporal, spatial (message-passing via edge adjacency $A$ ), and cycle (message-passing via $A_C$ ) blocks. Each block performs gated feature mixing, local attention, and a feed-forward step, then concatenates the results for output mixing.

The inclusion of the cycle message-passing block provably distinguishes graphs with identical tree unfoldings but different cycles, encoding homological features inaccessible to standard edge-based MPNNs. Theoretical analysis establishes that ordinary edge message passing is bounded by the 1-WL test applied to the universal covering tree, while serving messages along $A_C$ enables detection of co-cycle constraints and parity-type invariants (Lee et al., 29 Jan 2024).

3. Higher-Order Complexes, Cellular and Simplicial Messaging

TNNs operating on cell and simplicial complexes generalize even further:

Cellular Isomorphism Networks (CIN/CIN++): A 2D regular cell complex $\mathcal{C}$ contains vertices, edges, and rings, with message passing among boundary, coboundary, upper (same-dimension via co-boundary neighbor), and lower (same-dimension via shared boundary) neighborhoods. CIN++ enhances expressivity by including lower messages between rings and edges sharing a vertex, yielding direct routing and mitigating oversquashing (Giusti et al., 2023).
Simplicial Attention Networks (SAN) and Cell Attention Networks (CAN) introduce anisotropic attention mechanisms over lower/upper neighborhoods of simplices or cells, using softmax-normalized compatibility scores for each adjacency type. This design enables nuanced aggregation and improves modeling of group dependencies and cavities (Giusti, 10 Feb 2024).

Together, the multi-relational message-passing framework and attention-driven aggregation capture both localized and global topological signatures (e.g., cycles, cavities, higher Betti numbers).

4. Computational Methods and Algorithmic Implementations

TNNs utilize a variety of computational strategies tailored to their domain:

Cycle Basis Extraction (Cy2Mixer): Paton's algorithm [Commun. ACM 12(9):514–518, 1969] is employed to compute cycle bases in $O(E+N)$ time, informing the construction of $A_C$ for cycle-aware message passing (Lee et al., 29 Jan 2024).
Cell Complex Messaging (CIN++): Each cell aggregates boundary, upper, and lower messages using small constant-sized MLPs for each dimension and layer, followed by update functions. There is no parameter sharing between dimensions or layers (Giusti et al., 2023).
Attention Weights: SAN/CAN architectures calculate compatibility scores via learnable vectors and matrices followed by LeakyReLU and softmax normalization, with outputs summarizing the importance of different regions in the topological domain (Giusti, 10 Feb 2024).

Pseudocode for representative blocks is standard:

Input: H ∈ ℝ^{T×N×d_h}, adjacency A, clique adjacency A_C
Z = LayerNorm(H)·W_z                           # T×N×2d_h
[Z1, Z2] = split(Z)
Zc_hat = Z1_c ⊙ MPNN(Z2_c; A_C)                # T×N×d_h
Y_c = FC(Attn(Zc_hat))
return Y_c

5. Expressivity, Bottleneck Mitigation, and Theoretical Guarantees

Message-passing TNNs overcome several expressivity and bottleneck limitations:

Expressivity Beyond MPNN/1-WL: The inclusion of cycle blocks or higher-order cell messaging enables detection of non-tree substructures (e.g., cycles, rings) and supports distinguishing non-isomorphic domains undetectable by the 1-WL test (Lee et al., 29 Jan 2024, Sun et al., 2023).
Over-squashing Mitigation: Lower message routes in cell complexes (CIN++) and multi-relational neighborhood traversals provide shortcuts that prevent exponential decay of sensitivity between distant nodes or cells, addressing bottlenecks typical in deep local GNNs. Empirical and theoretical analysis links sensitivity bounds to the combinatorial influence matrices and the size of the hidden dimension (Giusti et al., 2023, Taha et al., 6 Jun 2025, Giusti, 10 Feb 2024).
Relational Rewiring: Topological message-passing layers can incorporate rewiring strategies by collapsing multi-relational adjacencies and adding shortcuts, thus alleviating bottlenecks in both graphs and lifted complexes (Taha et al., 6 Jun 2025).

6. Empirical Benchmarks and Application Domains

TNNs consistently achieve state-of-the-art performance on diverse benchmarks:

Spatio-temporal Traffic Forecasting (Cy2Mixer): On PEMS04, the addition of cycle blocks reduced MAE from 18.81 to 18.14 and RMSE from 30.65 to 30.02. Across six benchmarks, Cy2Mixer achieved either top or second-best MAE/RMSE/MAPE on 14 of 18 tasks, with notable improvements for nodes connected only via cycle adjacency (Lee et al., 29 Jan 2024).
Molecular Property Prediction (CIN++): On ZINC-Subset, CIN++ achieved MAE of 0.077±0.004 (best) and on ZINC-Full, 0.027±0.007 (second-best). CIN++ also exceeded prior methods on long-range peptide tasks and bioinformatics datasets (Giusti et al., 2023).
Synthetic Topological Tasks (SMP/TNN): SMP reached nearly 100% accuracy in cycle detection and outperformed GIN, GAT, PNA, DGN on multi-task regression and molecular benchmarks (Vignac et al., 2020).
Cosmological Parameter Estimation: ClusterTNN and FullTNN demonstrated 22–60% improvements in key cosmological parameters over conventional GNNs by leveraging combinatorial complex messaging and enforcing $E(3)$ -invariance (Lee et al., 29 May 2025).

7. Current Limitations and Future Directions

Challenges persist in scaling message-passing TNNs to large domains due to memory and computational complexity—particularly for higher-order P-tensor networks and fully regular complexes. Some current limitations include:

Memory/Compute Overheads: Storing and processing $n^2$ or $n^K$ dimension tensors per layer (for SMP/P-tensors) quickly becomes prohibitive for large graphs (Vignac et al., 2020, Sun et al., 2023).
Expressivity/Generalization Trade-offs: Increasing hidden width counters oversquashing but may reduce generalization; universality proofs rely on full basis function implementation, which is approximated in practice.
Domain Construction: Embedding real data into complexes or combinatorial domains can be non-trivial, especially when forcing artificial high-order relationships (e.g. via clique expansions) (Papillon et al., 2023).

Research is active on developing lightweight, universal equivariant layers, scalable sampling for complex construction, dynamic domains, and unified spectral/spatial frameworks. Theoretical analysis linking topological descriptors, effective resistance, and sensitivity bounds continues to inform new architectures and their utility in applications from chemistry and biology to network science and cosmology.

References:

"Enhancing Topological Dependencies in Spatio-Temporal Graphs with Cycle Message Passing Blocks" (Lee et al., 29 Jan 2024).
"CIN++: Enhancing Topological Message Passing" (Giusti et al., 2023).
"Building powerful and equivariant graph neural networks with structural message-passing" (Vignac et al., 2020).
"Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks" (Papillon et al., 2023).
"Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing" (Taha et al., 6 Jun 2025).
"Cosmology with Topological Deep Learning" (Lee et al., 29 May 2025).
"Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions" (Giusti, 10 Feb 2024).
"P-tensors: a General Formalism for Constructing Higher Order Message Passing Networks" (Sun et al., 2023).