---
title: Higher-order Graph Neural Networks
url: https://www.emergentmind.com/topics/higher-order-graph-neural-networks-hognns
type: topic
---

# Higher-order Graph Neural Networks

Higher-order Graph Neural Networks (HOGNNs) comprise a broad class of architectures that explicitly encode and leverage polyadic interactions—relations among more than two nodes—into the neural computation on graphs. These models move beyond simple pairwise edges, accessing and propagating information over larger-scale combinatorial structures such as hyperedges, node tuples, induced subgraphs, motifs, simplices, and cells. The explicit modeling of higher-order relationships fundamentally increases both the expressiveness and the structural inductive bias of GNNs, often aligning them with—and in advanced cases surpassing—the k-dimensional Weisfeiler-Leman (k-WL) graph isomorphism hierarchy [2406.12841].

## 1. Mathematical Foundations and Higher-Order Message Passing

A HOGNN is fundamentally characterized by its use of higher-order combinatorial objects and its associated message-passing paradigm. For a standard graph $G=(V,E)$ and adjacency matrix $A \in \{0,1\}^{n \times n}$, higher-order constructs take the form of adjacency tensors $A^{(k)} \in \{0,1\}^{n \times \cdots \times n}$ (with $k$ modes), whose entries $A^{(k)}_{i_1,\ldots,i_k}$ indicate polyadic relations, such as membership in a hyperedge or a simplex. The most general HOGNN update at layer $\ell+1$ is:

\[
h_v^{(\ell+1)} = \phi\bigl(\bigl\{\, \psi(h_{u_1}^{(\ell)},\dots,h_{u_k}^{(\ell)},\,A^{(k)}_{u_1\cdots u_k v})\,|\, (u_1,\dots,u_k)\in\mathcal{N}^{(k)}(v) \bigr\}\bigr)
\]

where $\mathcal{N}^{(k)}(v)$ collects all $k$-tuples relevant to $v$ (e.g., all $k$-simplices containing $v$), and $\psi$ and $\phi$ are user-specified message and aggregation functions. This framework subsumes models built on hypergraphs, node tuples, motifs, subgraphs, simplicial or cellular complexes [2406.12841].

Specializations include:

- Hypergraph GNNs: Messages between nodes and hyperedges (incidence-based propagation).
- Simplicial/Cell Complex GNNs: Hierarchical message passing across lower/upper adjacencies, boundary/coboundary relations.
- Tuple or subgraph-based HOGNNs: Explicit message-passing over node $k$-tuples or induced subgraphs, as in $k$-GNNs [1810.02244], Subgraph GNNs, or recursive pooling architectures [2012.03174].

## 2. Core Taxonomy of HOGNN Models

Recent comprehensive analyses [2406.12841] distinguish HOGNN families by two axes:

- **Higher-Order Graph Data Model (HOGDM):** The combinatorial objects governing relations—hypergraphs (arbitrary subsets), simplicial complexes (down-closed collections), cell complexes (hierarchical non-simplex cells), node-tuple collections, subgraph collections, motifs, and nested structures.
- **Message Passing Wiring:** The set of communication channels (adjacency structures): node–hyperedge incidence (IMP), boundary/co-boundary (BAMP), down-neighboring for $k$-tuples (DAMP), multi-hop or motif-based adjacency.

This taxonomy clarifies the distinctions among classical hypergraph GNNs [1809.09401], motif/structural encodings [1809.07697, 2209.03473], k-GNNs [1810.02244], subgraph-based approaches, topological (simplicial/cellular) GNNs [2402.06908], path-based aggregation [2002.10413], and mixed paradigms such as hybrid low/high-order convolutions [1908.00673]. The motif-convolution approach, for example, defines higher-order propagation via parameterized motif-matrices, allowing for node-specific selection and attention among motifs and diffusion scales [1809.07697, 2209.03473].

## 3. Representative HOGNN Architectures and Their Expressivity

Several canonical HOGNN architectures exemplify the spectrum of modeling higher-order relations:

| Model Type       | Combinatorial Domain     | Expressivity / Theoretical Bound                              |
|------------------|--------------------------|-------------------------------------------------------------|
| Hypergraph GNN   | Hyperedges (arbitrary)   | Generalizes GCN; can model group-wise relations [1809.09401]|
| Simplicial/Cell GNN | Simplicial/cell complexes | Capable of Hodge homology/CWL-level reasoning [2402.06908]   |
| $k$-GNN (tuple-based) | $k$-node tuples         | Matches $k$-WL hierarchy [1810.02244]                        |
| Path-GNN/Path-MPNN| Simple paths (length-$k$)| Exceeds 1-WL for $k>1$; includes geometric invariants [2002.10413]|
| Motif-GNN        | Chosen motifs (triangles, ...)| Diffusion via motif structure [1809.07697]               |
| Derivative GNN (HOD-GNN) | Node feature derivatives | Expressivity matches subgraph GNN to $k$-WL [2510.02565]     |

The $k$-GNN of Morris et al. [1810.02244] uses node-$k$-tuple features, with message passing that mimics $k$-WL color refinement; it achieves expressivity equal to $k$-WL, strictly subsuming standard GNNs. The High-Order Derivative GNN (HOD-GNN) notably achieves the full expressivity of $k$-order subgraph aggregation GNNs by leveraging high-order partial derivatives of a base MPNN with respect to node features, integrating Taylor expansion theory for marked node identification [2510.02565].

Recursive pooling approaches, such as RNP-GNN [2012.03174], can count subgraphs of size $k$ and interpolate between simple message passing and full $k$-WL, trading complexity for targeted expressive power.

## 4. Algorithmic Frameworks and Implementations

Practical instantiations of HOGNNs face significant computational and memory challenges due to the exponential growth in the number of higher-order objects ($O(n^k)$ for $k$-tuples, $O(n^{d+1})$ for $d$-simplices). To address this:

- Sparse and recursive pooling architectures exploit graph sparsity to reduce computational overhead ([2510.02565], [2012.03174]).
- Tools such as PyTorch Geometric High Order (PyGHO) [2311.16670] provide unified abstractions for high-order data structures (Masked/Sparse Tensor), message passing across arbitrary dimensions, and high-order batching, enabling fast prototyping and minimizing boilerplate for $k$-tuple and subgraph-based architectures.
- Specialized operators and encoders (e.g., IGN for derivative tensors) support expressivity-aligned design.

## 5. Empirical Performance, Use Cases, and Limitations

Empirical studies report that HOGNNs yield consistent and often substantial gains over classical GNNs in tasks where higher-order structure is significant. Examples include:

- Molecular property prediction: Explicit geometric and substructure awareness yields state-of-the-art results in chemical datasets, outperforming both GCN and MPNN baselines [2002.10413, 2010.08516, 2510.02565].
- Social, collaboration, and biological networks: Hypergraph and simplicial models capture complex group interactions and community structure, leading to higher clustering fidelity and improved node classification accuracy [1809.09401, 2209.03473].
- Motif-based and topological architectures improve robustness, clustering quality, and classification performance, especially on tasks sensitive to network motifs, cycles, or higher homology [1809.07697, 2409.08217].

Trade-offs center on computational cost versus expressivity. Complete $k$-GNN or subgraph GNN architectures are typically feasible for small $k$ and graphs but become impractical otherwise, motivating sampling, recursive, or sparsity-aware reductions.

## 6. Theoretical Insights and Recent Directions

Contemporary research has clarified key theoretical properties of HOGNN families:

- The alignment of k-GNN, Subgraph GNNs, and HOD-GNNs with the $k$-WL hierarchy defines a sequence of strictly increasing expressivity classes [2510.02565, 1810.02244].
- Invariant Graphon Networks extend these constructions to graph limits, showing that expressivity and transferability hold in dense graphon regimes, with explicit universal approximation results [2503.14338].
- Topological neural models (simplicial/cellular) demonstrate improved information flow and mitigate over-squashing by providing higher-dimensional “shortcuts” in computational graphs [2402.06908].
- Path-, motif-, and multi-scale architectures enable flexible modeling of both local and global higher-order dependencies [1809.07697].

Open challenges concern flexible, data-driven construction of HOGDMs, scalable and memory-efficient computation, extension to dynamic and heterogeneous domains, and systematic benchmarking and comparison [2406.12841].

## 7. Summary Table of Major HOGNN Families

| Family         | Principal Object      | Message Passing Domain      | Expressivity (relative to WL)         | Computational Cost          |
|----------------|----------------------|----------------------------|---------------------------------------|----------------------------|
| Hypergraph GNN | Hyperedges           | Node–hyperedge incidence   | Exceeds 1-WL in group interaction     | Linear in #nodes+edges     |
| k-GNN          | Node $k$-tuples      | $k$-tuple adjacency        | Matches $k$-WL                        | $O(n^k)$ per layer         |
| Subgraph GNN   | Induced subgraphs    | Subgraph message passing   | Matches $k$-WL (for subgraphs of size $k$)| $O(n^k)$                  |
| Simplicial GNN | Simplices            | Boundary/coboundary        | Capable of Hodge/CWL inference        | Linear in #simplices       |
| Motif-GNN      | Motif instances      | Motif-based adjacency      | Surpasses 1-WL                        | Motif enumeration + GCN    |
| HOD-GNN        | Node feature derivatives | Node + derivative encoding| Matches k-OSAN/Subgraph GNN ($k$-WL)  | Sparse: $O(d_{max} n s_{t-1})$|
| Path-GNN       | Simple paths         | k-path aggregation         | Exceeds 1-WL, context- and geometry-aware| Path enumeration (pruning for large $k$)|
| Topological NN | Simplicial/cell complexes | Multi-dimensional MP   | Beyond 1-WL, homology/topology-aware  | Depends on complex size    |

In conclusion, Higher-Order Graph Neural Networks provide a rich and theoretically grounded toolkit for modeling, reasoning, and learning over graph-structured data enriched with polyadic, combinatorial, and topological structure, with architectures and theoretical underpinnings that both subsume and extend the classic GNN paradigm [2406.12841, 2510.02565, 1810.02244, 2402.06908, 2510.02565, 2503.14338].

Source: https://www.emergentmind.com/topics/higher-order-graph-neural-networks-hognns