High-Order Derivative GNN

• HOD-GNN is a framework that uses higher-order derivative tensors to capture multi-hop graph structures and boost expressivity beyond traditional MPNNs.
• It integrates analytic derivative modules with message passing, employing Jacobians, Hessians, and specialized encoders to maintain computational sparsity.
• The design bridges sensitivity analysis with diffusion processes and operator-theoretic methods to enhance structural encoding and model performance.

High-Order Derivative GNN (HOD-GNN) denotes a class of graph neural network constructions in which higher-order derivatives are treated as primary carriers of structural information. In the most specific usage, HOD-GNN is an architecture that enhances the expressivity of Message Passing Neural Networks (MPNNs) by computing high-order derivatives of a base model with respect to input node features, encoding those derivative tensors into node representations, and processing them with a downstream GNN (Eitan et al., 2 Oct 2025). In broader usage, the same label is applied to graph models that operationalize higher-order derivatives of graph signals through Sobolev norms, polynomial Laplacian diffusion, or discrete differential operators such as boundary, coboundary, Hodge, and incidence operators on lifted higher-order domains (Giraldo et al., 2024, Li et al., 2023, Besta et al., 2024). Across these formulations, the common objective is to expose multi-hop or polyadic structure while retaining trainability and, where possible, sparsity.

1. Conceptual scope and nomenclature

The term “HOD-GNN” is not used uniformly across the literature. One line of work uses it as the explicit name of a derivative-tensor architecture built on top of a base MPNN. Other papers present operator-theoretic or geometric constructions that can be interpreted as high-order derivative GNNs even when the term itself is not the paper’s formal model name. This makes the expression both precise in a narrow sense and umbrella-like in a broader sense.

Usage of “HOD-GNN”	Core mechanism	Representative source
Explicit derivative architecture	High-order derivatives of a base MPNN wrt node features are encoded and passed to a second GNN	(Eitan et al., 2 Oct 2025)
Sparse Sobolev realization	Sobolev norms and Hadamard powers emulate higher-order graph derivatives while preserving sparsity	(Giraldo et al., 2024)
Diffusion realization	Polynomial Laplacian diffusion with a fidelity term; HiD-Net is a $K=2$ instance	(Li et al., 2023)
Taxonomic derivative family	Boundary, coboundary, Hodge, and incidence operators on simplicial, cellular, and hypergraph domains	(Besta et al., 2024)

A common point of confusion is to equate HOD-GNN with any higher-order GNN. The taxonomic survey “Demystifying Higher-Order Graph Neural Networks” states that it does not explicitly define or name a model called “High-Order Derivative GNN,” but identifies derivative-based higher-order GNNs with families built from discrete differential operators on higher-order structures, especially simplicial complex GNNs, cell complex GNNs, hypergraph incidence models, and spectral or high-order diffusion operators (Besta et al., 2024). This suggests that “HOD-GNN” is best understood as a derivative-centered viewpoint rather than a single architectural template.

2. Derivatives of a base MPNN

In the explicit formulation, HOD-GNN starts from a differentiable base MPNN on a graph $G=(A,X)$, with node features $X\in\mathbb{R}^{n\times d}$ and hidden states $h_v^{(0)}=x_v$. A generic message-passing layer is written as

$$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$

where $\Box$ is a permutation-invariant aggregator such as sum, mean, or max. A graph-level readout may be written as $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$ (Eitan et al., 2 Oct 2025).

The distinctive step is to differentiate hidden states or graph outputs with respect to the input features. For an intermediate hidden matrix $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$, the paper considers Jacobians, Hessians, and general higher-order derivative tensors:

$$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$

and

$$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$

These derivatives may be local, $G=(A,X)$0-hop, or global in scope, and they may include mixed partials across several source nodes (Eitan et al., 2 Oct 2025).

The node-indexed conventions used in the paper make the structure explicit. For order $G=(A,X)$1,

$G=(A,X)$2

and for graph outputs,

$G=(A,X)$3

For $G=(A,X)$4-HOD-GNN, mixed derivatives across $G=(A,X)$5 distinct nodes are organized into tensors $G=(A,X)$6 and $G=(A,X)$7 indexed by $G=(A,X)$8 and derivative orders $G=(A,X)$9 (Eitan et al., 2 Oct 2025).

These derivative tensors are not used directly. They are encoded into compact node features by permutation-equivariant modules. The output derivatives are processed by a DeepSets encoder $X\in\mathbb{R}^{n\times d}$0, node-pair or multi-index tensors are processed by $X\in\mathbb{R}^{n\times d}$1, implemented as a $X\in\mathbb{R}^{n\times d}$2-IGN for $X\in\mathbb{R}^{n\times d}$3-HOD or a $X\in\mathbb{R}^{n\times d}$4-IGN for $X\in\mathbb{R}^{n\times d}$5-HOD, and the resulting features are concatenated with the base embedding:

$X\in\mathbb{R}^{n\times d}$6

A practical normalization aggregates intermediate layers with factorial scaling,

$X\in\mathbb{R}^{n\times d}$7

and derivative encoders may include per-order scaling such as $X\in\mathbb{R}^{n\times d}$8-based normalization (Eitan et al., 2 Oct 2025).

The resulting pipeline is fully differentiable: a base MPNN computes $X\in\mathbb{R}^{n\times d}$9 and $h_v^{(0)}=x_v$0; an analytic derivative module computes derivative tensors up to order $h_v^{(0)}=x_v$1; encoders $h_v^{(0)}=x_v$2 and $h_v^{(0)}=x_v$3 produce node-level derivative-aware features; and a downstream GNN predicts the final target (Eitan et al., 2 Oct 2025).

3. Analytic derivative message passing and computational structure

HOD-GNN does not rely on brute-force higher-order automatic differentiation at runtime. Its derivative module uses closed-form chain-rule recurrences whose sparsity pattern mirrors message passing itself. For first-order derivatives,

$h_v^{(0)}=x_v$4

and for sum aggregation,

$h_v^{(0)}=x_v$5

This recurrence makes the sensitivity interpretation explicit: derivatives propagate along edges and quantify how perturbations of source nodes affect target nodes (Eitan et al., 2 Oct 2025).

Higher orders are handled by multivariate Faà di Bruno. For a pointwise nonlinearity $h_v^{(0)}=x_v$6 applied to $h_v^{(0)}=x_v$7,

$h_v^{(0)}=x_v$8

where $h_v^{(0)}=x_v$9 ranges over partitions of $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$0. In implementation terms, the aggregation derivative remains linear and sparse, while the pointwise update derivative applies Faà di Bruno locally per node (Eitan et al., 2 Oct 2025).

For GIN, the derivative aggregation has a particularly simple form. If

$$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$1

then

$$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$2

The linear layer is then differentiated, followed by the pointwise nonlinearity via Faà di Bruno. This is the core derivative message-passing mechanism (Eitan et al., 2 Oct 2025).

The computational rationale is sparsity. For $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$3-HOD mixed tensors, a dense representation would have $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$4 entries, but the derivative tensors remain sparse when depth is relatively small or the graph is sparse. The paper states that the number of nonzeros in $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$5 is at most $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$6, the time to compute $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$7 from $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$8 is $$h_v^{(l+1)}=\phi^{(l)}\big(h_v^{(l)},m_v^{(l)}\big),\qquad m_v^{(l)}=\Box_{u\in N(v)}\psi^{(l)}\big(h_v^{(l)},h_u^{(l)},e_{uv}\big),$$9, and encoding can be performed in time and space $\Box$0. By comparison, $\Box$1-OSAN and $\Box$2-IGN require $\Box$3 time and space (Eitan et al., 2 Oct 2025).

A canonical motivation is triangle counting. For a $\Box$4-layer linear GCN with identity activations and $\Box$5, one has $\Box$6, so summing diagonal derivatives yields $\Box$7, the number of triangles. HOD-GNN generalizes this mechanism from linear adjacency powers to nonlinear MPNNs and higher-order derivatives (Eitan et al., 2 Oct 2025).

4. Expressivity, marking, and relations to WL, subgraph GNNs, and structural encodings

The central expressivity claim is that derivatives provide a natural way to enhance the expressive power of MPNNs. Standard MPNNs are bounded by $\Box$8-WL, but derivative signals encode “what would change if we infinitesimally marked one or more nodes.” Because many activation functions are analytic, the effect of discrete marking can be approximated arbitrarily well by a Taylor expansion whose coefficients are precisely these derivatives. This creates a direct bridge between derivative tensors and marking-based symmetry breaking (Eitan et al., 2 Oct 2025).

The theoretical result is formulated through equivalence to subgraph GNNs. For any finite set of graphs and any $\Box$9-OSAN model with analytic activation, there exists a $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$0-HOD-GNN that approximates its outputs arbitrarily well. The consequence stated in the paper is that $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$1-HOD-GNN can distinguish certain pairs of non-isomorphic graphs that are indistinguishable by folklore $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$2-WL and can compute homomorphism counts of $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$3-apex forest graphs. The family therefore forms an expressivity hierarchy aligned with WL: $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$4 already exceeds $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$5-WL and RWSE-augmented MPNNs, while mixed derivatives over $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$6 nodes, together with appropriate IGN encoders, match and in known cases exceed folklore $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$7-WL (Eitan et al., 2 Oct 2025).

The relation to structural encodings is also explicit. Even with ReLU and only first-order derivatives, $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$8-HOD-GNN strictly subsumes MPNNs with Random Walk Structural Encodings (RWSE). The paper states that there exists a base MPNN initialization making diagonal derivatives equal to RWSE diagonals for multiple walk lengths, allowing the downstream GNN to reproduce RWSE-enhanced outputs exactly; it also gives quartic vertex-transitive graphs $y_{\mathrm{out}}=\rho(\{h_v^{(L)}\mid v\in V\})$9 and $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$0 as a separating example for suitable derivative filters (Eitan et al., 2 Oct 2025).

The same derivative quantities connect HOD-GNN to oversquashing, over-smoothing, and explainability. Entries such as $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$1 measure path-dependent influence and act as an information-flow metric. Mixed partials quantify synergy among sets of source nodes. Derivatives are also the objects used in gradient-based explanation methods such as Sensitivity, Guided Backpropagation, and Grad-CAM. HOD-GNN converts these sensitivities into trainable node features rather than treating them only as post hoc diagnostics (Eitan et al., 2 Oct 2025).

5. Sobolev and diffusion realizations of high-order derivatives on graphs

A broader operator-theoretic realization appears in Sparse Sobolev GNN (S2-GNN), which is explicitly framed as a High-Order Derivative GNN in the synthesis provided with the paper. Its starting point is the Sobolev norm of a graph signal,

$H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$2

with

$H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$3

As $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$4 grows, high frequencies are increasingly penalized, which is interpreted as a high-order derivative effect. The key observation is that regular powers $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$5 and Hadamard powers $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$6 behave similarly in the spectrum when normalized, so Hadamard powers can serve as sparse surrogates for dense higher-order filters. The sparse Sobolev term

$H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$7

preserves the support of $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$8, and by the Schur product theorem is positive definite for $H^{(l)}=[h_v^{(l)}]_v\in\mathbb{R}^{n\times d'}$9. S2-GNN builds branches

$$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$0

then fuses them by learnable coefficients or an MLP. The paper states that per-branch complexity remains $$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$1, giving per-layer complexity $$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$2 and total complexity $$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$3 for $$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$4 layers. It also reports that regular powers of $$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$5 become fully dense by $$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$6 on 20News, whereas Hadamard powers keep the sparsity constant (Giraldo et al., 2024).

A second operator-theoretic line arises from diffusion. In “A Generalized Neural Diffusion Framework on Graphs,” a generic HOD-GNN is written as

$$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$7

with explicit Euler discretization

$$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$8

This yields steady state

$$J_v^{(l)}=\frac{\partial h_v^{(l)}}{\partial x}\in\mathbb{R}^{d'\times (n\cdot d)},\qquad H_v^{(l)}=\frac{\partial^2 h_v^{(l)}}{\partial x^2}\in\mathbb{R}^{d'\times (n\cdot d)\times (n\cdot d)},$$9

Within that framework, HiD-Net is the $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$0 case, motivated by the empirical observation of monophily: labels of high-order neighbors, notably $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$1-hop neighbors, tend to be consistent even when $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$2-hop neighbors are heterophilous. The high-order diffusion equation mixes first- and second-order terms and is equivalent to a second-order random walk with teleport. The paper gives the parameterization $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$3, $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$4, and $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$5, so the signed second-order term induces a band-pass steady-state filter. Under $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$6, the diffusion-based message passing converges linearly, and the paper proves a non-oversmoothing guarantee stating that if $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$7, then $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$8 as $$D_v^{(k,l)}=\frac{\partial^k h_v^{(l)}}{\partial x^{\otimes k}}\in\mathbb{R}^{d'\times (n\cdot d)^k}.$$9 (Li et al., 2023).

These two realizations are derivative-based in different senses. S2-GNN interprets higher-order derivatives through Sobolev penalization of graph frequencies while preserving sparsity with Hadamard powers. HiD-Net interprets them through polynomial Laplacian dynamics with a fidelity term and a direct random-walk semantics over $G=(A,X)$00-hop and $G=(A,X)$01-hop transitions. Together they show that HOD-GNN is not restricted to derivative tensors of a base MPNN.

6. Derivative-based higher-order GNNs in the HOGNN taxonomy

The taxonomic survey of higher-order GNNs places derivative-based constructions inside a broader class of higher-order graph learning methods. In that taxonomy, the most relevant families are simplicial complex GNNs, cell complex GNNs, hypergraph GNNs, and spectral or high-order diffusion models. The underlying reason is that discrete derivatives on higher-order domains are expressed by boundary and coboundary operators, incidence matrices, and the associated Laplacian-like compositions (Besta et al., 2024).

For hypergraphs, the fundamental object is the incidence matrix $G=(A,X)$02, and a normalized hypergraph convolution is written as

$G=(A,X)$03

This corresponds to incidence-based message passing (IMP), in which messages flow from nodes to hyperedges and back. For simplicial and cellular complexes, the relevant formalism is boundary-adjacency based message passing (BAMP), with neighborhoods defined by boundaries, co-boundaries, lower adjacencies, and upper adjacencies. Representative models include S2CNN, which mixes normalized forms of $G=(A,X)$04, $G=(A,X)$05, $G=(A,X)$06, and $G=(A,X)$07; SAT, which adds orientation-aware attention over upper and lower neighborhoods; CCNN, which uses cell co-boundary adjacency; and CW Networks, which aggregate over boundaries and upper adjacencies (Besta et al., 2024).

This derivative perspective usually requires a lifting step. A plain graph may be lifted to a simplicial complex by clique completion or to a cell complex by attaching cells to cliques or cycles. The survey emphasizes that such liftings preserve isomorphisms under the stated constructions, but may cause combinatorial growth in the number of simplices or cells. That trade-off differentiates derivative-based geometric models from node-tuple architectures. Node-tuple models scale as $G=(A,X)$08 messages per iteration, whereas simplicial or cellular models can be substantially cheaper if the lifted complex remains moderate in size (Besta et al., 2024).

The same survey states that derivative-based simplicial and cellular models can surpass $G=(A,X)$09-WL: MPSN and CW Networks “attain a strictly higher expressiveness than 1-WL and are provably not less powerful than 3-WL,” with appropriate liftings. This places discrete differential operators alongside subgraph methods and node-tuple methods as one of the main routes to higher-order expressivity, but with a distinct topological inductive bias focused on flows, orientations, boundaries, faces, and cycles (Besta et al., 2024).

7. Empirical record, limitations, and recurring misconceptions

The explicit derivative-tensor version of HOD-GNN has been evaluated on graph regression, molecular classification, long-range graph benchmarks, and synthetic substructure counting. Reported results include approximately $G=(A,X)$10 MAE on ZINC-12K, $G=(A,X)$11 ROC-AUC on OGB molhiv, $G=(A,X)$12 on moltox21, $G=(A,X)$13 on molbace, $G=(A,X)$14 AP on Peptides-func, and $G=(A,X)$15 MAE on Peptides-struct. On substructure counting, with analytic activations and higher derivative orders, normalized MAE is often below $G=(A,X)$16 on several motifs. The paper further states that HOD-GNN improves GCN, GIN, and GPS consistently on ZINC, and that its runtime and memory are favorable compared with full-bag subgraph GNNs, using less than half GPU memory versus subgraph policies while retaining comparable or better per-epoch time (Eitan et al., 2 Oct 2025).

The operator-theoretic variants show a comparable pattern. S2-GNN is reported to achieve best or second-best accuracy on Cancer-B, Cancer-M, HAR, and Isolet on constructed graphs, and to top Cora at $G=(A,X)$17 and OGBN-proteins at $G=(A,X)$18, while remaining competitive on Citeseer and Pubmed. The paper also reports that Hadamard powers outperform simply adding more branches without elementwise powers, and that the sparse Sobolev norm outperforms the regular norm on all but one dataset, namely 20News (Giraldo et al., 2024). HiD-Net, evaluated as a second-order diffusion realization, is reported to attain the best results among diffusion GNN baselines on Cora, Citeseer, Pubmed, Chameleon, Squirrel, Actor, and ogbn-arxiv, with examples including $G=(A,X)$19 F1-micro on Cora, $G=(A,X)$20 on Citeseer, $G=(A,X)$21 on Pubmed, and $G=(A,X)$22 accuracy on ogbn-arxiv; it also consistently outperforms several baselines under random edge perturbations and feature noise (Li et al., 2023).

Several limitations recur across the literature. In explicit HOD-GNN, memory grows with interaction order $G=(A,X)$23 and derivative order $G=(A,X)$24; high-order derivatives may be small or noisy; non-analytic activations such as ReLU make higher-order derivatives vanish, although first-order derivatives still provide gains; and current sparse-kernel support in GNN libraries is described as suboptimal (Eitan et al., 2 Oct 2025). In S2-GNN, weighted edges are required when using the adjacency-based shift, the method assumes undirected graphs for symmetry and positive semidefiniteness, and very large $G=(A,X)$25 or $G=(A,X)$26 can increase sensitivity to perturbations (Giraldo et al., 2024). In diffusion-based HOD-GNNs, higher-order polynomials can introduce spectral ripples, explicit powers become expensive at high order, and very high orders may propagate noise or adversarial perturbations further (Li et al., 2023). In geometric higher-order GNNs, the lifting step may become the dominant cost, and careful normalization and orientation handling are necessary (Besta et al., 2024).

A recurring misconception is that HOD-GNN always refers to a single model. The literature instead supports a narrower and a broader reading. In the narrow sense, HOD-GNN is the end-to-end derivative-tensor architecture built from a base MPNN and a downstream GNN (Eitan et al., 2 Oct 2025). In the broader sense, it names a derivative-based design principle that includes sparse Sobolev filters, high-order diffusion equations, and discrete differential operators on simplicial, cellular, and hypergraph domains (Giraldo et al., 2024, Li et al., 2023, Besta et al., 2024). A second misconception is that higher-order derivatives necessarily imply dense operators and prohibitive complexity. The sparse derivative message passing of HOD-GNN, the Hadamard-power construction of S2-GNN, and the sparse realization of $G=(A,X)$27 in HiD-Net all show that higher-order derivative information can be computed or approximated in a sparsity-preserving manner.