Graph Neural Differential Equations

• Graph Neural Differential Equations are continuous-depth models that use differential equations to evolve node features, integrating graph inductive biases and ODE solvers.
• They enable dynamic, spatio-temporal modeling and size transferability by converging to Graphon-NDEs, ensuring accurate prediction across various graph sizes.
• Theoretical analysis guarantees uniform-in-time convergence, offering practical guidelines for efficient training and performance transfer on large-scale graphs.

Graph Neural Differential Equations (GNDEs) characterize a paradigm where the evolution of node- or edge-level features on a graph is governed by the solution to a differential equation, typically an ordinary or partial differential equation, whose dynamics incorporate the structural inductive biases of a graph neural network (GNN). By generalizing the discrete, layer-wise architecture of standard GNNs to continuous-depth or continuous-time flows, GNDEs integrate techniques from dynamical systems, numerical analysis, and graph representation learning, enabling modeling of (potentially time-varying) relational processes with principled guarantees and increased flexibility. This approach is particularly well-suited for dynamic, spatio-temporal, and large-scale graph-structured data, where continuous modeling can offer benefits in computational efficiency, generalization, and transferability.

1. Continuous-Depth GNNs and ODE Parameterization

In the canonical framework of GNDEs, the discrete sequence of transformations in a standard GNN is recast as an initial value problem for a neural ordinary differential equation (ODE) defined on the graph. The propagation of node features $H(s) \in \mathbb{R}^{N \times F}$ through the layers (parameterized by a depth variable $s$) is replaced by the continuous flow

$$\frac{dH(s)}{ds} = F_{\mathcal{G}}(s, H(s), \theta)$$

with initial condition $H(0) = X_e$ (where $X_e$ is an embedding of the input features). The function $F_\mathcal{G}$ parameterizes the neural vector field and is typically realized by a graph convolution operator, potentially augmented with nonlinearity and explicit depth/time dependence. The output is acquired by integrating the ODE to a target depth:

$$Y = H(1) = X_e + \int_0^1 F_{\mathcal{G}}(\tau, H(\tau), \theta) d\tau$$

In static (non-temporal) contexts, this yields a continuous-depth analog to residual GNN architectures and enables the integration of advanced ODE solvers (e.g., Runge–Kutta methods, adaptive-step integrators) directly into the forward pass (Poli et al., 2019). When extended to dynamic or temporal graphs, this ODE flow can be hybridized with discrete update mechanisms (such as gated recurrent units), forming a hybrid dynamical system flexible enough to capture event-driven regime changes and time-localized observations (Poli et al., 2019, Poli et al., 2021).

2. Mathematical Analysis and Well-Posedness

Recent work rigorously formalizes GNDEs as well-posed dynamical systems on graphs and studies their infinite-node limits. Size transferability—the property that models trained on moderate-sized graphs generalize to larger graphs of similar structure—is justified by analyzing the convergence of GNDEs to Graphon Neural Differential Equations (Graphon-NDEs) in the infinite-node limit (Yan et al., 4 Oct 2025).

A Graphon-NDE is formulated as

$$\partial_t \Phi(u, t) = \Phi(W; \Phi(u, t); \Theta(t)), \quad \Phi(u, 0) = Z(u), \quad u \in I$$

where $W(u, v)$ is the graphon (a symmetric measurable function representing the limiting adjacency structure), and $Z(u)$ represents initial features on the continuum of nodes. The paper establishes that, under mild conditions (e.g., Lipschitz activation functions, continuous time-variation of filters), the entire solution trajectory of the GNDE converges uniformly in time to that of the corresponding Graphon-NDE. For graphs drawn from $\alpha$-Hölder continuous (smooth) graphons, the convergence rate is $O(n^{-\alpha})$, and for discontinuous ($\{0,1\}$-valued) graphons, the rate depends on the fractal dimension $b$ of the graphon's support boundary: $O(n^{-(1-(b+\epsilon)/2)})$ (see formulas (3)-(4) in (Yan et al., 4 Oct 2025)).

This theoretical foundation gives explicit, practical guidance for size transfer: if a GNDE model is trained on an $n$-node graph sampled from a given graphon, its solution trajectory—and hence its prediction accuracy—will remain close when applied directly to a larger graph sampled from the same family, with the transfer error quantitatively controlled by the sampling regime and the underlying regularity of the graphon.

3. Convergence Rates and Uniformity in Time

A distinguishing element of this convergence analysis is its trajectory-wise nature: the discrepancy between the node representations generated by the finite GNDE and the infinite Graphon-NDE is bounded uniformly across the entire integration interval $[0, T]$, not just at isolated endpoints. This property is established using induced graphon representations for the finite GNDE, embedding both finite and infinite models in a common $L^2$ function space:

$$\| X_n - X \|_{C([0, T]; L^2(I))} \leq C n^{-\alpha}$$

for smooth graphons, with $C$ independent of $n$ (see formula (3) in (Yan et al., 4 Oct 2025)). The result requires only that model parameters (filters) vary continuously in time (AS0) and that the activation function is normalized Lipschitz (AS1), conditions satisfied by most practical GNDE instantiations.

These convergence results extend to unweighted graphs with discontinuous support (e.g., stochastic block models, fractal graphons). In such cases, the complexity of the support's boundary (as measured by its upper box-counting dimension) directly impacts the rate, degrading as the boundary becomes more intricate (see formula (4) in (Yan et al., 4 Oct 2025)).

4. Size Transferability: Practical Implications

The trajectory-wise convergence of GNDEs to Graphon-NDEs has direct implications for model transferability:

• Explicit size transferability bounds: For two graphs of sizes $n_1$ and $n_2$, both sampled from the underlying graphon, the solution trajectories satisfy

$$\| X_{n_1} - X_{n_2} \|_{C([0, T]; L^2(I))} \leq C \left( n_1^{-\alpha} + n_2^{-\alpha} \right)$$

for smooth weighted graphons, providing a theoretical performance guarantee when deploying GNDEs trained on smaller graphs to larger ones without retraining (Yan et al., 4 Oct 2025).

• Empirical correspondence: Numerical experiments on synthetic (e.g., tent graphon, stochastic block models, hexaflake fractals) and real datasets (e.g., Cora, Citeseer, ogbn-arxiv) confirm the theoretically predicted rates and show small performance gaps between moderate and large graph size runs. In transfer experiments, node classification accuracy on full graphs remains close to that measured on training subgraphs, and as the subgraph size increases, the transfer error diminishes (see the empirical results in (Yan et al., 4 Oct 2025)).
• Computational efficiency: Training GNDEs on moderate-size subgraphs, followed by transfer to large graphs, yields significant reductions in compute time while preserving accuracy. This strategy leverages the proven stability of the solution trajectory against increases in graph size.

5. Applications and Numerical Validation

The convergence and transferability guarantees of GNDEs expand their practical domain. Demonstrated applications include:

• Semi-supervised node classification on citation networks (Cora, Citeseer), where GNDEs trained on subgraphs (with sizes from a few hundred to thousands of nodes) generalize successfully to larger graphs without significant loss in accuracy.
• Large-scale node classification on open-graph benchmarks (ogbn-arxiv), where transferability enables competitive accuracy with reduced training time.
• Synthetic benchmarks where the regularity of the generating graphon can be precisely controlled, elucidating the impact of graphon smoothness/fractality on convergence rate.

These applications exploit not just endpoint predictions but the entire feature evolution, as size transferability guarantees the fidelity of intermediate node representations as well.

6. Theoretical Contributions and Limitations

The principal theoretical contribution is the establishment of a trajectory-wise, uniform-in-time convergence framework for GNDEs with time-varying filters, including explicit rates under realistic graph generation regimes. By embedding both the finite and infinite models in a function space ($L^2(I)$) and leveraging Grönwall inequalities, the paper (Yan et al., 4 Oct 2025) provides error bounds that inform practice.

Underlying assumptions include continuous-time-varying filters and normalized Lipschitz activations. For weighted graphs, additional smoothness of the graphon is required; for binary graphs, convergence rate depends on the boundary complexity of the support. These rates hold under deterministic graph sampling regimes; extension to more complex or stochastic sampling remains an area for future paper. Finally, the theory presumes structural similarity between training and deployment graphs, as quantified by their graphon representations.

7. Summary

Continuous-depth graph neural networks formulated as Graph Neural Differential Equations offer a mathematically principled framework for modeling graph-based dynamics. The convergence of GNDEs to Graphon-NDEs in the infinite-node limit ensures that solution trajectories—and thus predictions—transfer robustly with explicitly characterized rates from models trained on small or moderate-sized graphs to larger, structurally similar graphs. This property is supported by rigorous analysis and validated numerically, providing a theoretical foundation for scalable, size-transferable graph learning systems operating in continuous-depth regimes (Yan et al., 4 Oct 2025).