Dynamic Graph Neural Networks

• Dynamic Graph Neural Networks are architectures for learning on graphs with evolving topology and time-varying attributes.
• They integrate methods like neural controlled differential equations and hybrid discrete/continuous paradigms to capture irregular sampling and dynamic dependencies.
• Applications include temporal forecasting, dynamic link prediction, and imputation in domains such as traffic networks, biological systems, and recommendation systems.

A dynamic Graph Neural Network (GNN) is a neural architecture for representation learning on graph-structured data where the underlying topology, node/edge attributes, or both evolve over continuous or discrete time. Unlike static GNNs, which assume a fixed adjacency, dynamic GNNs explicitly capture, propagate, and predict the evolving dependencies inherent in multivariate time series, interaction networks, physical systems, or streaming data. This domain encompasses both discrete-time (snapshot-based or event-driven) and continuous-time (neural differential equation-based) formalisms, with the principal challenge arising from the complex interplay between structural changes and feature dynamics.

1. Formal Problem Setting and Modeling Challenges

Dynamic GNNs operate on sequences $\{(t_0, G_{t_0}), \dots, (t_N, G_{t_N})\}$, where each $G_{t_k}$ is a graph comprising a node set $\mathcal{V}$, edge set $\mathcal{E}$, evolving adjacency $A_t$, and possibly time-varying node and edge features $F_t$, $E_t$. Alternatively, continuous-time models define a path $$A_t : [t_0, t_N] \rightarrow \mathbb R^{|\mathcal{V}| \times |\mathcal{V}|}$$ such that both structure and attributes can change smoothly and asynchronously.

Key modeling difficulties include:

• Temporal Evolution of Structure: Evolving topology creates challenges for message passing, neighborhood aggregation, and capturing nonstationary dependencies.
• Irregular Sampling and Missing Data: Most real-world data is collected at irregular times, and observations may be missing, requiring robustness in embedding evolution and prediction.
• Interplay of Node Dynamics and Topology Drift: Node latent states evolve both via their own temporal features and through changing neighborhoods, inducing nontrivial, highly coupled dynamics.
• Scalability and Instant Updates: Large, rapidly changing graphs necessitate update mechanisms with amortized constant or near-constant per-event cost, avoiding full recomputation per snapshot (Zheng et al., 2022).

The central objective is to learn time-dependent node/edge/graph embeddings $Z_t$ to support downstream tasks—node/edge classification, dynamic link prediction, temporal forecasting—at arbitrary query times.

2. Core Model Architectures

Dynamic GNNs span several architectural paradigms, each addressing the challenge of temporal variability in a distinctive way:

2.1 Neural Controlled Differential Equations

The GN-CDE model (Qin et al., 2023) posits that node embeddings $Z_t$ solve a matrix-valued controlled differential equation driven by a continuous path version of the adjacency:

$$Z_t = Z_{t_0} + \int_{t_0}^{t} f_\theta(Z_s, A_s) \; \mathrm{d}\bar{A}_s$$

where $f_\theta$ is a GNN-parameterized vector field (e.g., a multi-layer GCN), $\bar{A}_t$ is a continuous interpolation of the observed adjacency snapshots (via natural cubic splines or similar), and $\mathrm{d}\bar{A}_s$ quantifies instantaneous adjacency evolution. This formulation enables:

• Direct modeling of trajectories in node embedding space that respond both to changing graph structure and observed data streams.
• Calibration: immediate incorporation of new observations by resetting $Z_{t^*}$ at an intermediate time and re-integrating forward.
• Robustness to missing/irregular data: well-defined interpolated paths ensure embeddings remain valid even under sparse or missing graph snapshots.

2.2 Hybrid Discrete/Continuous Paradigms

The neural graph differential equation (Neural GDE) framework (Poli et al., 2021) formalizes the continuous-depth GNN limit, whereby layer-wise propagation becomes an ODE:

$$\frac{\mathrm{d}Z(t)}{\mathrm{d}t} = f^\mathfrak{g}(t, Z(t), \theta(t))$$

For dynamic graphs, hybrid systems alternate between continuous flows under fixed topology (integration segments) and discrete "jumps" (e.g., GRU-based updates) when the graph or features change. This leads to hybrid arcs: $Z(t)$ is integrated until an event time $t_k$, whereupon the system is updated via a discrete operator. This method is especially effective for accommodating aperiodic and irregularly sampled data, and for complex, high-frequency switching in graph structure.

2.3 Dynamic Structure Learning and Adaptive Graphs

Models such as SDGL (Li et al., 2021) and GVNN (Roy et al., 24 Sep 2025) combine static, long-term skeleton graphs with signal- or feature-dependent dynamic adjacency construction. Typical pipelines involve:

• Static Graph Construction: Learn node embeddings $E$ from which an affinity matrix $A_s$ is computed; regularized for smoothness and sparsity.
• Dynamic Graph Generation: For each time $t$, blend $E$ with the current features and compute $A_t$ using multi-head attention and a momentum bias to ensure dynamic graphs fluctuate around the static support.
• GNN layers operate with both $A_s$ and $A_t$, and their outputs are fused for forecasting or prediction tasks.

3. Online, Streaming, and Incremental Computation

Dynamic GNNs for event-driven or streaming contexts require updates with sublinear cost per event. Key systems include:

• InstantGNN (Zheng et al., 2022): Maintains local Personalized PageRank estimates $(\hat{p}, r)$ per input dimension and incrementally updates only the affected nodes upon each graph event (edge addition/deletion). This approach achieves $O(1)$ amortized time per edge event, supports both structural and feature updates, and provides an adaptive retraining strategy that triggers costly head (MLP) updates only when the accumulated embedding drift exceeds a threshold.
• D3-GNN (Guliyev et al., 2024): Implements a distributed streaming system for dynamic GNNs under a hybrid-parallel computation graph (Flink operators as layers/partitions). Core mechanisms include:
  • Mergeable and commutative "synopsis" aggregator states for incremental and invertible message passing at each operator/layer.
  • Two-level windowed forward pass (intra-layer and inter-layer) defers updates to batch rapid events and tame "neighborhood explosion."
  • Hybrid vertex- and model-parallel strategies balance load and ensure high throughput (up to $76\times$ compared to DGL streaming) with low latency and strong scalability.

4. Spatio-Temporal and Tensor Graph Extensions

Dynamic GNNs can explicitly model both spatial and temporal graph dependencies via tensor representations and advanced graph signal processing methods:

• DSTGNN (Jia et al., 2020) constructs dynamic spatial tensor graphs (STG) and temporal tensor graphs (TTG) to capture, respectively, instantaneous inter-node similarity and intra-node temporal patterns. These two are entangled via a Projected Entangled Pair State (PEPS) tensor network, exploiting low-rank structures and enabling cross-modal information flow.
• TM-GCN (Malik et al., 2019) employs the tensor $M$-product for spatio-temporal message passing, enabling simultaneous aggregation across nodes and time. The associated tensor spectral convolution proves compatible with classical MPNN propagation, with empirical gains in edge classification, link prediction, and early detection tasks.

5. Task Objectives, Robustness, and Evaluation

Dynamic GNNs are evaluated on a range of supervised and unsupervised prediction tasks:

• Node/Edge Attribute Prediction: Losses vary from mean-square errors for regression to cross-entropy for classification. Models target interpolation (within observed time interval) and extrapolation (beyond observed window) settings (Qin et al., 2023).
• Dynamic Link Prediction: Candidate edge pairs are scored based on joint embeddings, e.g., via an MLP head on $[z_i(t);z_j(t)]$.
• Forecasting and Imputation: Models are tested on multivariate time-series tasks, missing-value imputation, and extrapolation under irregular sampling.
• Stability and Online Adaptivity: Theoretical guarantees (for instance, stability to topological perturbations or convergence of distributed/online retraining (Gao et al., 2021)) are established where possible.

Empirical evaluation spans synthetic and real-world domains—traffic sensor networks (METR-LA, PeMS), biological networks (repressilator), recommendation systems, and temporal knowledge graphs. Dynamic GNNs consistently outperform static or snapshot-based baselines, often achieving both substantial accuracy gains and orders-of-magnitude efficiency improvements (e.g., $60\times$ faster propagation; $6\%$ lower MAE vs. SOTA).

6. Extensions, Limitations, and Future Directions

Dynamic GNNs are rapidly evolving and face several remaining challenges:

• Scalability: Interpolating high-dimensional paths (as in GN-CDE) or contracting large tensor networks can become prohibitive on very large graphs. Approximate or fused adjacency representations, subgraph batching, or windowed aggregation reduce these costs (Qin et al., 2023, Guliyev et al., 2024).
• Extension to Heterogeneous, Multigraph, and Directional Structures: Several formalisms support seamless extension to directed graphs, multigraphs, and heterogeneous networks by selecting the appropriate template GNN in the propagation field.
• Integration of Physical and Domain Constraints: Emerging methods incorporate domain-specific priors (symmetry, conservation laws, physical invariances) via constrained dynamic operators (DMD-GNN: (Shi et al., 2024)).
• Event-Driven and Edge-Arrival Models: Streaming models must robustly process out-of-order, bursty, or adversarial event streams. Adapting existing architectures to multi-relation and hypergraph settings is an ongoing area of research.
• Interpretability and Analysis: Spectral and tensor-based approaches offer interpretability benefits (e.g., via DMD modes or PEPS tensors) but require further scaling and evaluation in complex physical/causal domains.

7. Representative Model and Performance Comparison

Method	Dynamic Structure Type	Learning Objective	Distinctive Mechanism	SOTA Claim / Key Result
GN-CDE	Continuous-time	Node/edge attributes, forecasting	Controlled differential equation	Large error reduction (Qin et al., 2023)
SDGL	Discrete snapshot-based	Multivariate time series forecast	Static + dynamic embedding fusion	$6\%$ MAE reduction (Li et al., 2021)
InstantGNN	Event-driven (edge arrival)	Node classif., link pred., imputation	Incremental Personalized PageRank	$60\times$ faster (Zheng et al., 2022)
D3-GNN	Streaming/event-driven	All GNN downstream tasks	Fully incremental, distributed MPGNN	$76\times$ throughput gain (Guliyev et al., 2024)
TM-GCN	Discrete-time tensor	Edge classif., link pred., early warn	Tensor $M$-product, spatiotemporal conv	Most tasks best (Malik et al., 2019)
DSTGNN	Tensor (spatio-temporal)	Traffic forecasting	STG+TTG+PEPS tensor entanglement	Outperforms WaveNet/DCRNN (Jia et al., 2020)

In summary, dynamic GNNs unify the modeling of continuous, nonstationary, and highly structured processes, leveraging tools from neural ODEs/CDEs, online optimization, dynamic graph learning, and tensor algebra. They provide a rigorous and generalizable toolbox for high-fidelity prediction, forecasting, and understanding in dynamic relational environments, with ongoing advances focused on scaling, robustness, and domain adaptation.