Spatiotemporal Multigraph Representation

• Spatiotemporal multigraph representation is a framework that integrates multiple spatial modalities and temporal evolutions into one learnable object.
• It employs diverse constructions such as modality-specific graphs, tensor formulations, product graphs, and event-based models to capture complex dependencies.
• The approach leverages convolution, message passing, and fusion operators to enhance forecasting, link prediction, and interpretability in dynamic systems.

Spatiotemporal multigraph representation denotes a family of graph formalisms in which spatial structure, temporal evolution, and their coupling are encoded in a single learnable object rather than separated into an ordinary graph plus an external sequence model. In the literature represented here, that object appears in several mathematically distinct forms: as $M$ modality-specific graphs $G^{(m)}=(V,E^{(m)})$ over a common node set; as a third-order adjacency tensor $\mathcal A\in\mathbb R^{N\times N\times T}$; as a product graph on $V_t\times V_s$; as a block adjacency matrix augmented with learned temporal links; and as a continuous-time event sequence $\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$ in which repeated timestamped interactions define a multigraph. The shared purpose is to support representation learning, forecasting, link prediction, detection, or graph inference while preserving spatial dependencies, temporal dependencies, and interdependence between them (Geng et al., 2019, Wang et al., 2024, Sabbaqi et al., 2022, Ahmad et al., 2023, Wang et al., 2024).

1. Formal scope and mathematical definitions

A canonical discrete multigraph formulation uses a fixed vertex set $V$ with $N$ regions and $M$ relation modalities. Each modality induces a graph

$$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$

with degree matrix $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$ and symmetric normalized Laplacian

$G^{(m)}=(V,E^{(m)})$0

A time-indexed signal may be a scalar observation $G^{(m)}=(V,E^{(m)})$1, node features $G^{(m)}=(V,E^{(m)})$2, or a lag stack

$G^{(m)}=(V,E^{(m)})$3

This formulation is explicit in urban forecasting, where neighborhood, POI-similarity, and road-connectivity are modeled as separate spatial modalities (Geng et al., 2019).

A second formulation treats the entire evolving graph as a tensor. In STGCNDT, a dynamic graph is represented by a third-order adjacency tensor

$G^{(m)}=(V,E^{(m)})$4

and a node-feature tensor

$G^{(m)}=(V,E^{(m)})$5

The first two modes are spatial and the third mode is temporal. In GVNN, a related tensorial object is the graph-variate tensor

$G^{(m)}=(V,E^{(m)})$6

where $G^{(m)}=(V,E^{(m)})$7 is a stable support and $G^{(m)}=(V,E^{(m)})$8 is an instantaneous connectivity matrix produced by a node-pair function $G^{(m)}=(V,E^{(m)})$9 (Wang et al., 2024, Roy et al., 24 Sep 2025).

A third formulation expands the node set itself. In graph-time models, one defines a spatial graph $\mathcal A\in\mathbb R^{N\times N\times T}$0 with $\mathcal A\in\mathbb R^{N\times N\times T}$1 and a temporal graph $\mathcal A\in\mathbb R^{N\times N\times T}$2 with $\mathcal A\in\mathbb R^{N\times N\times T}$3, then forms a product graph $\mathcal A\in\mathbb R^{N\times N\times T}$4 with node set $\mathcal A\in\mathbb R^{N\times N\times T}$5, hence $\mathcal A\in\mathbb R^{N\times N\times T}$6. In GL-LRSS, the same idea appears as a space-time multigraph with nodes $\mathcal A\in\mathbb R^{N\times N\times T}$7, spatial edges $\mathcal A\in\mathbb R^{N\times N\times T}$8–$\mathcal A\in\mathbb R^{N\times N\times T}$9, and temporal edges $V_t\times V_s$0–$V_t\times V_s$1, represented by

$V_t\times V_s$2

This suggests that “multigraph” in this literature may refer either to multiple edge modalities on a common node set or to a lifted graph whose vertices are space-time pairs (Sabbaqi et al., 2022, Liu et al., 2019).

A fourth formulation is explicitly event-based. In continuous-time dynamic graphs, the interaction sequence

$V_t\times V_s$3

is interpreted as a spatiotemporal multigraph because a node pair may interact multiple times at different timestamps. In event-based object detection, a multigraph is written directly as

$V_t\times V_s$4

with decoupled spatial edges $V_t\times V_s$5 and temporal edges $V_t\times V_s$6 coexisting over the same event-node set $V_t\times V_s$7 (Wang et al., 2024, Verma et al., 20 Jul 2025).

2. Principal construction paradigms

The literature uses several recurring constructions to make spatiotemporal dependencies learnable.

Construction	Joint object	Characteristic use
Multi-modal graph family	$V_t\times V_s$8	multiple spatial relations
Dynamic graph tensor	$V_t\times V_s$9	time-indexed adjacency slices
Product graph	$\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$0	explicit space-time coupling
Mended block adjacency	$\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$1	learned temporal off-diagonal links
Decoupled event multigraph	$\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$2	separate spatial and temporal edge sets

In multi-modal urban forecasting, the defining construction is a family of graphs sharing one node set but differing by modality. Lower layers permit inter-modality convolutions across ordered pairs $\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$3, whereas higher layers retain only intra-modality outputs while imposing a tensor-normal prior across modalities. Final prediction is obtained by averaging the modality-wise scalar outputs,

$\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$4

The same work states that the input uses the past $\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$5 slices $\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$6 to capture closeness, period, and trend (Geng et al., 2019).

In graph-time formulations, the core construction is the product graph. Kronecker, Cartesian, and strong products yield

$\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$7

A parametric product graph introduces four learnable scalars,

$\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$8

or equivalently $\mathcal E=\{(u_i,v_i,t_i,e_i)\}_{i=1}^M$9, enabling the spatiotemporal coupling itself to be learned from data (Sabbaqi et al., 2022, Isufi et al., 2021).

A contrasting strategy starts from a disconnected union of time-indexed graphs. The block adjacency matrix

$V$0

contains only intra-time spatial edges. “Mending” augments it by a feature-aware projection and Transformer encoder, then symmetrizes and rectifies: $V$1 The resulting graph has learned temporal edges in the off-diagonal blocks, and the paper reports that the original block adjacency has $V$2 zero Laplacian eigenvalues, whereas the modified graph has exactly one zero eigenvalue and a strictly positive Fiedler value (Ahmad et al., 2023).

Event-based asynchronous vision uses an explicitly decoupled multigraph. Spatial neighbors satisfy

$V$3

with $V$4, $V$5, and $V$6. Temporal neighbors instead use an inverted ellipsoid with semi-major axis along time, $V$7, semi-minor axis $V$8, and $V$9. This separation is then justified computationally by replacing a 3D spline kernel with a 2D anisotropic spline plus temporal attention (Verma et al., 20 Jul 2025).

Further variants specialize the multigraph to the data domain. DSTGNN constructs a spatial tensor graph $N$0 and a temporal tensor graph $N$1, then entangles them through PEPS. ST-GraphRL uses a weighted directed graph $N$2 whose edge tensor stores

$N$3

that is, movement frequency, spatial distance, and average duration. The paper states that this directed graph with edge attributes captures both spatial and temporal characteristics of human movements as a multigraph (Jia et al., 2020, Huang et al., 2023).

3. Convolution, message passing, and fusion operators

Spatiotemporal multigraph representation is tightly linked to the operator used to propagate information on the joint structure. In the multi-modal Chebyshev setting, grouped GCN defines a weight tensor for every ordered modality pair,

$N$4

and updates modality $N$5 by

$N$6

To avoid overfitting over the $N$7 parameter blocks, the model uses a grouped $N$8 penalty

$N$9

with tunable $M$0. Higher layers switch to a multi-linear-relationship GCN whose weights are collected in a four-way tensor $M$1 with tensor-normal prior $M$2 (Geng et al., 2019).

Tensorized dynamic-graph models replace graph polynomials by tensor products. STGCNDT defines an M-product

$M$3

and a GTCN layer

$M$4

The transform $M$5 is the temporal filter. The paper studies DFT, DCT, and HWT, and ensembles their outputs by

$M$6

with $M$7 in the experiments (Wang et al., 2024).

Product-graph approaches instead retain the ordinary shift-and-sum principle on the enlarged graph. With a product-graph shift $M$8, a $M$9-th-order filter is

$$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$0

and for multi-feature input the output feature $$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$1 is

$$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$2

GTCNN layers then combine graph-time convolution, zero-pad pooling, and nonlinearity; the parametric coefficients $$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$3 in the product graph are learned jointly with the filter taps, with an $$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$4-penalty on $$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$5 to induce sparsity (Sabbaqi et al., 2022, Isufi et al., 2021).

Other models keep spatial and temporal aggregation separate but fuse them explicitly. In eGSMV, spatial messages use a 2D tensor-product B-spline kernel

$$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$6

whereas temporal messages use motion-vector attention with queries $$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$7, keys $$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$8, headwise coefficients $$G^{(m)}=(V,E^{(m)}), \qquad A^{(m)}\in\mathbb R^{N\times N}, \quad m=1,\dots,M,$$9, and output

$D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$0

Fusion is then

$D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$1

DSTGNN alternates temporal graph convolution over $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$2 and spatial graph convolution over $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$3. ST-GraphRL similarly decouples spatial and temporal encoders before a joint space-time GNN re-entangles node features, edge features, and normalized weights $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$4 (Verma et al., 20 Jul 2025, Jia et al., 2020, Huang et al., 2023).

4. Learning objectives, priors, and theoretical properties

A notable property of spatiotemporal multigraph methods is that graph construction and representation learning are frequently optimized together. In the multi-modal forecasting model, the total loss combines an RMSE term with lower-layer grouped penalties and higher-layer tensor-normal penalties: $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$5 The paper states that temporal shifting is handled both by including period and trend slices in the input and by the MR-GCN prior, which freezes the feature-mode covariances $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$6 and $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$7 while updating $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$8 and $D^{(m)}=\mathrm{diag}(A^{(m)}\mathbf 1)$9 by flip-flop equations (Geng et al., 2019).

Unsupervised objectives can use the spatiotemporal graph only as an encoder while placing the learning signal in time. STDGI encodes each snapshot with a two-layer GCN and maximizes mutual information between node embeddings at time $G^{(m)}=(V,E^{(m)})$00 and raw node features at future times $G^{(m)}=(V,E^{(m)})$01, with $G^{(m)}=(V,E^{(m)})$02. The discriminator loss is

$G^{(m)}=(V,E^{(m)})$03

and the total objective is $G^{(m)}=(V,E^{(m)})$04. The paper also states explicitly that STDGI, as presented, is a single-relation model; multi-relation or multigraph extensions would require replacing each GCN layer by a relational GCN or using separate adjacency tensors $G^{(m)}=(V,E^{(m)})$05 (Opolka et al., 2019).

In graph learning, the multigraph itself may be the optimization target. GL-LRSS jointly estimates a latent low-rank signal $G^{(m)}=(V,E^{(m)})$06 and graph Laplacian $G^{(m)}=(V,E^{(m)})$07 through

$G^{(m)}=(V,E^{(m)})$08

Here $G^{(m)}=(V,E^{(m)})$09, the nuclear norm enforces low rank, and the trace term is the spatiotemporal smoothness prior (Liu et al., 2019).

Connectivity constraints can also be regularized spectrally or sparsely. The STBAM framework trains a classifier with cross-entropy plus a sparsity-promoting norm on the learned adjacency,

$G^{(m)}=(V,E^{(m)})$10

and reports that $G^{(m)}=(V,E^{(m)})$11 gave the best trade-off between connectivity and parsimony. In graph-time convolutional theory, stability under support perturbation $G^{(m)}=(V,E^{(m)})$12, $G^{(m)}=(V,E^{(m)})$13, is quantified by

$G^{(m)}=(V,E^{(m)})$14

which the paper interprets as an expressivity–robustness trade-off: larger $G^{(m)}=(V,E^{(m)})$15, deeper $G^{(m)}=(V,E^{(m)})$16, or more outputs $G^{(m)}=(V,E^{(m)})$17 improve discriminability but degrade robustness (Ahmad et al., 2023, Sabbaqi et al., 2022).

5. Continuous-time, instantaneous, and sample-wise multigraphs

Continuous-time dynamic graphs replace discrete slices by timestamped events and therefore require positional or memory mechanisms that operate at event time. CorDGT assumes a fixed node set $G^{(m)}=(V,E^{(m)})$18, raw features $G^{(m)}=(V,E^{(m)})$19, and event stream

$G^{(m)}=(V,E^{(m)})$20

Under a homogeneous Poisson point-process assumption for prior interaction times $G^{(m)}=(V,E^{(m)})$21, the interaction intensity is estimated parameter-free by

$G^{(m)}=(V,E^{(m)})$22

This enters the Temporal Distance

$G^{(m)}=(V,E^{(m)})$23

which is combined with hop-based Spatial Distance in sinusoidal encodings, unitary encoding $G^{(m)}=(V,E^{(m)})$24, and correlated encoding

$G^{(m)}=(V,E^{(m)})$25

This makes self-attention sensitive to whether a context node is simultaneously close in space and time to both target endpoints (Wang et al., 2024).

CTDG-SSM addresses long-range temporal propagation by reformulating continuous-time graph memory as a state-space model. A topology-aware polynomial filter

$G^{(m)}=(V,E^{(m)})$26

is used to project HiPPO memory onto graph topology, yielding the continuous-time ODE

$G^{(m)}=(V,E^{(m)})$27

and readout

$G^{(m)}=(V,E^{(m)})$28

Under zero-order hold between irregular event times $G^{(m)}=(V,E^{(m)})$29, this becomes

$G^{(m)}=(V,E^{(m)})$30

The paper states that the $G^{(m)}=(V,E^{(m)})$31-th-order filter captures up to $G^{(m)}=(V,E^{(m)})$32-hop neighbors, while the underlying HiPPO projection compresses the entire past into a fixed-size state in online $G^{(m)}=(V,E^{(m)})$33 time (Raghuvanshi et al., 3 Jun 2026).

GVNN occupies an intermediate position between discrete and continuous viewpoints. At each time $G^{(m)}=(V,E^{(m)})$34, it builds an instantaneous graph

$G^{(m)}=(V,E^{(m)})$35

and fuses it with a stable support $G^{(m)}=(V,E^{(m)})$36 by Hadamard product,

$G^{(m)}=(V,E^{(m)})$37

The layer update then mixes the raw signal and the graph-variate convolution through

$G^{(m)}=(V,E^{(m)})$38

The paper emphasizes three properties: sample-by-sample interactions via $G^{(m)}=(V,E^{(m)})$39, regularization by the stable support $G^{(m)}=(V,E^{(m)})$40, and linear complexity in sequence length,

$G^{(m)}=(V,E^{(m)})$41

instead of the $G^{(m)}=(V,E^{(m)})$42 cost of a naive Kronecker spatiotemporal convolution (Roy et al., 24 Sep 2025).

6. Empirical behavior, interpretability, and conceptual boundaries

Across domains, spatiotemporal multigraph representations are evaluated not only by downstream accuracy but also by robustness, convergence, interpretability, and computational profile. In ride-hailing demand forecasting, the combined GGCN+MRGCN model reduces RMSE by $G^{(m)}=(V,E^{(m)})$43 over the single-graph baselines MGCN and STMGCN, cuts convergence time roughly in half when the MR-prior is included, and remains stable as test-set temporal divergence from training grows. The learned modality covariance $G^{(m)}=(V,E^{(m)})$44, visualized as a Hinton diagram, shows strong positive coupling between POI-similarity and road-connectivity and between POI-similarity and neighborhood, but weak or no coupling between neighborhood and road, matching the structural overlap of those graphs (Geng et al., 2019).

Dynamic graph tensor methods report similar advantages from unified spatial-temporal processing. STGCNDT states that its fused tensor convolution significantly outperforms state-of-the-art models on four communication-network dynamic graphs in link-weight estimation because diversified transformations capture periodic, smooth, and bursty behaviors more effectively than a single temporal basis (Wang et al., 2024).

In event-based detection, the efficiency claim is unusually explicit. eGSMV reports over a $G^{(m)}=(V,E^{(m)})$45 improvement in detection accuracy compared to previous graph-based works, with a $G^{(m)}=(V,E^{(m)})$46 speedup, reduced parameter count, and no increase in computational cost. The anisotropic decomposition is quantified by

$G^{(m)}=(V,E^{(m)})$47

so that for $G^{(m)}=(V,E^{(m)})$48,

$G^{(m)}=(V,E^{(m)})$49

that is, $G^{(m)}=(V,E^{(m)})$50 fewer parameters and FLOPs per message. The reported Gen1 and eTraM scores are $G^{(m)}=(V,E^{(m)})$51 mAP and $G^{(m)}=(V,E^{(m)})$52 mAP@50 with $G^{(m)}=(V,E^{(m)})$53M parameters, compared with $G^{(m)}=(V,E^{(m)})$54 mAP and $G^{(m)}=(V,E^{(m)})$55M parameters for AEGNN on Gen1 (Verma et al., 20 Jul 2025).

Learned temporal connectivity in block-adjacency models is likewise tied to both accuracy and graph connectivity. STBAM-64 achieves $G^{(m)}=(V,E^{(m)})$56 accuracy on C2D2, compared with $G^{(m)}=(V,E^{(m)})$57 for 3D-ResNet-34 and $G^{(m)}=(V,E^{(m)})$58 for STAG-NN-BA-GSP, using $G^{(m)}=(V,E^{(m)})$59M parameters. On SurgVisDom it reports weighted $G^{(m)}=(V,E^{(m)})$60, global $G^{(m)}=(V,E^{(m)})$61, and balanced accuracy $G^{(m)}=(V,E^{(m)})$62 (Ahmad et al., 2023).

Trajectory modeling gives a different interpretability criterion: whether representation distance correlates with distributional similarity. ST-GraphRL reports that the correlation $G^{(m)}=(V,E^{(m)})$63 between representation-space distance and true joint spatiotemporal distribution distance rises from approximately $G^{(m)}=(V,E^{(m)})$64 for DGI or $G^{(m)}=(V,E^{(m)})$65 for Summary-Trajectory to $G^{(m)}=(V,E^{(m)})$66 on Chengdu, and that it outperforms sequence-only, graph-only, temporal-only, and ablated baselines in Accuracy, Precision, and $G^{(m)}=(V,E^{(m)})$67 by margins up to $G^{(m)}=(V,E^{(m)})$68 (Huang et al., 2023).

A common misconception is that any spatiotemporal graph model is automatically a multigraph. The literature here does not support that equivalence. STDGI is explicitly described as a single-relation spatiotemporal graph model with fixed adjacency $G^{(m)}=(V,E^{(m)})$69, even though it is compatible with multi-relation extensions. Conversely, product-graph, graph-tensor, and event-stream models may all qualify as spatiotemporal multigraph representations despite using very different mathematical objects. A plausible implication is that the term is best understood as a representational family unified by joint space-time graph structure, rather than as one canonical adjacency format (Opolka et al., 2019, Sabbaqi et al., 2022, Wang et al., 2024, Wang et al., 2024).