Spatial-Temporal Relational Graph (STRG)

• Spatial-Temporal Relational Graph (STRG) is a structured model that integrates heterogeneous spatio-temporal data, semantic hierarchies, and multi-modal features.
• It employs LLM-enhanced semantic initialization and GCN-based propagation to capture explicit relational dynamics in mobility analytics.
• The unified framework improves next-location recommendations by aligning diverse data modalities and maintaining robustness under data-sparse scenarios.

A Spatial-Temporal Relational Graph (STRG) is a data structure that encodes and fuses heterogeneous spatio-temporal context, semantic hierarchies, and multi-modal information—most notably for mobility dynamics and next-location recommendation. STRG links are informed by LLM-enhanced spatial-temporal knowledge graphs (STKGs), and their construction and utilization involve explicit relational semantics, structured spatio-temporal transitions, and aligned multi-modal feature fusion. The STRG paradigm offers a unified approach to capture spatial, temporal, and functional dependencies in generalized mobility analytics (Dai et al., 27 Dec 2025).

1. Formal Definition and Entity-Relationship Structure

An STRG is derived from a foundational LLM-enhanced STKG, which is formally defined as a directed multi-relation graph: $G_{\mathrm{KG}} = (V, E, R)$ where

• $V = U \cup P \cup C \cup A$ encapsulates users ($U$), POIs ($P$), location categories ($C$), and activity types ($A$),
• $R$ consists of:
  • functionality relations ($r_f$),
  • time-indexed visit relations ($r^t_{\tau}$, $\tau = 1, \ldots, |T|$),
  • sequential transition relations ($r_t$),
• $E$ is the set of typed edges $(v_i, r, v_j)$.

Each edge relation is mapped to a binary adjacency matrix: $$A_r(i,j) = \begin{cases} 1, & \text{if } (v_i, r, v_j) \in E, \ 0, & \text{otherwise}. \end{cases}$$ The STRG is then constructed as a modality-specific, similarity-weighted undirected graph over same-type entities (i.e., POI-POI, category-category, activity-activity) using spatial-temporal transitions: $$d(e_i, e_j) = \| \mathbf{e}_{e_i} + \mathbf{r}_t - \mathbf{e}_{e_j} \|_2\,, \quad \mathrm{sim}(e_i, e_j) = \exp(-d(e_i, e_j))$$ where $\mathbf{e}_{e_i}$ is the STKG embedding of entity $e_i$ and $\mathbf{r}_t$ is the transition relation embedding (Dai et al., 27 Dec 2025).

2. LLM-Enhanced Semantic Initialization

The STRG construction leverages LLM-driven semantic enrichment of graph nodes. For a node $v_i$, the LLM is applied to its description: $$s_i = \mathrm{LLM}(\mathrm{desc}(v_i)) \in \mathbb{R}^{d_s}$$ and projected into the embedding space: $\phi_s(v_i) = W_s s_i + b_s \in \mathbb{R}^d$ where $W_s$ and $b_s$ are learnable parameters. This process enables enhanced initialization for both categorical and activity nodes, providing activity-aware structure for the subsequent STRG affinity graph (Dai et al., 27 Dec 2025).

3. Multi-Modal STRG Construction and Representation Learning

For each modality (e.g., IDs, images), a modality-specific STRG is induced. For POIs, the similarity matrix $M^p$ (with entries given by the sim function above) is sparsified to $k$-nearest neighbors to obtain the adjacency $G^p$. Initial features or image embeddings, concatenated into matrices $Z^p$ (for IDs) or $Z^{\text{img}}$ (for images), are propagated over these graphs via a single-layer GCN: $$Z^{p,\mathrm{fusion}} = \sigma(D_p^{-1} G^p Z^p W_{\mathrm{gcn}})$$

$\widehat{Z}^p = Z^{p,\mathrm{fusion}} + Z^p$

where $D_p$ is the degree matrix and $W_{\mathrm{gcn}}$ is a trainable weight. For image features, remote-sensing patches per POI are encoded via ViT (CLIP) and projected, then GCN-aggregated over the same STRG topology (Dai et al., 27 Dec 2025).

4. Gating and Cross-Modal Alignment

Each POI's multi-modal embeddings (ID-derived and image-derived) are fused using a gating mechanism: $$\alpha_i = \sigma(W_g [ h^p_i \| h^{\mathrm{img}}_i ] + b_g)$$

$$h^{\mathrm{fused}}_i = \alpha_i \odot h^p_i + (1-\alpha_i) \odot h^{\mathrm{img}}_i$$

where $\odot$ denotes element-wise multiplication and $W_g, b_g$ are learnable. Additionally, a bidirectional contrastive loss aligns the fused image and STKG representations: $$\mathcal{L}_{\mathrm{align}} = -\frac{1}{N_p} \sum_{i=1}^{N_p} \left[ \log \frac{\exp(\langle z_i^{\mathrm{KG}}, z_i^{\mathrm{img}} \rangle)}{\sum_{j=1}^{N_p}\exp(\langle z_i^{\mathrm{KG}}, z_j^{\mathrm{img}} \rangle)} + \log \frac{\exp(\langle z_i^{\mathrm{img}}, z_i^{\mathrm{KG}} \rangle)}{\sum_{j=1}^{N_p}\exp(\langle z_i^{\mathrm{img}}, z_j^{\mathrm{KG}} \rangle)} \right]$$ with projections $$z_i^{\mathrm{KG}} = \mathrm{Proj}_{\mathrm{KG}}(\mathbf{e}_{p_i})$$ and $$z_i^{\mathrm{img}} = \mathrm{Proj}_{\mathrm{KG}}(h^{\mathrm{fused}}_i)$$ (Dai et al., 27 Dec 2025).

5. STRG-Driven Mobility Modeling and Recommendation

The fused POI embeddings, now integrating STKG structure, LLM semantics, and multi-scale visual features, serve as the basis for user-trajectory representation. For a sequence of visits by a user $u$, each input vector aggregates user embedding, fused POI embedding, categorical/activity/time embeddings, and image features. A sequence model (e.g., Transformer decoder) operates over the trajectory, and candidate next locations $v$ are scored as follows: $$\hat{y}_{u,t+1}(v) = \mathrm{softmax}\left( \mathrm{MLP}_p\left[ \hat{h}_{|S_u|} \| h_v \| Z^t \right] \right)$$ where $\hat{h}_{|S_u|}$ is the sequence summary, $Z^t$ is the time-slot embedding, and $\mathrm{MLP}_p$ maps to output logits. Training jointly minimizes cross-entropy on multi-headed prediction of next location, category, activity, time, plus the alignment loss: $$\mathcal{L} = \mathcal{L}_p + \mathcal{L}_c + \mathcal{L}_a + \lambda_t \mathcal{L}_t + \lambda_{\mathrm{align}} \mathcal{L}_{\mathrm{align}}$$ with each head predicting the assigned label and $\lambda_t, \lambda_{\mathrm{align}}$ as tunable weights (Dai et al., 27 Dec 2025).

6. Comparative Significance, Generalization, and Multi-Modal Impact

Experimental evaluations on six benchmark datasets demonstrate that STRG-based approaches not only outperform unimodal and conventional GNN-based methods under normal circumstances, but also maintain superior generalization in abnormal, data-sparse, or distribution-shifted scenarios. The method's efficacy is attributed to the explicit injection of spatial-temporal relational structure, adaptive fusion with static visual context, and alignment with semantic, functional, and spatial hierarchies orchestrated by the LLM-enhanced STKG (Dai et al., 27 Dec 2025).

7. Relationship to Broader STKG Paradigms and Future Directions

STRG constitutes a derived, modality-specific relational graph engineered to inherit spatio-temporal relationality from a foundational STKG while enabling integration with additional modalities and semantic layers. Its formulation aligns with trends toward multi-modal spatio-temporal knowledge representation, cross-modal alignment, explainability through interpretable relational semantics, and LLM-driven functional enrichment (Dai et al., 27 Dec 2025). A plausible implication is that STRGs will facilitate transparent, dynamically adaptive mobility analytics in increasingly heterogeneous urban sensing environments. Future research may further investigate STRG construction strategies under evolving entity sets, time-varying spatial relationships, or online, streaming data regimes.