Graph-Integrated Module (GIM)

• Graph-Integrated Module (GIM) is a neural subarchitecture that models evolving spatio-temporal patterns by dynamically constructing adjacency matrices and employing interval-aware dropout.
• It utilizes both learned-projection and Gaussian kernel methods to build time-varying graphs and applies multi-order convolutions to capture long-range sensor dependencies.
• Empirical ablations show that its dynamic graph, multi-order propagation, and dropout regularization significantly reduce RMSE, enhancing performance under diverse missing data conditions.

A Graph-Integrated Module (GIM) is a neural subarchitecture primarily responsible for modeling primary spatio-temporal patterns that emerge from internal correlations within multivariate sensor networks, especially under non-stationary and incomplete data regimes. GIM originally appeared as a core component in the Primary-Auxiliary Spatio-Temporal network (PAST) for traffic time series imputation, where its design facilitates capturing dynamically evolving dependencies and maintaining robust inference when faced with various missing-data patterns, such as random, fiber, and block-wise deletions (Hu et al., 17 Nov 2025).

1. Architectural Role and Module Coupling

GIM forms part of a dual-stream architecture alongside the Cross-Gated Module (CGM). At each time step $t$, GIM receives:

• the current observation $X_t \in \mathbb{R}^{n \times f}$ (for $n$ sensors, $f$ features), which may be incomplete,
• the previous spatio-temporal hidden state $H_{t-1} \in \mathbb{R}^{n \times d}$,
• and a missing-data mask $M_t \in \{0,1\}^{n \times 1}$.

GIM computes a primary hidden representation $H_t^p \in \mathbb{R}^{n \times d}$ encoding internal sensor correlations. This output is coupled additively with the auxiliary embedding $H_t^a$ produced by CGM, yielding the shared hidden state $H_t = H_t^p + H_t^a$. The summation enables information exchange and mutual adaptation between primary (GIM) and auxiliary (CGM) pattern modeling, supporting long-range temporal propagation under both complete and incomplete observations.

2. Dynamic Graph Construction

GIM recursively constructs a dynamic, time-varying adjacency matrix $A_t \in \mathbb{R}^{n \times n}$, whose entries encode similarity between nodes for each time step. Two approaches are specified:

• Learned-projection with scaled-dot-product
  • $Q_t = H_{t-1} W_Q,\quad K_t = H_{t-1} W_K$ with $W_Q, W_K \in \mathbb{R}^{d \times d_k}$
  • For $i,j$:

  $$A_t^{ij} = \frac{ \exp\left(\mathrm{LeakyReLU}\!\left(Q_t^i(K_t^j)^T / \sqrt{d_k}\right)\right) }{ \sum_{j'=1}^n \exp\left(\mathrm{LeakyReLU}(Q_t^i(K_t^{j'})^T / \sqrt{d_k})\right) }$$

• Gaussian kernel over static node embeddings $E \in \mathbb{R}^{n \times d_e}$

$$A_t^{ij} = \frac{\exp\left(- \|e_i - e_j\|^2 / \delta\right)}{\sum_{j'} \exp\left(- \|e_i - e_{j'}\|^2 / \delta\right)}$$

where $\delta$ is a temperature hyperparameter.

This per-step construction enables the graph to reflect evolving sensor relationships and adapt to new missing positions.

3. Interval-Aware Dropout Mechanism

To increase robustness to missing data and prevent overfitting to transient or spurious correlations, GIM integrates interval-aware dropout on the graph edges:

• For each edge $(i, j)$, sample a Bernoulli variable:

$$R_t^{ij} \sim \mathrm{Bernoulli}(1 - [p_\mathrm{obs} \cdot (1 - M_t^i M_t^j) + p_\mathrm{mis} \cdot (M_t^i M_t^j)])$$

with $p_\mathrm{obs} < p_\mathrm{mis}$, ensuring edges among observed nodes are retained more frequently.

The masked adjacency matrix is then

$\widetilde{A}_t = A_t \circ R_t$

which is used for all subsequent graph convolution operations. This approach preserves connectivity among observed nodes while stochastically regularizing edges tied to missing values.

4. Multi-Order Graph Convolutions

GIM utilizes multi-order (0th to $K$-th) graph convolutions at each layer $l$ to capture dependencies at various spatial hops:

• Define $A_t^0 = I_n$, $A_t^k = (\widetilde{A}_t)^k$ for $k = 1 \ldots K$, and $D_t^k = \mathrm{diag}(A_t^k \cdot \mathbf{1}_n)$
• The forward update:

$$H_t^{(l+1)} = \sigma\left( \sum_{k=0}^K W_k \cdot H_t^{(l)} \cdot (D_t^k)^{-1} A_t^k \right)$$

where $W_k\in\mathbb{R}^{d\times d}$ are learnable parameters and $\sigma$ is applied element-wise (e.g., ReLU or GELU).

• $k=0$ recovers self-features.
• $k=1$ incorporates immediate neighbors.
• $k>1$ enables information flow from multi-hop neighbors, capturing long-range spatial patterns relevant to phenomena such as upstream/downstream propagation in traffic networks.

5. Training, Self-Supervision, and Inference

GIM is trained under an ensemble self-supervised framework:

• Masking for self-supervision: Alongside the genuine missing mask $M_t$, sample an auxiliary mask $S_t$ (uniform Bernoulli rate $r$). Train the model to reconstruct $X_t$ on entries where $S_t=1$ and $M_t=0$.
• Reconstruction loss: Typically L1 or L2 loss over the masked entries:

$$\mathcal{L}_{rec} = \sum_{t=1}^T \|(1 - M_t) \odot S_t \odot (\hat{X}_t - X_t)\|_1$$

• Ensembled predictions: Repeat masking $V$ times, average predictions:

$$\hat{X}_t = \frac{1}{V} \sum_{v=1}^V \hat{X}_t^{(v)}$$

• Implementation settings: Graph-convolution order $K=2$, dropout rates $p_\mathrm{obs}\approx 0.2$, $p_\mathrm{mis}\approx 0.4$, hidden dimension $d=64$, GIM depth $L=2$, ensemble views $V=5$.

Computational complexity is $O(n^2 d_k)$ for $A_t$ (or $O(n k d_k)$ with sparse truncation), $O(K n^2 d)$ for multi-order powers, and overall $O(K|E|d + n d^2)$ per time step (with $|E|$ retained edges).

6. Ablation Analysis and Comparative Advantages

Empirical ablations demonstrate the relative contributions of GIM’s architectural choices on the PeMS-Bay traffic dataset:

• Removing dynamic graph construction ($A_t$ fixed) increases RMSE by 5.3%.
• Restricting to single-order ($K=1$) convolutions increases RMSE by 3.9%.
• Omitting interval-aware dropout causes overfitting in random-missing regimes, raising RMSE by 2.7%.

Dynamic adjacency computation enables adaptation to time-varying patterns and robustness to extensive missingness, outperforming static-topology GCNs particularly under block or fiber-missing scenarios. Multi-order convolutions broaden the receptive field, capturing higher-order dependencies absent in strictly local graph convolutions. Interval-aware dropout regularizes against observation sparsity and maintains learning stability.

7. Context, Applicability, and Broader Significance

The GIM formulation responds specifically to limitations observed in disentangled spatio-temporal models which separately handle spatial and temporal patterns but struggle with adaptation to nonstationary missing mechanisms and long-range dependencies. By allowing the graph topology to evolve with hidden state dynamics and data availability, GIM achieves robust and accurate imputation across 27 missing data conditions, with empirical improvements over seven contemporary baselines—up to 26.2% in RMSE and 31.6% in MAE (Hu et al., 17 Nov 2025). A plausible implication is that graph-integrated modules employing adaptive connectivity and multi-order propagation could prove effective in broader time series domains beset by irregularity and heterogeneity in missingness patterns.

Component	Role in GIM	Notes
Dynamic Graph $A_t$	Models evolving node similarity per time step	Learned or kernel-based
Interval-aware Dropout	Regularizes edge connectivity	$p_\mathrm{mis} > p_\mathrm{obs}$
Multi-order Convolution	Aggregates information up to $K$ hops	Enables long-range dependencies

Further extensions may integrate additional side information in the dynamic graph or leverage more sophisticated self-supervision strategies, but the fundamental methodological foundation of GIM centers on time-dependent, mask-aware, multi-order graph processing for robust primary pattern modeling under pervasive data incompleteness.