Temporal Context Graph Selection

• Temporal Context Graph Selection is a framework for selecting and aggregating time-relevant subgraphs using adaptive attention, sampling, and decay techniques.
• It integrates long-range, heterogeneous, and nonlocal temporal information to boost performance in tasks like link prediction and node classification.
• Its design reduces computational complexity by leveraging latent token bottlenecks and transformer-based attention for scalable dynamic graph processing.

Temporal Context Graph Selection (T-CGS) is a class of mechanisms for selecting, ranking, and aggregating temporally relevant context from evolving graphs to inform node and edge embeddings. T-CGS underlies state-of-the-art models in link prediction, node classification, and other temporal graph analytic tasks by enabling the integration of long-range, heterogeneous, and nonlocal temporal information, often in conjunction with explicit structural (spatial) context. Its implementations span transformer-based adaptive attention, relevance-weighted subgraph sampling, and learnable, multi-stage context selectors with temporal decay and pruning. T-CGS addresses the challenge of selecting the most informative temporal subgraphs or interaction histories for downstream models on dynamic and relational graph data, providing substantial gains in predictive accuracy and robustness (Varghese et al., 2023, Lachi et al., 6 Nov 2025, Zhang et al., 2024).

1. Formal Definitions and Problem Setting

T-CGS approaches operate on temporal or dynamic graphs, where the vertex, edge, and neighborhood structure evolves over continuous or discrete time:

• In heterogenous temporal relational graphs, $G = (\mathcal{V}, \mathcal{E}, \phi, \psi, \tau)$, vertices and edges are equipped with timestamps, and optionally node and edge types via $\phi$, $\psi$. The timestamp map $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$ assigns times of occurrence to all entities. The core objective is, for a given query (or seed) node $v_\text{seed}$ with reference time $t_\text{seed} = T_v(v_\text{seed})$, to select a subgraph $$G_\text{temp} = (\mathcal{V}_\text{temp}, \mathcal{E}_\text{temp})$$ that captures temporally “proximate” edges and nodes (Lachi et al., 6 Nov 2025).
• In temporal interaction graphs, $G = (V, E, T)$, where $E = \{(i, j, t)\}$ indicates time-stamped interactions, T-CGS aims to select, for each node $i$ at time $\phi$0, an adaptive context subgraph $\phi$1 (the historical neighborhood), for expressive, temporally relevant aggregation (Zhang et al., 2024).
• In transformer-based approaches, for each node $\phi$2 and time $\phi$3, a sequence of zero-padded neighborhood indicators over a lookback window $\phi$4 is assembled to build the historical context $\phi$5 (Varghese et al., 2023).

2. T-CGS Mechanisms and Algorithms

The central aspect of T-CGS is explicit, data-driven selection or weighting of temporally salient context, either as a subgraph or as a sequence, for integration in the embedding process. Implementations fall into three broad paradigms:

2.1 Transformer-Based Adaptive Attention

TransformerG2G computes per-node, per-time embeddings via stacked self-attention over temporal neighborhood vectors:

1. Input Encoding: Each $\phi$6 (node $\phi$7 neighborhood at time $\phi$8) is projected and positionally encoded, yielding $\phi$9.
2. Self-Attention: Scaled dot-product attention is applied:

$\psi$0

1. Temporal Attention Weights: The bottom row of $\psi$1 gives attention $\psi$2 to times $\psi$3 when constructing the node embedding at $\psi$4.
2. Adaptive Step Selection: Thresholding $\psi$5 induces an effective lookback $\psi$6, adaptively focusing attention on the most informative prior steps without explicit hyperparameters.
3. Latent Projection: Outputs are aggregated and mapped to latent Gaussian parameters $\psi$7 via an MLP (Varghese et al., 2023).

2.2 Temporal Subgraph Sampling

In the Relational Graph Perceiver framework, a temporal subgraph sampler extracts a context graph by relevance scoring:

• Edge Scoring: For each edge $\psi$8, $\psi$9.
• Selection: Either edges within a window $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$0 or top-$\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$1 edges with smallest $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$2 are included.
• Integration: Temporally and structurally sampled subgraphs are jointly encoded via cross-attention over shared latent tokens, then fused via transformer layers for downstream tasks (Lachi et al., 6 Nov 2025).

2.3 Adaptive and Learnable Neighborhood Selection

The SEAN model instantiates T-CGS as a two-stage process:

1. Representative Neighbor Selection: Softmax-normalized, occurrence-aware attention weights $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$3 combine semantic similarity, historical co-occurrence, and time encodings. Neighbor diversity is encouraged via a penalty term, avoiding overfitting to high-degree or frequent neighbors.
2. Temporal-Aware Aggregation: LSTM-based modules maintain cell states with adaptive pruning gates and exponential outdated-decay on short-term memory, providing a mechanism for discounting temporally distant or obsolete information (Zhang et al., 2024).

A sample high-level comparison is organized below:

T-CGS Variant	Context Selection	Aggregation Mechanism
TransformerG2G	Softmax attention over past snapshots	Transformer encoder with learned adaptive time step
RGP Subgraph Sampler	Rank/threshold edges by temporal proximity	Cross-attention latent bottleneck followed by transformer
SEAN	Occurrence-aware, learnable attention	LSTM with outdated-decay and adaptive pruning

3. Integration in Downstream Temporal Graph Embedding Models

T-CGS is a modular operation that can be embedded into a wide range of GNN, transformer, or hybrid architectures:

• TransformerG2G (dynamic graphs): T-CGS is realized by attention over temporal snapshot sequences, with adaptive step selection improving robustness and sample efficiency, especially for “novel” patterns (Varghese et al., 2023).
• Relational Graph Perceiver: T-CGS samples both temporal and structural subgraphs, with output embeddings constructed through a bottlenecked cross-attention, enabling scalable, multi-task relational learning (Lachi et al., 6 Nov 2025).
• SEAN (Selective Encoding for Adaptive Neighborhoods): T-CGS is a differentiable, multi-phase context filter and aggregator, introducing personalized, timestamp-aware context selection compatible with any temporal graph network backbone (Zhang et al., 2024).

The shared design principle is the explicit, data-driven prioritization and weighting of temporally relevant context, allowing models to focus computational and representational resources on the most informative local and nonlocal histories.

4. Experimental Evaluation and Impact

Robust empirical studies have established that T-CGS modules deliver substantial improvements on benchmark tasks:

• Datasets: RelBench (AUC-ROC), CTU Prague (macro F1), SALT (MRR), as well as public and financial temporal graph datasets (Lachi et al., 6 Nov 2025, Zhang et al., 2024).
• Ablation Results: Removing the temporal sampler in RGP yields a 3–4% absolute drop in AUC on driver prediction tasks, highlighting the value of temporal over purely structural context (Lachi et al., 6 Nov 2025).
• Novelty Handling: In TransformerG2G, attention weights “shift” dynamically: rapidly-evolving node neighborhoods put weight on very recent history, while stable regions utilize longer context. This endows the model with strong temporal priors when edge novelty is low, but also with flexibility to attend to more distant history when predicting novel edge patterns (Varghese et al., 2023).
• Adaptivity and Robustness: SEAN’s T-CGS ensures both personalization and resistance to neighborhood inflation, consistently delivering state-of-the-art performance across models and datasets (Zhang et al., 2024).

5. Computational Complexity and Scalability

A defining feature of several T-CGS implementations is improved scalability over prior approaches:

• Latent token bottleneck (RGP): Cross-attention from $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$4 latents to $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$5 nodes reduces cost to $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$6, a major reduction from the $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$7 cost in legacy full-attention graph transformers. This enables processing of larger subgraphs and wider context per seed node (Lachi et al., 6 Nov 2025).
• Pure attention-based models: TransformerG2G maintains linear cost in lookback window $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$8 and node dimension $\tau: \mathcal{V} \cup \mathcal{E} \to \mathbb{R}$9, with temporal context selection requiring only thresholding or softmax selection over $v_\text{seed}$0 steps per node (Varghese et al., 2023).

6. Limitations and Extensions

T-CGS, while effective, has several known limitations and trajectories for further development:

• Hard vs. Learnable Selection: Current samplers often use hard windowing or ranking; replacing these with learnable temporal kernels (e.g., parameterized RBF, as suggested in (Lachi et al., 6 Nov 2025)) represents a plausible extension.
• Noisy/Missing Timestamps: Performance can revert to purely structural sampling when timestamps are sparse or unreliable, losing temporal modeling advantages (Lachi et al., 6 Nov 2025).
• Streaming Graphs: Most T-CGS variants sort or precompute edge timesteps; supporting streaming or online updates demands more dynamic context selection.
• Comparison to Alternate Graph Samplers: Traditional GNN temporal samplers in RelBench and RGT only use timestamp filters for future-leak prevention, not as a modeling signal. T-CGS explicitly prioritizes by temporal proximity, enabling the capture of structurally distant but temporally synchronous events—a key differentiator from sequential message-passing approaches (TGAT, TGN) (Lachi et al., 6 Nov 2025).

7. Summary and Outlook

Temporal Context Graph Selection constitutes a critical mechanism for equipping graph neural architectures with fine-grained, adaptive access to time-varying relational information. By automatically selecting, weighting, and aggregating temporally relevant neighborhoods—via attention, relevance scoring, or learnable aggregation modules—T-CGS confers improved predictive performance, robustness to novelty and data drift, and scalability to large, heterogeneous graphs across diverse domains including healthcare, e-commerce, and finance.

Ongoing research directions include learning fully differentiable temporal relevance functions, unified joint structural-temporal samplers, and the extension of T-CGS modules to high-velocity streaming data where topology and event arrivals evolve continuously (Varghese et al., 2023, Lachi et al., 6 Nov 2025, Zhang et al., 2024).