---
title: Temporal Graph Networks Overview
url: https://www.emergentmind.com/topics/temporal-graph-networks-tgns
type: topic
---

# Temporal Graph Networks Overview

Temporal Graph Networks (TGNs) are deep learning architectures designed to model dynamic graphs—where nodes, edges, and their attributes evolve in continuous time. TGNs formalize dynamic relational systems as sequences of discrete-time or continuous-time interaction events, enabling node-level and edge-level predictions that respect temporal order and context. Established through foundational works such as Rossi et al. ("Temporal Graph Networks for Deep Learning on Dynamic Graphs" [2006.10637]), TGNs combine recurrent node memory, time encoding, and graph-based aggregation to address both structural and temporal complexity in diverse domains including social networks, recommender systems, and time-resolved financial platforms.

## 1. Formulation of Temporal Graphs and the TGN Framework

Dynamic graphs are typically represented in continuous-time event format, where each event is a tuple $(i, j, t, e_{ij}(t))$, with nodes $i$, $j$ interacting at time $t$ and edge features $e_{ij}(t)$. For each node $i$, TGNs maintain a time-indexed memory $s_i(t) \in \mathbb{R}^{d_\text{mem}}$ that summarizes its historical context. Upon an interaction, a raw message is constructed—typically by concatenating pre-event memories, time encodings (such as sinusoidal or learned linear forms), and edge features:
\[
m_i(t) = [ s_i(t^-) \parallel s_j(t^-) \parallel \text{TE}(\Delta t) \parallel e_{ij}(t) ]
\]
where $\Delta t = t - \text{last\_update}(i)$ and TE(·) denotes time encoding. Aggregated messages are written into memory via recurrent update cells (GRU, RNN). Embeddings are computed either directly from node memory or by aggregating over temporally-aware neighborhoods using schemes such as attention or sum/GCN aggregators [2006.10637][2305.01128][2403.16066].

## 2. Core Modules: Memory, Neighborhood Sampling, and Aggregation

TGNs are distinguished by three interacting modules: memory update, neighborhood sampling, and aggregation functions.

- **Memory update**: Nodes use event-driven recurrent modules (typically GRUs) to maintain hidden states, only updating at event times and remaining constant otherwise; this efficiently encodes long-term node and edge state [2006.10637][2412.20256].

- **Neighborhood sampling**: For embedding, nodes sample $k$-hop temporal neighborhoods, most often the $k$ most-recent neighbors sorted by timestamp (most-recent sampling), which empirical evidence shows outperforms uniform or random selection [2412.20256]. Sampling complexity is minimized via temporal-CSR data formats.

- **Aggregation**: Node embeddings are produced by aggregating sampled neighbor memories—with attention aggregators consistently yielding superior predictive metrics compared to MLP-mixer or simple sum [2412.20256][2305.01128][2403.16066]. Attention scores can be computed through scalar MLPs over concatenated memory and edge features, followed by learned output projections.

## 3. Temporal Encoding and Inductive Bias

Time encoding is fundamental in TGNs, injecting recency sensitivity and temporal semantics into the learning pipeline. Two common implementations are sinusoidal encoding:
\[
\text{TE}(\Delta t)_{2k} = \sin(\Delta t / 10{,}000^{2k/d})\,,\quad
\text{TE}(\Delta t)_{2k+1} = \cos(\Delta t / 10{,}000^{2k/d})
\]
and learned linear encoding: $\text{TE}(\Delta t) = W_t \Delta t + b_t$ [2403.16066]. These encodings are concatenated into every message, enabling the model to assess both time gaps and event frequency within dynamic contexts.

## 4. Expressivity, Limitations, and Enhancements

Standard TGNs are expressive but possess limitations. Notable results [2209.15059][2411.03596]:

- Expressivity hinges on injectivity of aggregation and memory update functions. Mean pooling or proportion-invariant (softmax) attention is non-injective, failing to distinguish certain temporal computation trees.
- TGNs cannot represent moving averages, persistent forecasting, or general autoregressive models over messages unless source-target identification (ID encoding) is directly incorporated into message construction (as in TGNv2) [2411.03596]. Augmenting messages with sender/receiver IDs enables exact representation and recall of ordered interaction histories.
- Injective TGNs can match the temporal Weisfeiler–Leman (WL)-test upper bound in distinguishability for anonymous graphs; relative positional encodings further subsume walk-aggregate TGNs [2209.15059].

Trajectory encoding TGNs (TETGN) leverage learnable, auto-expandable node IDs as positional features and perform message passing over these IDs, combining them with standard TGN streams via multi-head attention. This unifies the strengths of anonymous and non-anonymous models for both transductive and inductive settings [2504.11386].

## 5. Implementation, Training Protocols, and Efficiency Techniques

TGNs operate in mini-batch chronological order, preserving temporal causality. Training objectives vary: link prediction is optimized by a negative-sample binary cross-entropy, while recommendation tasks often use Bayesian Personalized Ranking (BPR) loss [2403.16066].

- Optimization typically uses Adam, learning rates $[10^{-4}, 10^{-3}]$, and standard dimensions $d_\text{mem}, d_\text{node}, d_\text{time}$ set by empirical ablation [2403.16066][2305.01128].
- Graph sampling and CSR conversion can become bottlenecks; transformer-based TGNs (TF-TGN) amortize these with highly parallel C++/OpenMP routines and integrate flash-attention kernels, enabling acceleration by factors up to $1{,}466\times$ on massive graphs [2409.05477].
- Continuous training approaches (HAL) convert idle computation due to sparse supervision into productive learning by inserting history-averaged pseudo-labels per batch—reducing gradient variance and accelerating convergence up to $15\times$ without modifying architecture [2505.12526].

## 6. Empirical Results, Ablation Studies, and Design Guidelines

Quantitative benchmarks demonstrate TGNs achieve state-of-the-art performance. Example: on MovieLens, TGN attains Recall@20 of $0.2211$ (+15.4% over DyRep), and RetailRocket Recall@20 of $0.3610$ (+61.2% over SASRec) [2403.16066]. Attention aggregation modules consistently yield the highest recall and precision metrics; memory (GRU-based) modules are critical for long-term dependency tracking, and single-layer TGN architectures suffice when memory is present. Ablations confirm that removing memory or attention results in substantial performance degradation (up to 4 points in AP/AUC) [2305.01128][2006.10637]. Most-recent neighbor sampling plus attention aggregation and dynamic/static memory selection based on dataset repetition yields optimal runtime-accuracy tradeoffs [2412.20256].

## 7. Extensions: Adaptive Neighborhoods, Explanation, and Evaluation Metrics

Recent advances include adaptive neighborhood selection, plug-and-play modules (SEAN), and volatility-aware training:

- SEAN introduces representative neighbor selection (semantic + occurrence-aware attention, diversity penalties) and temporal-aware aggregation (pruned LSTM, outdated-information decay), yielding robust gains in link prediction and AUROC—especially under noisy or expanded-hop conditions [2406.11891].
- Built-in explanation frameworks (TGIB) exploit information bottleneck theory by stochastically gating event subgraphs based on relevance, producing explanations and predictions end-to-end; TGIB achieves higher sparsity-accuracy scores and subsecond explanation latency compared to post-hoc explainers [2406.13214].
- Traditional instance-based evaluation metrics (AP/AUC) fail to capture volatility clustering in temporal errors. Volatility Cluster Statistic (VCS) and its regularized training lead to more temporally uniform error distributions, an important axis for model selection in latency-sensitive settings [2412.07273].

## References

- Rossi et al., "Temporal Graph Networks for Deep Learning on Dynamic Graphs" [2006.10637]
- Tjandra et al., "A Temporal Graph Network Framework for Dynamic Recommendation" [2403.16066]
- Yang et al., "Towards Ideal Temporal Graph Neural Networks" [2412.20256]
- Seo et al., "Self-Explainable Temporal Graph Networks based on Graph Information Bottleneck" [2406.13214]
- Wang et al., "Provably expressive temporal graph networks" [2209.15059]
- Huang et al., "Retrofitting Temporal Graph Neural Networks with Transformer" [2409.05477]
- Li et al., "Towards Adaptive Neighborhood for Advancing Temporal Interaction Graph Modeling" [2406.11891]
- Tjandra et al., "Never Skip a Batch: Continuous Training of Temporal GNNs via Adaptive Pseudo-Supervision" [2505.12526]
- Zhang et al., "Trajectory Encoding Temporal Graph Networks" [2504.11386]
- Zeng et al., "Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networks" [2412.07273]
- Wang et al., "Enhancing the Expressivity of Temporal Graph Networks through Source-Target Identification" [2411.03596]

Source: https://www.emergentmind.com/topics/temporal-graph-networks-tgns