---
title: Temporal Graph Benchmark (TGB)
url: https://www.emergentmind.com/topics/temporal-graph-benchmark-tgb
type: topic
---

# Temporal Graph Benchmark (TGB)

The Temporal Graph Benchmark (TGB) is a comprehensive, open-source benchmarking suite designed to facilitate fair, realistic, and reproducible evaluation of machine learning models on temporal (dynamic) graphs. Developed in response to the lack of large, domain-diverse, and protocol-standardized resources for evolving-graph learning, TGB offers curated datasets, rigorous evaluation tasks, and an automated experimental pipeline, catalyzing progress in dynamic graph representation learning and inference.

## 1. Objectives, Scope, and Evolution

TGB aims to provide a unified framework for benchmarking temporal graph learning methods, particularly event-based graph neural networks (TGNNs), across edge prediction (“dynamic link property prediction,” DLPP) and node prediction (“dynamic node property prediction,” DNPP) tasks [2307.01026, 2307.12510]. The benchmark addresses critical gaps in prior work—insufficient scale, domain homogeneity, inconsistent splits, and overoptimistic protocols—by introducing:

- Large-scale, real-world datasets from varied domains (social, transactional, transportation, interaction).
- Strict temporal splits for train/validation/test to avoid future leakage.
- Standardized evaluation metrics and negative sampling.
- Automated pipelines for loading, experimentation, and reproducibility.

Subsequent TGB releases have extended this foundation: TGB 2.0 adds multi-relational heterogeneous graphs for temporal knowledge graph link extrapolation, while TGB-Seq challenges sequence modeling by suppressing edge repetition and emphasizing sequential dynamics [2406.09639, 2502.02975].

## 2. Dataset Suite, Task Formulation, and Protocols

TGB data are represented as chronological streams of timestamped edges, optionally with edge/node features. The suite initially comprised nine large datasets, split into tasks as follows [2307.01026, 2307.12510, 2307.01026]:

| Dataset         | |V|    | |E|        | Domain           | Task(s)           | Temporal Split      |
|-----------------|-------|-------------|------------------|-------------------|---------------------|
| tgbl-wiki       | 9,227 | 157,474     | Wikipedia edits  | DLPP              | Monthly, 70/15/15   |
| tgbl-review     | 352K  | 4.87M       | Amazon/Electronics | DLPP           | 21y, 70/15/15       |
| tgbl-coin       | 638K  | 22.8M       | Crypto-transfers | DLPP              | 8mo, 70/15/15       |
| tgbl-comment    | 995K  | 44.3M       | Reddit replies   | DLPP              | 5y, 70/15/15        |
| tgbl-flight     | 18K   | 67.1M       | Airline bookings | DLPP              | 3y, 70/15/15        |
| tgbn-trade      | 255   | 468,245     | UN trade flows   | DNPP              | 32y, 70/15/15       |
| tgbn-genre      | 1,505 | 17.8M       | User-genre       | DNPP              | 1mo, 70/15/15       |
| tgbn-reddit     | 11.8K | 27.2M       | User-subreddit   | DNPP              | 14y, 70/15/15       |
| tgbn-token      | 61.8K | 72.9M       | ERC-20 tokens    | DNPP              | 1wk, 70/15/15       |

The dynamic link property prediction task is formalized as: for a stream of timestamped events $E = \{ (u, v, x, t) \}$, predict for each source $u$ at time $t$ which of several candidate targets $v$ in a negative pool represents the true next edge. Evaluation is by Mean Reciprocal Rank (MRR), computed as:
$$
\mathrm{MRR} = \frac{1}{|Q|}\sum_{q=1}^{|Q|} \frac{1}{\mathrm{rank}(q)}
$$
where $\mathrm{rank}(q)$ is the position of the true edge for query $q$ among candidates [2307.01026].

The dynamic node property prediction task requires, for each node-time pair $(u, t)$, prediction of a real-valued property (e.g., trade volume, subscriber count) of the node; evaluation is by Normalized Discounted Cumulative Gain (NDCG@K), defined as:
$$
\mathrm{NDCG}@K = \frac{1}{|Q|} \sum_{q} \frac{DCG@K_q}{IDCG@K_q}
$$
where $DCG@K_q = \sum_{i=1}^K \frac{2^{rel_i} - 1}{\log_2(i+1)}$ measures gain for the $K$ top-ranked predictions [2307.12510].

All splits are strictly chronological: first 70% train, next 15% validation, final 15% test, forbidding any test-label leakage.

TGB 2.0 generalizes this to multi-relational temporal knowledge and heterogeneous graphs, using
$$
G = (V, R, T, E),\quad E \subseteq V \times R \times V \times T
$$
and adapts sampling and metric filtration for edge-type and node-type heterogeneity [2406.09639].

## 3. Benchmarking Infrastructure and Methodological Standards

TGB is distributed as a unified Python package (“py-tgb”), providing:

- Data loader: Efficient downloading, preprocessing, and conversion to standard PyTorch Geometric or numpy formats.
- Streamed negative sampling: Supports historical and random negatives for filtered-MRR evaluation; inductive scenario handling.
- Consistent trainer and evaluator: Modular training/evaluation loop for all model types, with in-batch vectorized MRR/NDCG computation.
- Config-driven experiment management: Supports YAML/dict-based hyperparameter search, random seed control, and reproducibility.
- Public leaderboard: Standardized result reporting and online comparison.

The DyGLib_TGB fork further standardizes architectural primitives—neighbor sampling, early stopping, and time encoding—enabling exactly comparable cross-method evaluations [2307.12510]. This addresses substantial “implementation drift” that previously weakened baseline comparisons.

## 4. Algorithms and Empirical Insights

TGB benchmarks a broad array of models, including memory-based, attention-based, hybrid, and non-parametric heuristics [2307.12510]. Representative methods and their distinguishing update formulas include:

- JODIE: Node-specific GRU memory: $m_u^{t} = \mathrm{GRU}(m_u^{t^-}, [h_v^{t^-} \| x_{uv}])$.
- DyRep: Joint self-attention plus temporal point process updates.
- TGAT, TCL: Temporal self-attention and contrastive learning with sinusoidal time encoding.
- TGN: GRU-updated memory plus small GNN over temporal neighbors.
- CAWN: Causal Anonymous Walks aggregated by attention over position and scaled time.
- GraphMixer, DyGFormer: MLP-Mixer or Vision Transformer blocks over neighbor sequences.
- EdgeBank: Memorizes most recent u–v interaction timestamp (“non-parametric”).
- Persistent Forecast, Moving Average: Baselines that predict the last or moving-average observed value for each node.

Experimental results consistently show strong model-dataset interaction. Models with deep temporal processing (TCL, DyGFormer) excel on dense and strongly temporal datasets; memory-centric models (CAWN, TGN) outperform on sparse graphs. Notably, on dynamic node property prediction, trivial non-parametric models (persistence, moving average) often exceed complex TGNNs, signaling insufficient focus on node-centric regression design.

## 5. Protocols, Evaluation Metrics, and Model Pathologies

TGB mandates evaluation using ranking-based metrics directly aligned with end-application decisions, such as filtered MRR (for link prediction) and NDCG@K (for node affinity) [2307.01026]. Negative sampling protocols are tightly controlled, with historical (previously observed) and random negatives mixed; splits ensure strict chronological partitioning.

Critical work has identified that standard negative sampling inflates MRR and enables pathological output saturation (all popular nodes ranked identically), an issue addressed by measures such as MRR$_\text{topN}$—full ranking among top-recently-popular targets—and “Recently Popular Negative Sampling” (RP-NS) [2309.15730]. These improved protocols reveal that “recently popular nodes” baselines (PopTrack: exponential smoothing of node frequencies) can outperform sophisticated models when global temporal dynamics (e.g., mode shifts, social trends) dominate.

Additionally, recent theoretical developments prove that standard permutation-invariant message passing architectures such as TGN cannot recover pairwise statistics required for persistent forecasting or moving-average baselines. This motivates augmentations (TGNv2) that include source–target identification in messages, closing the expressivity gap and substantially improving node-affinity prediction [2411.03596].

## 6. Limitations, Extensions, and Future Directions

Several extensions underscore the dynamic evolution of TGB:

- TGB-Seq [2502.02975]: Focuses on sequential dynamics, suppressing repeated edges to force models to learn $k$-step higher-order interaction sequences. Most extant TGNNs exhibit dramatic performance degradation in this regime, revealing an overreliance on edge repetition memorization and 1-hop aggregation.
- TGB 2.0 [2406.09639]: Addresses the multi-relational and heterogeneous graph regime at scale. Most state-of-the-art methods fail to scale to 10M+ nodes/edges or to utilize edge-type semantics effectively; simple heuristics (EdgeBank, RecB) remain competitive on high-recurrence subgraphs.
- BenchTemp and DynBenchmark [2308.16385, 2510.06245]: Offer further standardization for efficiency, inductive regime stress-testing, node/event classification tasks, and community detection tracking.

Persistent challenges include scalable multi-relational modeling, negative sampling for rare-event and low-recurrence relations, and modeling of fine-grained sequential intention. The empirical dominance of simple heuristics on many TGB tasks, as well as the exposure of architectural limitations (e.g., expressivity barriers and oversquashing), suggests that temporal graph learning remains an open and rapidly evolving field.

## 7. Software, Resources, and Community Practices

All code, datasets, and public leaderboards for TGB and its extensions are available at the canonical repositories and their associated documentation pages:

- TGB: https://tgb.complexdatalab.com/, PyPI package `py-tgb`, GitHub repo [2307.01026].
- DyGLib_TGB: https://github.com/yule-BUAA/DyGLib_TGB (benchmarks, implementations, configuration files) [2307.12510].
- TGB 2.0: https://github.com/JuliaGast/TGB2, dataset scripts, negative samples, and evaluation protocols [2406.09639].
- TGB-Seq: https://tgb-seq.github.io (datasets, code, leaderboards) [2502.02975].
- BenchTemp: https://github.com/qianghuangwhu/benchtemp (modular evaluation pipeline) [2308.16385].

Adherence to TGB protocols, chronological splitting, recorded negative sampling seeds, and transparent leaderboard tracking are emerging as best practices for dynamic graph learning assessment. The open-source and community-driven nature of TGB ensures ongoing updates, expansion to new domains, and rapid dissemination of benchmark-driven insights.

Source: https://www.emergentmind.com/topics/temporal-graph-benchmark-tgb