GraphPFN: Graph Foundation Model

• The paper presents GraphPFN’s integration of a PFN transformer backbone with attention-based graph message passing to perform node-level prediction.
• GraphPFN leverages a synthetic graph prior generated from multi-level SBMs and preferential attachment to create realistic training tasks.
• The model achieves state-of-the-art accuracy on benchmark datasets while incurring higher inference latency and memory consumption than traditional GNNs.

GraphPFN is a Prior-data Fitted Graph Foundation Model for node-level prediction that adapts the prior-data fitted networks (PFN) paradigm to graph-structured data. It combines a PFN Transformer backbone derived from LimiX with attention-based graph neighborhood aggregation layers, and it is pretrained on synthetic attributed graphs sampled from a graph-specific prior. In the primary report, GraphPFN is presented as the first publicly available PFN designed and pretrained specifically on synthetic graph datasets for node-level tasks; in a later fair reevaluation of graph foundation models for node property prediction, it is the overall best-performing evaluated model, while also exhibiting substantially higher inference latency and memory consumption than well-tuned graph neural networks (GNNs) (Eremeev et al., 25 Sep 2025, Platonov et al., 23 Jun 2026).

1. PFN formalization on graphs

GraphPFN inherits the PFN objective of learning amortized posterior predictive inference from synthetic tasks. In the generic PFN formulation, tasks $D$ are sampled from a prior $p(D)$, and the model minimizes the expected negative log-likelihood over held-out examples conditioned on the remaining support set:

$$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$

For graph node property prediction, the conditional likelihood is written as

$p_\theta(y_v \mid G, X, Y_S),$

where $G$ is the graph, $X$ are node features, $Y_S$ are labels for a support subset $S$, and $v$ is the query node (Platonov et al., 23 Jun 2026).

The in-context learning mechanism follows the PFN masking scheme used in transformers: context samples can attend to all context samples, while query samples attend only to context samples and not to one another. On graphs, this yields context-conditioned prediction without per-dataset training, with a single forward pass over labeled support nodes and unlabeled query nodes. A plausible implication is that GraphPFN reframes transductive node prediction as amortized Bayesian inference over support/query partitions rather than as conventional dataset-specific parameter fitting (Eremeev et al., 25 Sep 2025).

2. Architecture and graph-native design

GraphPFN is graph-native rather than a tabularization pipeline. Its backbone is LimiX, which represents each sample as multiple tokens, one per feature, forming a token grid over features and samples. Each transformer block contains two feature-level multi-head attentions, which model within-sample cross-feature interactions, and one sample-level multi-head attention, which enables in-context transfer and uses the PFN mask so that queries only attend to context (Eremeev et al., 25 Sep 2025).

GraphPFN inserts an attention-based message-passing adapter after every LimiX transformer block. This adapter performs sparse multi-head scaled dot-product attention masked by the adjacency, so that a node attends only to its 1-hop neighbors along observed edges. The operation is carried out across samples for each feature independently, using the same adjacency mask across features, and is followed by a feed-forward network; both attention and feed-forward sublayers are wrapped with residual connections and layer normalization. The result is a local graph aggregation mechanism that complements LimiX’s global PFN sample-level attention (Eremeev et al., 25 Sep 2025).

This design is explicitly contrasted with earlier PFN-based graph foundation models such as G2T-FM and TAG. Those methods “bring graphs to the table” by augmenting node features with graph-derived statistics and then applying a tabular PFN backbone, thereby bypassing native graph message passing. GraphPFN instead integrates graph neighborhood aggregation directly inside the PFN. The fair reevaluation summarizes its key design choices as a PFN Transformer backbone with inter-sample attention that supports context-to-context and query-to-context attention, but not query-to-query; graph neighborhood aggregation modules inside the PFN; a synthetic prior over attributed graph node prediction tasks; initialization of non-graph-specific parameters from LimiX-16M; and inference-time ensembling based on 10 stochastic forward passes (Platonov et al., 23 Jun 2026).

3. Synthetic graph prior and pretraining objective

A central component of GraphPFN is its synthetic prior over attributed graphs. Structure generation begins with several first-level graphs drawn from degree-corrected stochastic block models (SBMs), with varying numbers and sizes of blocks and degree sequences. A second-level SBM is then generated with number of nodes equal to the total nodes across the first-level graphs, and a random bijection maps each first-level node to a unique second-level node; edges from the first-level graphs are transferred through this mapping. The model then adds a preferential attachment process, using the second-level graph as a seed and sequentially inserting low-degree nodes that connect to existing nodes with probability proportional to degree. This combination is intended to produce irregular and overlapping communities together with a core–periphery pattern (Eremeev et al., 25 Sep 2025).

Attribute generation is based on graph-aware structured causal models. Starting from a neural SCM similar to those used in TabICL and TabPFN, GraphPFN introduces two graph-aware modifications. First, hidden units are mixed between MLP-type and GNN-type neurons: for each synthetic dataset, a mixing probability $p$ is sampled from $p(D)$0, and a GNN type is sampled from $p(D)$1. Second, with probability $p(D)$2, Laplacian positional encodings are added, with dimension sampled uniformly from $p(D)$3. Synthetic dataset sizes are drawn log-uniformly from $p(D)$4–$p(D)$5 in Stage 1 and $p(D)$6–$p(D)$7 in Stage 2 (Eremeev et al., 25 Sep 2025).

The pretraining loss combines supervised PFN training with masked graph modeling:

$p(D)$8

Here $p(D)$9 is cross-entropy for classification or mean squared error for regression on query nodes conditioned on a randomly sampled context subset. The masked graph modeling term follows MaskGAE: a fraction $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$0 of edges is sampled as positives and removed from the input graph, an equal number of non-edges is sampled as negatives, and a small MLP is trained on the element-wise product of terminal node embeddings to distinguish the two. Pretraining uses $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$1 synthetic graph datasets in two stages: Stage 1 runs for $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$2 optimizer steps on sample sizes $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$3–$$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$4 and takes approximately two days; Stage 2 runs for $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$5 optimizer steps on sample sizes $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$6–$$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$7 and takes approximately five days. Training is conducted on $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$8 NVIDIA A100 80GB GPUs, with one synthetic dataset per GPU per step and 20 gradient accumulation steps, for 160 datasets per optimizer step. AdamW is used with weight decay $$L(\theta) = E_{D\sim p(D)} E_{(x,y)\in D} \left[- \log p_\theta(y \mid x, D \setminus \{(x,y)\})\right].$$9, a cosine schedule with $p_\theta(y_v \mid G, X, Y_S),$0 warmup, base learning rate $p_\theta(y_v \mid G, X, Y_S),$1, and an exponential moving average with decay $p_\theta(y_v \mid G, X, Y_S),$2 in Stage 1 and $p_\theta(y_v \mid G, X, Y_S),$3 in Stage 2. During this synthetic pretraining phase, all LimiX layers are frozen and only the graph adapters and masked-graph-modeling head are trained (Eremeev et al., 25 Sep 2025).

4. In-context learning, fine-tuning, and evaluation regime

In GraphPFN’s in-context learning mode, a small context set of labeled nodes and a query set of unlabeled nodes are supplied from the same graph. The PFN mask ensures that queries attend only to context nodes at the sample-attention level, while the graph adapters propagate information bidirectionally along edges among the nodes included in the forward pass. No conventional training is required on the target dataset; predictions are produced in a single forward pass. Test-time ensembling, defined as averaging predictions over 10 seeds, is reported to further improve in-context learning performance (Eremeev et al., 25 Sep 2025).

GraphPFN also supports fine-tuning. In the original report, fine-tuning is described as full fine-tuning, and, following prior work, 8 random features are concatenated during each forward pass to regularize and improve performance. In the later fair reevaluation, PFN-based graph foundation models, including GraphPFN, are evaluated in both in-context learning and fine-tuning regimes where supported; for fine-tuning, learning rate is tuned by greedy search over 10 values, and other hyperparameters follow official implementations. This reevaluation contrasts PFN-based methods with classic GNNs trained from scratch per dataset under extensive Optuna/TPE hyperparameter search with AdamW, up to 3000 steps, and early stopping with patience 1000 (Eremeev et al., 25 Sep 2025, Platonov et al., 23 Jun 2026).

The fair reevaluation uses 10 real-world node property prediction datasets from GraphLand with the official “RL” splits: $p_\theta(y_v \mid G, X, Y_S),$4 train, $p_\theta(y_v \mid G, X, Y_S),$5 validation, and $p_\theta(y_v \mid G, X, Y_S),$6 test, stratified by label. Directed graphs are converted to undirected. The setting is transductive node property prediction: the full graph is available, only a subset of node labels is observed in train and validation, and the goal is to predict labels on test nodes. Metrics are accuracy for multiclass classification, Average Precision for binary classification, and $p_\theta(y_v \mid G, X, Y_S),$7 for regression; aggregate comparison uses average rank, where lower is better, and average normalized score, obtained by linear normalization per dataset with best model $p_\theta(y_v \mid G, X, Y_S),$8 and worst $p_\theta(y_v \mid G, X, Y_S),$9. All reported metrics are means $G$0 standard deviation over 10 runs, and PFN-based inference includes 10-run ensembling by default (Platonov et al., 23 Jun 2026).

5. Reported empirical performance

In the original GraphPFN report, the model is evaluated on GraphLand datasets—artnet-exp, artnet-views, avazu-ctr, city-reviews, city-roads-M, hm-prices, tolokers-2, and twitch-views—and on classic datasets—amazon-ratings, facebook, pubmed, questions, and wiki-cs. Metrics are Average Precision for binary classification, accuracy for multiclass classification, and $G$1 for regression, with $G$2 train/validation/test splits and 10 trials per setting. Among in-context learning baselines on GraphLand, GraphPFN $G$3 achieves the best average rank, with illustrative results of artnet-exp AP $G$4, city-reviews AP $G$5, tolokers-2 AP $G$6, artnet-views $G$7, avazu-ctr $G$8, city-roads-M $G$9, hm-prices $X$0, and twitch-views $X$1; the average rank is $X$2 on classification and $X$3 over all tasks. After fine-tuning, GraphPFN $X$4 is reported to set new state of the art on 10 of 13 datasets, including artnet-exp AP $X$5, city-reviews AP $X$6, artnet-views $X$7, avazu-ctr $X$8, city-roads-M $X$9, hm-prices $Y_S$0, twitch-views $Y_S$1, facebook accuracy $Y_S$2, questions AP $Y_S$3, and wiki-cs accuracy $Y_S$4; the average rank across classic datasets in fine-tuning is $Y_S$5 (Eremeev et al., 25 Sep 2025).

The later fair reevaluation places GraphPFN in a stricter comparative setting against nine recent graph foundation models and strong GNN baselines. Under this protocol, GraphPFN in fine-tuning is best on all 10 GraphLand datasets and achieves the top aggregate metrics, with average rank $Y_S$6 and average normalized score $Y_S$7. The reported per-dataset results are hm-categories multiclass accuracy $Y_S$8, tolokers-2 binary AP $Y_S$9, city-reviews binary AP $S$0, artnet-exp binary AP $S$1, hm-prices regression $S$2, avazu-ctr regression $S$3, city-roads-M regression $S$4, city-roads-L regression $S$5, twitch-views regression $S$6, and artnet-views regression $S$7. In in-context learning, GraphPFN remains competitive, with average rank $S$8, average normalized score $S$9, and a $v$0 dataset advantage over G2T-FM in the in-context setting. By comparison, the best GNN baseline, Crit.-GAT, achieves average rank $v$1 and average normalized score $v$2. The conclusion of the reevaluation is that only PFN-based graph foundation models surpass strong, well-tuned classic GNNs on these real-world datasets, and GraphPFN is the strongest model among those evaluated (Platonov et al., 23 Jun 2026).

6. Trade-offs, limitations, and distinctions

GraphPFN’s predictive gains are accompanied by significant computational cost. In the fair reevaluation, classic GNNs are consistently sub-second per inference on the tested tasks—for example, Crit.-GCN requires $v$3 and Crit.-GAT $v$4 on city-reviews—whereas GraphPFN requires $v$5 on tolokers-2, $v$6 on city-reviews, $v$7 on artnet-views, $v$8 on city-roads-M, $v$9 on city-roads-L, and $p$0 on hm-prices. Its inference VRAM is likewise high: $p$1 on city-reviews, $p$2 on city-roads-L, and $p$3 on hm-prices, whereas GNNs typically fit within $p$4–$p$5 on the same tasks. The reevaluation attributes this to PFN attention scaling with the number of context and query tokens, the per-feature tokenization used by PFNs, and the fact that 10-run ensembling multiplies inference time when not parallelized. It therefore identifies GraphPFN as advantageous for rapid model development, label-scarce or few-shot regimes, and heterogeneous feature or target spaces across datasets, while recommending GNNs for inference-constrained industrial applications and extremely large graphs (Platonov et al., 23 Jun 2026).

The original GraphPFN report identifies additional limitations. The current implementation processes the full graph in one pass, which constrains very large-scale applicability. The structural prior is described as being tuned toward social and information networks and may not optimally reflect spatial or road networks. The current task scope is node-level prediction; link-level and graph-level tasks are not yet supported, despite masked graph modeling during pretraining. GraphPFN also inherits LimiX’s default limit of at most 10 classes. At the same time, the original ablations report that the graph prior matters: under the same Stage-1 budget, pretraining on simpler random graphs underperforms the proposed multi-level SBM plus preferential attachment prior on most datasets, for example on hm-prices $p$6, where “Ours” gives $p$7 versus ER $p$8, SBM $p$9, and PA $p(D)$00, and on tolokers-2 AP, where “Ours” gives $p(D)$01 versus ER $p(D)$02. Pretraining itself also matters: replacing pretrained adapters with randomly initialized ones causes large drops, including tolokers-2 AP from $p(D)$03 to $p(D)$04, artnet-views $p(D)$05 from $p(D)$06 to $p(D)$07, and hm-prices $p(D)$08 from $p(D)$09 to $p(D)$10 (Eremeev et al., 25 Sep 2025).

A recurrent source of confusion is the similarity between the names “GraphPFN” and “GPFN.” They refer to different models. “GPFN” in "Infinite-Horizon Graph Filters: Leveraging Power Series to Enhance Sparse Information Aggregation" is the Graph Power Filter Neural Network, a shallow GNN framework based on convergent power-series graph filters for infinite-horizon aggregation on sparse graphs. It is not a prior-data fitted network, not a graph foundation model in the PFN sense, and not the model evaluated as GraphPFN in the graph foundation model literature (Zhang et al., 2024).