G2LFormer: Global-to-Local Graph Transformer

• The paper introduces a novel global-to-local attention mechanism that initially captures global dependencies before refining local representations with GNN layers.
• It employs an explicit cross-layer fusion strategy to mitigate over-smoothing and over-globalization by preserving key information from early layers.
• G2LFormer achieves linear complexity and competitive performance on both node-level and graph-level tasks while using substantially less memory than comparable models.

G2LFormer is a Graph Transformer architecture introduced to explore a global-to-local attention scheme in graph representation learning. It applies a global attention layer first, then refines node representations with local GNN layers, and ties the two together with an explicit cross-layer information fusion mechanism. The model is designed to keep linear complexity while mitigating both GNN over-smoothing/over-squashing and Transformer over-globalization, and it is evaluated on node-level and graph-level tasks against state-of-the-art linear GTs and GNNs (Wang et al., 18 Sep 2025).

1. Position within Graph Transformer design

Graph Transformers typically integrate Graph Neural Networks with global attention mechanisms either in parallel or as a precursor to attention mechanisms, yielding a local-and-global or local-to-global attention scheme. In the local-and-global scheme, a local GNN stack and a global attention stack run in parallel and their outputs are combined at the end, for example by summation or concatenation. In the local-to-global scheme, GNN layers first learn local representations and Transformer layers are applied on top to capture global interactions. Models such as SGFormer’s two-view architecture instantiate the former pattern, while GraphTrans and Polynormer typically follow the latter (Wang et al., 18 Sep 2025).

The motivation for G2LFormer is formulated around three issues. First, information loss in local-and-global arises because parallel paths are merged with very simple operations; this shallow integration is described as “oversimplified” and insufficient for effective information exchange. Second, information degradation in local-to-global is linked to over-smoothing and over-squashing in the GNN stack, so noisy or over-smoothed features are passed into global attention and may be amplified. Third, Graph Transformers can overemphasize distant nodes and underutilize immediate neighbors, so deep global layers may shift focus away from local structure. G2LFormer addresses these issues by placing global attention in shallow layers and local GNN modules in deeper layers, so that global layers provide prior global context and local layers perform refined neighborhood aggregation while anchored by that context (Wang et al., 18 Sep 2025).

The resulting design reverses the usual local-to-global pipeline. The stated objective is to prevent final representations from being overly global and forgetting immediate neighbors. A further challenge is that information from shallow global layers can still be forgotten or diluted in deeper local layers, which motivates the explicit cross-layer fusion mechanism.

2. Architectural organization and computation

The backbone of G2LFormer consists of a single linear-complexity attention layer taken from SGFormer and task-dependent local GNN layers. For node classification on large graphs, the local backbone is Cluster-GCN; for graph-level tasks, it is GatedGCN. The overall flow begins with input node features $X \in \mathbb{R}^{N \times d}$, applies one global attention layer to produce the global embedding $h_{TL}$, filters and stores global-layer information through a cross-layer information fusion strategy, and then applies a stack of local GNN layers that iteratively perform local message passing and cross-layer fusion. The final output is denoted $h_{GL}$ (Wang et al., 18 Sep 2025).

The global layer uses a simplified linear attention mechanism:

$Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$

$$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$

$$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$

$$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$

The term $\tilde{Q}(\tilde{K}^\top V)$ captures all-pair interactions in a bilinear factorized form, and $\mathcal{D}$ rescales node outputs to ensure proper normalization and avoid explosion. The paper states that this achieves an equivalent expressive power to full attention while avoiding explicit $N \times N$ similarity matrices, and that the formulation reduces the computation complexity from $h_{TL}$0 to $h_{TL}$1 (Wang et al., 18 Sep 2025).

The local layers are abstracted into a unified GNN update:

$h_{TL}$2

where $h_{TL}$3 is a possibly normalized adjacency matrix, or cluster-wise adjacency in Cluster-GCN, and $h_{TL}$4 is the learnable weight matrix of layer $h_{TL}$5. This is explicitly aligned with the standard MPNN formulation

$h_{TL}$6

with $h_{TL}$7 corresponding to multiplication with $h_{TL}$8, and possibly gating, and $h_{TL}$9 corresponding to the FFN (Wang et al., 18 Sep 2025).

The architectural rationale is stated in explicitly asymmetric terms: global first provides a prior over long-range dependencies, while local later re-emphasizes immediate neighbors. The intended result is that final node embeddings are locally dominated but globally informed.

3. Cross-layer information fusion and NOSAF-style filtering

G2LFormer does not simply stack a global layer followed by local layers. It inserts a cross-layer information fusion strategy between and within local layers, based on NOSAF (Node-Specific Layer Aggregation and Filtration). The purpose is to maintain a running memory of information from all previous layers, including the global layer; to learn node-specific importance scores; and to filter node embeddings in a way that mitigates over-smoothing and preserves global context (Wang et al., 18 Sep 2025).

The fusion mechanism is organized around the variables $h_{GL}$0, $h_{GL}$1, and $h_{GL}$2. Let $h_{GL}$3 denote the output of the global attention layer, $h_{GL}$4 the representation at layer $h_{GL}$5, and $h_{GL}$6 the memory variable storing fused information up to layer $h_{GL}$7. The aggregation variable is defined as

$h_{GL}$8

where $h_{GL}$9 denotes feature-wise concatenation, $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$0 is a zero matrix, and $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$1 are learnable matrices. At layer $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$2, $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$3 encodes global features only. At later layers, $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$4 concatenates projected memory and projected current representation, so that $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$5 represents node-specific aggregated information up to layer $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$6 (Wang et al., 18 Sep 2025).

From $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$7, G2LFormer computes node importance scores:

$Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$8

where $Q = f_Q(X),\quad K = f_K(X),\quad V = f_V(X)$9, $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$0, $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$1, and $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$2. Each node receives an importance score $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$3 that measures how much information at layer $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$4 should contribute to the running memory.

Filtering is defined by

$$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$5

where $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$6 is the Hadamard product and $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$7 broadcasts the node-wise scores across feature dimensions. Memory is then updated by

$$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$8

so that early global information is stored in $$\tilde{Q} = \frac{Q}{\|Q\|_F},\quad \tilde{K} = \frac{K}{\|K\|_F}$$9 and then propagated throughout the local stack. The layerwise definition of $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$0 is

$$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$1

In functional terms, the fusion mechanism acts as a node-wise gating and residual aggregation process across layers. The paper attributes two effects to this construction: it keeps global information alive so that deep local layers do not fully overwrite early global context, and it combats over-smoothing by allowing some nodes to retain distinctive features (Wang et al., 18 Sep 2025).

4. Complexity, scalability, and efficiency

The complexity analysis decomposes G2LFormer into three parts. The global attention layer has complexity $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$2 and does not construct an explicit $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$3 attention matrix. The local GNN layers incur $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$4 message-passing cost per layer, so with a fixed number of layers the overall contribution remains $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$5. The cross-layer fusion contributes $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$6, which is reduced to $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$7 because $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$8 and $$\mathcal{D} = \text{diag}^{-1}\left( \mathrm{I} + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top \mathbf{1}) \right)$$9 are small and constant with respect to $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$0. The paper summarizes the total complexity as

$$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$1

and notes that, for sparse benchmark graphs with $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$2 and $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$3, this is effectively linear in graph size (Wang et al., 18 Sep 2025).

This places G2LFormer in the same asymptotic class as linear Graph Transformers such as SGFormer and Polynormer, while adding only $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$4 overhead for fusion. In the experimental efficiency analysis, G2LFormer is reported to be slower than SGFormer and GCN due to fusion, but to use substantially less memory than Polynormer—specifically 18.4–45.2% lower—and not to suffer out-of-memory failure where Polynormer does, including on ogbn-proteins with batch size 50,000. On synthetic Erdős-Rényi graphs with 10k–100k nodes, both training time and memory scale approximately linearly with $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$5, which is presented as empirical confirmation of $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$6 behavior (Wang et al., 18 Sep 2025).

A plausible implication is that the model’s efficiency profile is defined less by asymptotic deviation from linear Graph Transformers than by the practical cost of the fusion module. The paper presents this as an acceptable trade-off in scalability.

5. Empirical results on node-level and graph-level tasks

The node-level evaluation covers ogbn-arxiv, ogbn-proteins, ogbn-products, and Pokec. The reported metrics are Accuracy for ogbn-arxiv, ogbn-products, and Pokec, and ROC-AUC for ogbn-proteins. The setup notes that ogbn-arxiv uses enriched text features from TAPE, improving baseline performance and training speed. The baselines for node-level tasks include classic GNNs such as GCN and GAT, scalable GNNs such as SIGN and SGC, and linear Graph Transformers such as DIFFormer, SGFormer, and Polynormer. All baselines are either used with published results, when settings match, or re-implemented under the same settings (Wang et al., 18 Sep 2025).

On node classification, G2LFormer reports $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$7 on ogbn-arxiv, compared with the best baseline DIFFormer at $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$8. On ogbn-proteins, G2LFormer reaches $$h_{TL} = \mathrm{FFN}\left( \mathcal{D} \cdot \left( V + \frac{1}{N}\,\tilde{Q}(\tilde{K}^\top V) \right) \right)$$9 ROC-AUC, compared with the best baseline SGFormer at $\tilde{Q}(\tilde{K}^\top V)$0. On ogbn-products, it achieves $\tilde{Q}(\tilde{K}^\top V)$1, compared with Polynormer at $\tilde{Q}(\tilde{K}^\top V)$2. On Pokec, it obtains $\tilde{Q}(\tilde{K}^\top V)$3, compared with Polynormer at $\tilde{Q}(\tilde{K}^\top V)$4. The paper notes that classic GNNs fall significantly behind the GT baselines on these large graphs (Wang et al., 18 Sep 2025).

For graph-level tasks, the evaluation includes Peptides-struct with MAE, Peptides-func with AP, PascalVOC-SP with F1 score, COCO-SP with F1 score, and ogbg-molhiv with ROC-AUC. Baselines include GCN, GIN, GatedGCN, GraphGPS, Exphormer, GRIT, GECO, GPS+Mamba, and GMN (Graph Mamba). G2LFormer reports $\tilde{Q}(\tilde{K}^\top V)$5 ROC-AUC on ogbg-molhiv against GECO at $\tilde{Q}(\tilde{K}^\top V)$6; $\tilde{Q}(\tilde{K}^\top V)$7 AP on Peptides-func against GRIT at $\tilde{Q}(\tilde{K}^\top V)$8; $\tilde{Q}(\tilde{K}^\top V)$9 MAE on Peptides-struct against GRIT at $\mathcal{D}$0; $\mathcal{D}$1 F1 on PascalVOC-SP against GECO at $\mathcal{D}$2; and $\mathcal{D}$3 F1 on COCO-SP against GMN at $\mathcal{D}$4. The paper also remarks that state space models such as GMN and GPS+Mamba perform strongly on some long-range tasks, especially COCO-SP, but that G2LFormer generally matches or exceeds their performance (Wang et al., 18 Sep 2025).

The ablation study compares global-to-local, local-to-global, and local-and-global schemes, with and without cross-layer fusion, on Pokec. Cross-layer fusion improves performance for the global-to-local and local-to-global schemes, but does not improve the local-and-global scheme. Among all tested configurations, global-to-local with fusion yields the best performance. The paper interprets this as evidence that the global-to-local ordering helps alleviate over-globalization and that fusion is critical for fully leveraging both global and local information (Wang et al., 18 Sep 2025).

6. Interpretation, limitations, and research directions

The interpretive argument for G2LFormer is that a global attention layer can build an initial embedding in which each node already “sees” the whole graph, helping to disambiguate nodes with similar local neighborhoods but different global contexts. The subsequent local GNN layers then refine these representations by focusing on the immediate neighborhood, while node-specific fusion gates determine how much of the global signal should be retained or adjusted. This suggests a division of labor in which long-range dependencies are injected early and local structural patterns are restored late, thereby countering the tendency toward over-globalization (Wang et al., 18 Sep 2025).

The paper also states several limitations and trade-offs. Although asymptotic complexity is linear, G2LFormer is slower than plain SGFormer or GCN because of fusion overhead. The effectiveness of the global-to-local scheme may depend on the specific attention and GNN backbones chosen; SGFormer’s attention is described as particularly strong even with a single layer, and Cluster-GCN and GatedGCN are well-tuned baselines for their respective task types. The paper explicitly notes that the optimality of the global-to-local scheme is not fully established and could vary across backbones. It further observes that improvements are modest on some tasks, including ogbn-arxiv and ogbg-molhiv, and that G2LFormer is not universally dominating in every setting but consistently competitive. Hyperparameters such as the fusion dimensions $\mathcal{D}$5 and $\mathcal{D}$6, the number of local layers $\mathcal{D}$7, and backbone choice are identified as factors that can influence both performance and cost, while detailed sensitivity analysis is left for future work (Wang et al., 18 Sep 2025).

Future directions listed in the paper include further study of attention scheme categories, including other interleaving patterns such as alternating global and local layers; backbone improvements, by replacing SGFormer, Cluster-GCN, or GatedGCN with more efficient attention mechanisms or stronger GNNs tailored to specific tasks; theoretical analysis of expressive power, convergence, or bounds for global-to-local schemes; and possible integration with state space models, motivated by the strong performance of GMN and GPS+Mamba. Within the scope of the reported study, G2LFormer is presented as an empirical demonstration that the global-to-local attention scheme is feasible, beneficial, and compatible with linear complexity (Wang et al., 18 Sep 2025).