Hub-and-Spoke Graph Attention (HGA)

• Hub-and-Spoke Graph Attention is a graph transformer paradigm that uses virtual hub nodes to mediate long-range interactions in large graphs.
• Its design integrates a bipartite hub-spoke structure with adaptive hub reassignment to maintain computational efficiency while fully utilizing global context.
• Empirical benchmarks show that the ReHub architecture outperforms traditional message-passing networks by balancing scalability with improved accuracy on diverse tasks.

Hub-and-Spoke Graph Attention (HGA) refers to a graph transformer architecture paradigm that introduces a set of virtual “hub” nodes to mediate long-range interactions among original graph nodes, or “spokes.” This mechanism, exemplified by the ReHub architecture, achieves explicit long-range communication and outperforms traditional message-passing networks in terms of both scalability and effectiveness on benchmark tasks, particularly in large graphs where standard dense attention is computationally prohibitive (Borreda et al., 2024).

1. Architectural Principle and Formalism

Hub-and-Spoke Graph Attention organizes information flow by introducing a bipartite structure between $N_s$ graph nodes (spokes) and $N_h$ virtual nodes (hubs). Each node embedding $s_{i_s} \in \mathbb{R}^d$ is connected via a sparsity-enforcing assignment matrix $E \in \{0,1\}^{N_s \times N_h}$, where $E_{i_s, i_h}=1$ if spoke $i_s$ connects to hub $i_h$. A key constraint is that every spoke connects to exactly $k=O(1)$ hubs, with $N_h = r\sqrt{N_s}$ (where $r \approx 1$), thereby maintaining a moderate number of hubs even as $N_s$ grows.

Attention is computed only for the pairs indicated by $E$, enabling efficient bipartite exchange. For spoke-to-hub attention, each hub aggregates messages from its assigned spokes using standard attention-weighted sums. The entire hub set participates across layers via an adaptive reassignment scheme, ensuring broad information flow without quadratic complexity.

2. Bipartite Attention and Layerwise Operations

HGA utilizes masked bipartite attention to enable communication between spokes and hubs. A single layer comprises:

1. Local Spoke Update: Each spoke is updated by standard message-passing (e.g., MPNN) along original graph edges.
2. Spoke-to-Hub Attention: Hub embeddings are updated using sparse attention pooling over their assigned spokes, via the current assignment mask $E$.
3. Hub-to-Hub Self-Attention: Hubs update themselves based on full attention (all-to-all) within the hub set, using a dense mask.
4. Hub-to-Spoke Attention: Each spoke updates its embedding by attending to the connected hubs, using the adjacency $E^\top$.

These operations ensure that long-range communication, typically facilitated by dense node-to-node attention, is instead mediated in a computationally tractable way through the moderate-size hub set.

3. Adaptive Hub Reassignment Mechanism

To avoid under-utilization of the global hub set—an issue in fixed-assignment variants—ReHub adaptively reassigns spokes to hubs at each layer based on hub-hub similarity:

• Attention Score Computation: After each hub-to-spoke attention pass, each spoke $i_s$ retains attention scores $\Gamma_{i_s, i_h}$ to its $k$ connected hubs.
• Similarity-Based Selection: The most-attended hub $i_h^*$ for each spoke is identified.
• Nearest-Hubs Assignment: Each spoke is then connected to its $k$ nearest hubs (with respect to squared distance $\Delta_{p, q} = \lVert h_p - h_q \rVert^2$ from the most-attended hub), updating the assignment matrix $E$ for the next layer.

This reassignment ensures dynamic coverage of the full hub set over multiple layers without increasing per-layer computational complexity.

4. Computational Complexity and Scalability

Hub-and-Spoke Graph Attention achieves linear time and memory complexity relative to the number of graph nodes, distinguishing it from dense graph transformers with $O(N_s^2 d)$. The complexity components per layer include:

Operation	Complexity	Condition
Local MPNN	$O(N_s d)$	Sparse graphs
Spoke $\to$ Hub / Hub $\to$ Spoke	$O(N_s k d)$	Constant $k$
Hub $\to$ Hub	$O(N_h^2 d)$	$N_h=O(\sqrt{N_s})$
Reassignment	$O(N_h^2 d + N_s k)$	$N_h=O(\sqrt{N_s})$

Total per layer: $O((k + r^2 + 1) N_s d) = O(N_s d)$, linear in the number of spokes. By contrast, methods such as Neural Atoms with $H = O(N_s)$ hubs incur $O(N_s^2)$ cost unless $H$ is reduced to $O(\sqrt{N_s})$, in which case complexity becomes $O(N_s^{3/2})$ (Borreda et al., 2024).

5. Empirical Results and Benchmarks

Empirical evaluations on the Long-Range Graph Benchmark (LRGB) demonstrate that ReHub outperforms Neural Atoms and ranks among the top two across all major tasks and metrics, with competitive or superior accuracy compared to state-of-the-art sparse graph transformers such as GraphGPS, SAN+LapPE, and Exphormer, while requiring fewer computational resources.

Notable results with GatedGCN backbone:

Task	GatedGCN+NeuralAtoms	GatedGCN+ReHub (sparse)
Peptides-Func (AP)	0.6562 ± 0.0075	0.6685 ± 0.0074
Peptides-Struct (MAE)	0.2585 ± 0.0017	0.2512 ± 0.0018
PCQM-Contact (MRR)	0.3258 ± 0.0003	0.3534 ± 0.0014

On large-scale node classification (e.g., OGBN-Arxiv, Coauthor-Physics), ReHub achieves similar accuracy to competitive baselines at significantly lower memory footprint. On synthetic graphs of up to 700,000 nodes, ReHub exhibits strictly linear memory usage and outperforms alternative sparse-attention models in resource efficiency (Borreda et al., 2024).

6. Relation to Prior Work and Design Trade-offs

The use of virtual hub nodes follows Neural Atoms (Li et al., 2024), which showed the performance gains associated with hub-based long-range modeling. However, previous approaches imposed a trade-off between the number of hubs and computational complexity: increasing the hubs improved performance but at the cost of scaling. Hub-and-Spoke Graph Attention resolves this by ensuring each spoke interacts with only a fixed constant number of hubs per layer, while adaptive reassignment grants all spokes periodic access to the entire hub set, maximizing representational power without increased cost.

A plausible implication is that the hub-spoke bipartite structure, coupled with dynamic reassignment, enables both scalability and high expressivity on heterogeneous or large graphs.

7. Summary and Significance

Hub-and-Spoke Graph Attention, as realized in ReHub, mediates explicit long-range node interactions in large graphs through a sparse, dynamically reassigned hub-mediated attention architecture. This framework systematically achieves linear complexity and full utilization of virtual hubs, outperforming earlier hub-based graph transformers on public benchmarks. The adaptive reassignment and bipartite attention design collectively address the scalability bottlenecks of dense attention mechanisms while preserving, and often improving, task performance (Borreda et al., 2024).