SGNNBench: Benchmark for Spiking GNNs

• SGNNBench is a benchmark for spiking graph neural networks that systematically quantifies accuracy, energy efficiency, and architectural trade-offs across diverse graph datasets.
• It employs three evaluation modules—Effectiveness, Energy Efficiency, and Design-Space Exploration—to compare 9 SGNN variants with 7 full-precision GNNs under unified protocols.
• The benchmark reveals that SGNNs achieve near-GNN accuracy at 5–30× lower energy while also exposing sensitivity to heterophily and the overhead of multi-step processing.

Searching arXiv for the specified SGNNBench paper and closely related benchmark work. SGNNBench is a benchmark for Spiking Graph Neural Networks (SGNNs) on large-scale graph data, introduced to provide a systematic comparison of their effectiveness, energy efficiency, and architectural design across a unified experimental protocol (Zhang et al., 16 Sep 2025). It is motivated by the observation that conventional Graph Neural Networks (GNNs) achieve strong performance on node classification, link prediction, and graph classification through high-precision message passing over graph structure, but incur substantial computation and memory costs on large graphs; in parallel, Spiking Neural Networks (SNNs) on neuromorphic hardware promise lower energy per operation by using sparse binary spikes and replacing most floating-point multiply–accumulate operations with accumulate operations (Zhang et al., 16 Sep 2025). SGNNBench therefore positions SGNNs as an energy-oriented graph learning paradigm whose empirical trade-offs must be quantified rather than assumed.

1. Motivation and problem setting

SGNNBench is framed around the scalability limits of standard GNN pipelines. The benchmark notes that training on a graph with $10^9$ nodes or edges can take days and hundreds of gigabytes of DRAM, making large-scale graph representation learning increasingly difficult to sustain (Zhang et al., 16 Sep 2025). This motivates the search for more energy-efficient architectures.

Within this setting, SGNNs combine SNNs with GNNs by representing node embeddings as spike trains over $T$ time-steps, while different layers communicate through sparse and binary spikes (Zhang et al., 16 Sep 2025). The intended benefit is reduced computation and storage of intermediate graph representations. At the same time, SGNNBench emphasizes that more than a dozen SGNN variants had already appeared without a systematic comparison of their accuracy, memory, and energy trade-offs, nor a unified analysis of how design choices such as neuron model, coding scheme, normalization, and layer depth interact (Zhang et al., 16 Sep 2025).

A central contribution of SGNNBench is therefore methodological rather than architectural: it establishes a fair, reproducible benchmark of 9 SGNNs against 7 classic full-precision GNNs on 18 public graph datasets, organized through three evaluation modules and accompanied by concrete architectural recommendations for a general SGNN paradigm (Zhang et al., 16 Sep 2025).

2. Benchmark framework and evaluation modules

SGNNBench is organized into three modules: an Effectiveness Evaluation Module, an Energy Efficiency Analysis Module, and a Design-Space Exploration Module (Zhang et al., 16 Sep 2025). The benchmark’s structure reflects the view that SGNNs cannot be evaluated solely by predictive performance; their stated rationale requires simultaneous analysis of resource use and architectural sensitivity.

The Effectiveness Evaluation Module trains each model with grid-searched hyperparameters. The search space includes learning rate from $0.1$ to $10^{-4}$, weight decay from $0$ to $10^{-4}$, hidden dimensions in $\{32,64,128,256,512\}$, GNN layers in $\{1,2,4,6,8,10,12\}$, and time-steps $T \in \{2,5,10,15,20\}$ (Zhang et al., 16 Sep 2025). Static datasets are evaluated with accuracy, while dynamic datasets use Macro-F1 and Micro-F1; each setting is run with five random runs (Zhang et al., 16 Sep 2025).

The Energy Efficiency Analysis Module records peak GPU memory in GB and parameter count in MB, and computes a theoretical energy consumption for SNN and ANN variants. The reported formulation is

$$E_{\mathrm{snn}} = \sum_l (E_{\mathrm{ac}} \times FP_l \times T \times R_l), \qquad E_{\mathrm{ann}} = \sum_l (E_{\mathrm{mac}} \times FP_l),$$

where $T$0 pJ, $T$1 pJ, $T$2 denotes the FLOPs in layer $T$3, $T$4 is the number of time-steps, and $T$5 is the average firing rate (Zhang et al., 16 Sep 2025). In the more detailed methodology, $T$6 is measured on the validation split (Zhang et al., 16 Sep 2025).

The Design-Space Exploration Module performs ablations over normalization and neuron choices and over architectural additions. Specifically, it studies BatchNorm versus STFN, LIF versus PLIF neurons, and the effects of a Membrane shortcut (MS), Jumping-Knowledge (JK), and a pre-linear layer (Zhang et al., 16 Sep 2025). This makes SGNNBench not only a leaderboard-style benchmark but also an empirical design study.

3. Model coverage and dataset suite

SGNNBench evaluates 9 state-of-the-art SGNNs and compares them with 7 classic full-precision GNNs (Zhang et al., 16 Sep 2025). The SGNN baselines span both static and dynamic graph settings and include graph convolutional, attention-based, transformer-based, Riemannian, and dynamic-reactive formulations.

Component	Items
SGNN baselines	GC-SNN, GA-SNN, SpikingGCN, SpikeNet, SpikeGCL, SpikeGT, MSG, DRSGNN, SiGNN
Static homophily datasets	Cora, CiteSeer, PubMed, Computers, Photo, Coauthor-CS, Coauthor-Physics, ogbn-proteins, ogbn-arxiv
Static heterophily datasets	Squirrel, Chameleon, Roman-Empire, Amazon-Ratings, Minesweeper, Questions
Dynamic datasets	DBLP, Tmall, Patent

The nine SGNN baselines are specified as follows. GC-SNN and GA-SNN integrate LIF neurons into GCN and GAT backbones with Spatial-Temporal Feature Normalization (STFN); SpikingGCN uses 1-step message passing + BatchNorm + pre-linear layer; SpikeNet is a multi-step spiking GCN with learnable PLLIF neurons and neighbor sampling on dynamic graphs; SpikeGCL performs block-wise contrastive learning on static graphs with a GCN backbone and rate coding; SpikeGT is a spiking Graph Transformer combining MLP and spiking self-attention; MSG is a Riemannian SGNN with Euclid, Sphere, and Lorentz backbones and single-step spiking blocks; DRSGNN is a dynamic reactive SGNN with random-walk positional encoding and an MLP decoder; and SiGNN is a spike-induced GNN with time-varying threshold designed for dynamic graphs (Zhang et al., 16 Sep 2025).

The dataset suite comprises 18 public graph datasets. The static homophily group includes datasets such as Cora with 2,708 nodes, 10,556 edges, and 1,433 features, and ogbn-proteins with 132,534 nodes and 39M edges, evaluated by ROC-AUC (Zhang et al., 16 Sep 2025). The static heterophily group includes Roman-Empire, Amazon-Ratings, Minesweeper, and Questions, with accuracy or ROC-AUC depending on the task (Zhang et al., 16 Sep 2025). The dynamic group includes DBLP with $T$7, Tmall with 577,314 nodes and 4.8M edges at $T$8, and Patent with 2.7M nodes and 14M edges at $T$9, all evaluated by Macro-F1 and Micro-F1 (Zhang et al., 16 Sep 2025).

This breadth is significant because it permits SGNNBench to compare spiking models across homophily, heterophily, and dynamic temporal graphs rather than restricting evaluation to canonical citation benchmarks. A plausible implication is that the benchmark is intended to test whether the energy-oriented promise of SGNNs survives under diverse graph regimes rather than only in favorable static settings.

4. Evaluation methodology and energy model

For effectiveness, SGNNBench reports node classification Accuracy (%) averaged over 5 seeds on static and heterophilous graphs, and Macro-F1 and Micro-F1 on dynamic graphs at training ratios $0.1$0 (Zhang et al., 16 Sep 2025). For efficiency, the benchmark defines parameter size as the number of parameters multiplied by 4 bytes, reported in MB, and records peak GPU memory during training in GB (Zhang et al., 16 Sep 2025).

The benchmark’s theoretical energy model is central to its framing. By distinguishing AC-based computation for SNNs from MAC-based computation for full-precision ANNs, it operationalizes the standard claim that spike-based computation can reduce per-operation energy. However, SGNNBench does not treat this as sufficient evidence of whole-model efficiency. Instead, it uses measured firing rates and layer-wise FLOPs to quantify how time-steps and graph operations interact in practice (Zhang et al., 16 Sep 2025).

This distinction matters because SGNNBench explicitly identifies an often-overlooked issue: the dominant energy bottleneck in SGNNs remains the GNN message-passing layers that were originally designed for high-precision vectors; they do not fully exploit sparse binary spikes (Zhang et al., 16 Sep 2025). This directly counters a common misconception that converting activations to spikes automatically yields end-to-end energy efficiency. The benchmark’s results suggest that SGNN efficiency depends not only on spiking neurons but also on whether the surrounding graph aggregation pipeline is itself redesigned for event-driven computation.

5. Empirical findings on effectiveness and efficiency

On static homophilous graphs, SGNNBench reports that full-precision GNNs such as GCN, SAGE, GAT, and SGC achieve approximately 93.17% on Computers, 95.71% on Photo, 97.03% on Physics, 80.37% on Proteins, and 72.97% on arXiv, whereas SGNNs achieve on average 88.81%, 93.44%, 96.23%, 74.27%, and 67.60%, respectively (Zhang et al., 16 Sep 2025). The resulting overall accuracy gap is reported as approximately 2.7%–5.4% (Zhang et al., 16 Sep 2025). The benchmark’s conclusion summarizes this more compactly by stating that SGNNs attain approximately 90–99% of GNN accuracy at 5–30× lower energy (Zhang et al., 16 Sep 2025).

On static heterophilous graphs, SGNN performance is less stable. The benchmark reports large drops on Amazon-Ratings with −9.26%, Roman-Empire with −31.27%, and Minesweeper with −13.27% relative to the reported GNN averages (Zhang et al., 16 Sep 2025). This establishes heterophily sensitivity as a substantive weakness rather than a secondary anomaly.

On dynamic graphs, the results are more favorable for certain SGNN architectures. SiGNN is reported to top both Macro-F1 and Micro-F1 across DBLP, Tmall, and Patent, and to improve over TGAT by +3.5% to +4.6% Micro-F1 at all ratios (Zhang et al., 16 Sep 2025). SpikeNet and GC-SNN also rank in the top three (Zhang et al., 16 Sep 2025). These findings indicate that SGNN performance is not uniformly inferior; rather, it depends strongly on task regime and architectural specialization.

For efficiency, the benchmark reports that most SGNNs, except SpikeGT and SiGNN, have parameter sizes of no more than 5 MB, with SpikeGCL reaching 0.17 MB on PubMed (Zhang et al., 16 Sep 2025). In memory usage, single-step methods are much lighter than multi-step ones: on PubMed, SpikingGCN uses 0.64 GB and DRSGNN uses 0.41 GB, whereas GC-SNN uses 2.9 GB and GA-SNN uses 3.46 GB (Zhang et al., 16 Sep 2025).

The energy results on PubMed are especially prominent. SpikingGCN at 0.49 mJ and SpikeGCL at 0.52 mJ achieve 29.9× and 27.6× lower energy than SGC at 17.66 mJ, while GC-SNN at 4.6 mJ and GA-SNN at 9.25 mJ reduce the energy of GCN and GAT by 5.1× and 6.4×, respectively (Zhang et al., 16 Sep 2025). By contrast, SiGNN reaches 52.19 mJ because of repeated aggregator use (Zhang et al., 16 Sep 2025). The broader point is that SGNNs do not form a uniform efficiency class; their energy profile depends strongly on whether temporal repetition and graph aggregation are amortized or repeatedly invoked.

6. Design-space investigation and architectural implications

SGNNBench’s design-space study identifies several component-level effects. Replacing LIF with PLIF yields modest gains, such as +0.87% on PubMed for SpikingGCN, but the effect is reported as inconsistent across architectures (Zhang et al., 16 Sep 2025). Similarly, STFN can help certain models, as in GC-SNN on homophily with +3.95% on PubMed, but not uniformly (Zhang et al., 16 Sep 2025). These results suggest that neuron and normalization choices matter, but they are not universal solutions.

The strongest improvements arise from architectural additions. The Membrane Shortcut (MS) introduces a residual path from the membrane potential of layer $0.1$1 to layer $0.1$2, improving average homophily accuracy by 1.83% for GC-SNN, 2.84% for GA-SNN, and 4.38% for SpikingGCN, with gains up to +16.6% on Minesweeper for SpikingGCN+MS (Zhang et al., 16 Sep 2025). Jumping-Knowledge (JK) transfers all intermediate spike embeddings to the output layer and improves performance on deeper architectures, especially on Roman-Empire with +7.83% for GC-SNN (Zhang et al., 16 Sep 2025).

The pre-linear layer is associated with a particularly clear efficiency–accuracy trade-off. Inserting a small input linear projection before the spiking GNN reduces parameter count, memory, and energy while incurring only a limited accuracy loss. For SpikeNet, the reported change is 0.57 to 0.45 MB in parameters, 0.53 to 0.46 GB in memory, and 5.24 to 2.25 mJ in energy, with only 1–2% absolute accuracy drop (Zhang et al., 16 Sep 2025).

SGNNBench distills these observations into four principal recommendations: first, redesign graph aggregation so that it operates on sparse spike events; second, adopt residual shortcuts, jumping-knowledge, and pre-linear layers as defaults; third, avoid relying exclusively on rate coding, whose repetition of static features over $0.1$3 time-steps causes linear growth in memory and latency; and fourth, prefer single-step propagation when scaling to long dynamic sequences (Zhang et al., 16 Sep 2025). The benchmark explicitly cites SpikingGCN and MSG as examples of single-step propagation that dramatically lowers memory, enabling scaling to long dynamic sequences where $0.1$4 (Zhang et al., 16 Sep 2025).

7. Position within the benchmarking landscape and future directions

SGNNBench is described as the first holistic benchmark of spiking graph models on large-scale graphs, with the dual role of exposing their strengths and their weaknesses (Zhang et al., 16 Sep 2025). Its strengths include strong relative accuracy under favorable conditions and substantial theoretical energy reductions; its weaknesses include sensitivity to heterophily and the overhead of multi-step processing (Zhang et al., 16 Sep 2025).

A broader methodological context can be inferred from other recent graph benchmark work. For example, the spectral GNN benchmark of “Benchmarking Spectral Graph Neural Networks: A Comprehensive Study on Effectiveness and Efficiency” implements over 30 models under a unified framework and emphasizes joint evaluation of effectiveness and efficiency, including memory and runtime (Liao et al., 2024). This suggests a wider movement in graph learning toward benchmark designs that evaluate not only accuracy but also computational structure, scalability regime, and systems-level cost. In that sense, SGNNBench extends this benchmarking logic into the spiking domain (Liao et al., 2024).

The open problems identified by SGNNBench are explicit. They include advanced surrogate gradients tailored to graphs, adaptive time-step scheduling, theoretically principled spike encodings for static features, and co-design with neuromorphic hardware such as Loihi and TrueNorth for end-to-end energy measurement (Zhang et al., 16 Sep 2025). These directions indicate that current SGNNs remain only partially aligned with the event-driven assumptions that motivate them. A plausible implication is that future progress will depend less on simply inserting spiking neurons into existing GNN backbones and more on rethinking message passing, temporal coding, and hardware realization as a single design problem.