---
title: 'SGNNBench: Benchmark for Spiking GNNs'
url: https://www.emergentmind.com/topics/sgnnbench
type: topic
---

# SGNNBench: Benchmark for Spiking GNNs

Searching arXiv for the specified SGNNBench paper and closely related benchmark work.
SGNNBench is a benchmark for **Spiking Graph Neural Networks (SGNNs)** on large-scale graph data, introduced to provide a systematic comparison of their **effectiveness**, **energy efficiency**, and **architectural design** across a unified experimental protocol [2509.21342]. It is motivated by the observation that conventional Graph Neural Networks (GNNs) achieve strong performance on node classification, link prediction, and graph classification through high-precision message passing over graph structure, but incur substantial computation and memory costs on large graphs; in parallel, Spiking Neural Networks (SNNs) on neuromorphic hardware promise lower energy per operation by using sparse binary spikes and replacing most floating-point multiply–accumulate operations with accumulate operations [2509.21342]. SGNNBench therefore positions SGNNs as an energy-oriented graph learning paradigm whose empirical trade-offs must be quantified rather than assumed.

## 1. Motivation and problem setting

SGNNBench is framed around the scalability limits of standard GNN pipelines. The benchmark notes that training on a graph with $10^9$ nodes or edges can take days and hundreds of gigabytes of DRAM, making large-scale graph representation learning increasingly difficult to sustain [2509.21342]. This motivates the search for more energy-efficient architectures.

Within this setting, SGNNs combine SNNs with GNNs by representing node embeddings as **spike trains over $T$ time-steps**, while different layers communicate through sparse and binary spikes [2509.21342]. The intended benefit is reduced computation and storage of intermediate graph representations. At the same time, SGNNBench emphasizes that more than a dozen SGNN variants had already appeared without a systematic comparison of their **accuracy, memory, and energy trade-offs**, nor a unified analysis of how design choices such as **neuron model, coding scheme, normalization, and layer depth** interact [2509.21342].

A central contribution of SGNNBench is therefore methodological rather than architectural: it establishes a **fair, reproducible benchmark of 9 SGNNs against 7 classic full-precision GNNs on 18 public graph datasets**, organized through three evaluation modules and accompanied by concrete architectural recommendations for a general SGNN paradigm [2509.21342].

## 2. Benchmark framework and evaluation modules

SGNNBench is organized into three modules: an **Effectiveness Evaluation Module**, an **Energy Efficiency Analysis Module**, and a **Design-Space Exploration Module** [2509.21342]. The benchmark’s structure reflects the view that SGNNs cannot be evaluated solely by predictive performance; their stated rationale requires simultaneous analysis of resource use and architectural sensitivity.

The **Effectiveness Evaluation Module** trains each model with grid-searched hyperparameters. The search space includes learning rate from $0.1$ to $10^{-4}$, weight decay from $0$ to $10^{-4}$, hidden dimensions in $\{32,64,128,256,512\}$, GNN layers in $\{1,2,4,6,8,10,12\}$, and time-steps $T \in \{2,5,10,15,20\}$ [2509.21342]. Static datasets are evaluated with **accuracy**, while dynamic datasets use **Macro-F1** and **Micro-F1**; each setting is run with five random runs [2509.21342].

The **Energy Efficiency Analysis Module** records **peak GPU memory** in GB and **parameter count** in MB, and computes a **theoretical energy consumption** for SNN and ANN variants. The reported formulation is

$$
E_{\mathrm{snn}} = \sum_l (E_{\mathrm{ac}} \times FP_l \times T \times R_l), \qquad
E_{\mathrm{ann}} = \sum_l (E_{\mathrm{mac}} \times FP_l),
$$

where $E_{\mathrm{ac}} = 0.9$ pJ, $E_{\mathrm{mac}} = 4.5$ pJ, $FP_l$ denotes the FLOPs in layer $l$, $T$ is the number of time-steps, and $R_l$ is the average firing rate [2509.21342]. In the more detailed methodology, $R_l$ is measured on the validation split [2509.21342].

The **Design-Space Exploration Module** performs ablations over normalization and neuron choices and over architectural additions. Specifically, it studies **BatchNorm versus STFN**, **LIF versus PLIF neurons**, and the effects of a **Membrane shortcut (MS)**, **Jumping-Knowledge (JK)**, and a **pre-linear layer** [2509.21342]. This makes SGNNBench not only a leaderboard-style benchmark but also an empirical design study.

## 3. Model coverage and dataset suite

SGNNBench evaluates **9 state-of-the-art SGNNs** and compares them with **7 classic full-precision GNNs** [2509.21342]. The SGNN baselines span both static and dynamic graph settings and include graph convolutional, attention-based, transformer-based, Riemannian, and dynamic-reactive formulations.

| Component | Items |
|---|---|
| **SGNN baselines** | GC-SNN, GA-SNN, SpikingGCN, SpikeNet, SpikeGCL, SpikeGT, MSG, DRSGNN, SiGNN |
| **Static homophily datasets** | Cora, CiteSeer, PubMed, Computers, Photo, Coauthor-CS, Coauthor-Physics, ogbn-proteins, ogbn-arxiv |
| **Static heterophily datasets** | Squirrel, Chameleon, Roman-Empire, Amazon-Ratings, Minesweeper, Questions |
| **Dynamic datasets** | DBLP, Tmall, Patent |

The nine SGNN baselines are specified as follows. **GC-SNN** and **GA-SNN** integrate LIF neurons into GCN and GAT backbones with **Spatial-Temporal Feature Normalization (STFN)**; **SpikingGCN** uses **1-step message passing + BatchNorm + pre-linear layer**; **SpikeNet** is a multi-step spiking GCN with learnable **PLLIF neurons** and neighbor sampling on dynamic graphs; **SpikeGCL** performs block-wise contrastive learning on static graphs with a GCN backbone and rate coding; **SpikeGT** is a spiking Graph Transformer combining MLP and spiking self-attention; **MSG** is a Riemannian SGNN with Euclid, Sphere, and Lorentz backbones and single-step spiking blocks; **DRSGNN** is a dynamic reactive SGNN with random-walk positional encoding and an MLP decoder; and **SiGNN** is a spike-induced GNN with time-varying threshold designed for dynamic graphs [2509.21342].

The dataset suite comprises **18 public graph datasets**. The static homophily group includes datasets such as **Cora** with 2,708 nodes, 10,556 edges, and 1,433 features, and **ogbn-proteins** with 132,534 nodes and 39M edges, evaluated by ROC-AUC [2509.21342]. The static heterophily group includes **Roman-Empire**, **Amazon-Ratings**, **Minesweeper**, and **Questions**, with accuracy or ROC-AUC depending on the task [2509.21342]. The dynamic group includes **DBLP** with $T=27$, **Tmall** with 577,314 nodes and 4.8M edges at $T=186$, and **Patent** with 2.7M nodes and 14M edges at $T=25$, all evaluated by Macro-F1 and Micro-F1 [2509.21342].

This breadth is significant because it permits SGNNBench to compare spiking models across **homophily**, **heterophily**, and **dynamic temporal graphs** rather than restricting evaluation to canonical citation benchmarks. A plausible implication is that the benchmark is intended to test whether the energy-oriented promise of SGNNs survives under diverse graph regimes rather than only in favorable static settings.

## 4. Evaluation methodology and energy model

For **effectiveness**, SGNNBench reports **node classification Accuracy (%) averaged over 5 seeds** on static and heterophilous graphs, and **Macro-F1** and **Micro-F1** on dynamic graphs at training ratios $\{40\%, 60\%, 80\%\}$ [2509.21342]. For efficiency, the benchmark defines **parameter size** as the number of parameters multiplied by 4 bytes, reported in MB, and records **peak GPU memory during training** in GB [2509.21342].

The benchmark’s theoretical energy model is central to its framing. By distinguishing **AC-based** computation for SNNs from **MAC-based** computation for full-precision ANNs, it operationalizes the standard claim that spike-based computation can reduce per-operation energy. However, SGNNBench does not treat this as sufficient evidence of whole-model efficiency. Instead, it uses measured firing rates and layer-wise FLOPs to quantify how time-steps and graph operations interact in practice [2509.21342].

This distinction matters because SGNNBench explicitly identifies an often-overlooked issue: **the dominant energy bottleneck in SGNNs remains the GNN message-passing layers that were originally designed for high-precision vectors; they do not fully exploit sparse binary spikes** [2509.21342]. This directly counters a common misconception that converting activations to spikes automatically yields end-to-end energy efficiency. The benchmark’s results suggest that SGNN efficiency depends not only on spiking neurons but also on whether the surrounding graph aggregation pipeline is itself redesigned for event-driven computation.

## 5. Empirical findings on effectiveness and efficiency

On **static homophilous graphs**, SGNNBench reports that full-precision GNNs such as GCN, SAGE, GAT, and SGC achieve approximately **93.17%** on Computers, **95.71%** on Photo, **97.03%** on Physics, **80.37%** on Proteins, and **72.97%** on arXiv, whereas SGNNs achieve on average **88.81%**, **93.44%**, **96.23%**, **74.27%**, and **67.60%**, respectively [2509.21342]. The resulting overall accuracy gap is reported as approximately **2.7%–5.4%** [2509.21342]. The benchmark’s conclusion summarizes this more compactly by stating that SGNNs attain approximately **90–99% of GNN accuracy at 5–30× lower energy** [2509.21342].

On **static heterophilous graphs**, SGNN performance is less stable. The benchmark reports large drops on **Amazon-Ratings** with **−9.26%**, **Roman-Empire** with **−31.27%**, and **Minesweeper** with **−13.27%** relative to the reported GNN averages [2509.21342]. This establishes heterophily sensitivity as a substantive weakness rather than a secondary anomaly.

On **dynamic graphs**, the results are more favorable for certain SGNN architectures. **SiGNN** is reported to top both **Macro-F1** and **Micro-F1** across **DBLP**, **Tmall**, and **Patent**, and to improve over **TGAT** by **+3.5% to +4.6% Micro-F1 at all ratios** [2509.21342]. **SpikeNet** and **GC-SNN** also rank in the top three [2509.21342]. These findings indicate that SGNN performance is not uniformly inferior; rather, it depends strongly on task regime and architectural specialization.

For **efficiency**, the benchmark reports that most SGNNs, except **SpikeGT** and **SiGNN**, have parameter sizes of **no more than 5 MB**, with **SpikeGCL** reaching **0.17 MB on PubMed** [2509.21342]. In memory usage, **single-step methods** are much lighter than multi-step ones: on PubMed, **SpikingGCN** uses **0.64 GB** and **DRSGNN** uses **0.41 GB**, whereas **GC-SNN** uses **2.9 GB** and **GA-SNN** uses **3.46 GB** [2509.21342].

The energy results on PubMed are especially prominent. **SpikingGCN** at **0.49 mJ** and **SpikeGCL** at **0.52 mJ** achieve **29.9×** and **27.6×** lower energy than **SGC** at **17.66 mJ**, while **GC-SNN** at **4.6 mJ** and **GA-SNN** at **9.25 mJ** reduce the energy of GCN and GAT by **5.1×** and **6.4×**, respectively [2509.21342]. By contrast, **SiGNN** reaches **52.19 mJ** because of repeated aggregator use [2509.21342]. The broader point is that SGNNs do not form a uniform efficiency class; their energy profile depends strongly on whether temporal repetition and graph aggregation are amortized or repeatedly invoked.

## 6. Design-space investigation and architectural implications

SGNNBench’s design-space study identifies several component-level effects. Replacing **LIF** with **PLIF** yields **modest gains**, such as **+0.87% on PubMed for SpikingGCN**, but the effect is reported as **inconsistent across architectures** [2509.21342]. Similarly, **STFN** can help certain models, as in **GC-SNN on homophily with +3.95% on PubMed**, but not uniformly [2509.21342]. These results suggest that neuron and normalization choices matter, but they are not universal solutions.

The strongest improvements arise from architectural additions. The **Membrane Shortcut (MS)** introduces a residual path from the membrane potential of layer $l$ to layer $l+1$, improving average homophily accuracy by **1.83%** for **GC-SNN**, **2.84%** for **GA-SNN**, and **4.38%** for **SpikingGCN**, with gains up to **+16.6% on Minesweeper** for **SpikingGCN+MS** [2509.21342]. **Jumping-Knowledge (JK)** transfers all intermediate spike embeddings to the output layer and improves performance on deeper architectures, especially on **Roman-Empire** with **+7.83% for GC-SNN** [2509.21342].

The **pre-linear layer** is associated with a particularly clear efficiency–accuracy trade-off. Inserting a small input linear projection before the spiking GNN reduces **parameter count**, **memory**, and **energy** while incurring only a limited accuracy loss. For **SpikeNet**, the reported change is **0.57 to 0.45 MB** in parameters, **0.53 to 0.46 GB** in memory, and **5.24 to 2.25 mJ** in energy, with only **1–2% absolute accuracy drop** [2509.21342].

SGNNBench distills these observations into four principal recommendations: first, redesign graph aggregation so that it operates on sparse spike events; second, adopt **residual shortcuts**, **jumping-knowledge**, and **pre-linear layers** as defaults; third, avoid relying exclusively on **rate coding**, whose repetition of static features over $T$ time-steps causes linear growth in memory and latency; and fourth, prefer **single-step propagation** when scaling to long dynamic sequences [2509.21342]. The benchmark explicitly cites **SpikingGCN** and **MSG** as examples of single-step propagation that dramatically lowers memory, enabling scaling to long dynamic sequences where $T \gg 100$ [2509.21342].

## 7. Position within the benchmarking landscape and future directions

SGNNBench is described as the **first holistic benchmark of spiking graph models on large-scale graphs**, with the dual role of exposing their strengths and their weaknesses [2509.21342]. Its strengths include strong relative accuracy under favorable conditions and substantial theoretical energy reductions; its weaknesses include **sensitivity to heterophily** and the **overhead of multi-step processing** [2509.21342].

A broader methodological context can be inferred from other recent graph benchmark work. For example, the spectral GNN benchmark of “Benchmarking Spectral Graph Neural Networks: A Comprehensive Study on Effectiveness and Efficiency” implements over 30 models under a unified framework and emphasizes joint evaluation of effectiveness and efficiency, including memory and runtime [2406.09675]. This suggests a wider movement in graph learning toward benchmark designs that evaluate not only accuracy but also computational structure, scalability regime, and systems-level cost. In that sense, SGNNBench extends this benchmarking logic into the spiking domain [2406.09675].

The open problems identified by SGNNBench are explicit. They include **advanced surrogate gradients tailored to graphs**, **adaptive time-step scheduling**, **theoretically principled spike encodings for static features**, and **co-design with neuromorphic hardware such as Loihi and TrueNorth for end-to-end energy measurement** [2509.21342]. These directions indicate that current SGNNs remain only partially aligned with the event-driven assumptions that motivate them. A plausible implication is that future progress will depend less on simply inserting spiking neurons into existing GNN backbones and more on rethinking message passing, temporal coding, and hardware realization as a single design problem.

Source: https://www.emergentmind.com/topics/sgnnbench