---
title: 'GeNN Simulator: GPU-Accelerated SNN Simulation'
url: https://www.emergentmind.com/topics/genn-simulator
type: topic
---

# GeNN Simulator: GPU-Accelerated SNN Simulation

The GeNN (GPU Enhanced Neural Network) simulator is a code generation-based framework for the simulation and training of large-scale spiking neural networks (SNNs) on general purpose NVIDIA GPUs. GeNN employs advanced sparse data structures, a high-level C++ and Python interface, and an automated code-generation paradigm to facilitate efficient simulation of both standard synaptic plasticity and dynamically rewiring, structurally plastic networks. GeNN provides a platform for computational neuroscience and neuromorphic machine learning, optimizing simulation speed and memory usage, and supports novel learning algorithms operating on sparse, dynamically changing connectivity [2510.19764][1412.0595].

## 1. System Architecture and Code Generation Workflow

GeNN is structured around a code generation core that emits CUDA (and optionally reference C++) kernels, integrating user-defined neuron, synapse, and plasticity models at the source code level. The architecture features:

- **Backends**: A CUDA backend for GPU acceleration and a C++ backend for reference CPU execution, with planned HIP support.
- **Frontends**: A high-level C++ API and a Python interface (PyGeNN), allowing definition of model structure, plasticity rules, and runtime control [1412.0595].
- **Workflow**:
  1. *Model Specification*: Users instantiate models, define neuron/synapse populations, connectivity patterns, and equations.
  2. *Intermediate Representation*: The input is parsed into a computation graph (populations, synapses, equations, parameters).
  3. *Code Generation*: For each element, the framework emits CUDA C++ kernels, inlining user equations, with host-side code for simulation orchestration.
  4. *Compilation*: The CUDA sources are compiled via `nvcc` and linked into an executable.
  5. *Run-Time*: Host code allocates device memory, configures CUDA launch parameters, and orchestrates kernel invocation over simulation steps.

- **Structural Plasticity Integration**: GeNN implements a “ragged-matrix” format for sparse connectivity, custom update generators for “connectivity update models”, and runtime primitives for connection creation/removal without memory reallocation [2510.19764].

## 2. Modeling Structural Plasticity: e-prop and DEEP R

GeNN supports both synaptic and structural plasticity, including:

- **e-prop (Eligibility Propagation)**: An online learning algorithm for SNNs using local eligibility traces. For adaptive LIF (ALIF) neurons, the subthreshold voltage update and readout dynamics are:
  $$
  \begin{aligned}
  v_j^{t+1} &= \alpha\bigl(v_j^t - z_j^t\,v_{\rm thr}\bigr) + \sum_i W_{ji}^{\rm rec}\,z_i^t + \sum_i W_{ji}^{\rm in}\,x_i^t,\\
  a_j^{t+1} &= \rho\,a_j^t + z_j^t,\\
  z_j^t &= H\bigl(v_j^t - (v_{\rm thr} + \beta\,a_j^t)\bigr),
  \end{aligned}
  $$
  with eligibility and weight update rules:
  $$
  \begin{aligned}
  \epsilon_{ji}^{t+1} &= \psi_j^t\,\bar z_i^t + (\rho - \psi_j^t\,\beta)\,\epsilon_{ji}^t,\\
  \Delta W_{ji}^{\rm rec} &= \sum_t \left( \sum_k B_{jk}(\pi_k^t - \pi_k^{*,t}) \right) \bar e_{ji}^t,
  \end{aligned}
  $$
  where $B=(W^{\rm out})^T$ [2510.19764].

- **DEEP R (Dynamic Evolution of Efficient Plasticity for Rewiring)**: A rewiring algorithm that maintains strict sparsity throughout training by removing "dormant" synapses (those violating fixed sign constraints) and randomly forming new connections. Operations use GPU-parallel bitfields for connection presence and sign, and custom row- and host-level update code for elimination and formation cycles.

- **Integration with mlGeNN/EPropCompiler**: Sparse layers can be specified in a Keras-like interface, and DEEP R rewiring is automatically invoked within supervised e-prop training loops.

## 3. Sparse Data Structures and GPU Kernel Optimization

- **Ragged-Matrix Format**: The primary sparse storage in GeNN is a per-row, fixed-length padded structure where each presynaptic neuron has slots for postsynaptic indices, weights, and delays up to a predefined maximum out-degree $K_\text{max}$.
- **Parallelism**: Weight and target arrays are processed in parallel, with each CUDA thread responsible for propagating spikes or updating synapses. Memory accesses are optimized for coalescence.
- **Connectivity Updates**: Custom GPU "row_update_code" iterates over active connections, leveraging append/swap strategies for efficient addition/removal. Host-side "host_update_code" can compute random indices or counters for GPU.
- **Auxiliary Bitfields**: GPU device arrays encode presence and sign per possible connection, enabling rapid formation/elimination checks with bitwise operations, especially in structural plasticity (DEEP R, topographic map models).
- **Compressed Sparse Row (CSR) Alternative**: Earlier versions used CSR (arrays for values, postsynaptic indices, row pointers), allowing for memory reduction of 5–20× compared to dense storage [1412.0595].

## 4. Performance Benchmarks: Sparse Versus Dense SNNs

Benchmarks conducted on single NVIDIA A100 GPUs demonstrate:

| Sparsity          | Relative Training Time | Parameter Count | N-MNIST Accuracy      | DVS-gesture Accuracy  |
|-------------------|-----------------------|-----------------|----------------------|----------------------|
| 100% (Dense)      | 1×                    | 1×              | 99.19 ± 0.1 %        | 88.94 ± 0.6 %        |
| 5% (Fixed)        | 1/8×                  | 1/4×            | 98.2 ± 0.3 %         | 83.5 ± 1.0 %         |
| 5% + DEEP R       | 1/8×                  | 1/4×            | 99.15 ± 0.1 %        | 88.86 ± 0.3 %        |
| 1% (Fixed)        | 1/15×                 | 1/11×           | 96.8 ± 0.4 %         | 75.2 ± 2.5 %         |
| 1% + DEEP R       | 1/15×                 | 1/11×           | 99.12 ± 0.2 %        | 88.30 ± 0.5 %        |

Sparse DEEP R models match dense accuracy provided input connections remain $\gtrsim5\%$. Training time reductions approach 10–15× with up to 90-fold parameter saving for recurrent classifiers, while DEEP R overhead remains below 1% of step time [2510.19764].

## 5. Structural Plasticity in Unsupervised Topographic Map Formation

GeNN supports unsupervised learning scenarios such as spatial topographic map formation:

- **Model Structure**: Two grid-based layers with Poisson input sources deliver spatially-correlated stimuli; target layer employs all-to-all spike-timing-dependent plasticity (STDP) combined with structural rewiring at fixed time intervals.
- **Plasticity Rules**:
  - **Elimination Probability**: For synapse $(i,j)$,
    $$
    p_{\rm elim} =
      \begin{cases}
        p_{\rm elim\_dep} & \text{if }g_{ij}<g_\theta \\
        p_{\rm elim\_pot} & \text{if }g_{ij}\geq g_\theta
      \end{cases}
    $$
  - **Formation Probability**:
    $$
    p_{\rm form}(d) = p_{\rm peak} \exp\left({-d_{ij}^2/(2\sigma^2_{\rm form})}\right)
    $$
- **Dynamics**: Formation/elimination stabilizes after $\sim$200 ms of biological time. Mean in-/out-degree converge; receptive fields sharpen spatially. For $L_0\times L_0$ layers ($s=1\dots7$), simulation remains faster-than-realtime for smaller scales, with runtime dominated ($\sim$90%) by connectivity updates at high structural plasticity frequency.

| Scale $s$ | Neurons/Layer | Real-Time Factor | Connectivity Update Share |
|-----------|--------------|------------------|-------------------------|
| 1         | 256          | 2× faster        | ~90%                    |
| 7         | 12,544       | ~0.2× real-time  | ~90%                    |

The step time increases only by $\sim$10× for a $49×$ network size increase [2510.19764].

## 6. Guidelines and Practical Trade-offs for Sparse Simulation

- **Initialization**: Effective learning and performance require each neuron’s input degree to remain at least $5\%$ of possible connections.
- **DEEP R Sparsity**: Maintains constant connection density with sub-1% computational overhead. Sparse models often require longer training to reach the same accuracy (consistent with lottery-ticket hypothesis).
- **Memory and Performance**: Ragged-matrix padding yields sublinear memory savings as sparsity increases. For high-frequency rewiring (topomap scenario), connectivity updates may dominate overall runtime.
- **Optimization**: Code-generated CUDA kernels are automatically tuned for occupancy $\geq75\%$; block sizes are multiples of 32 (warp size) [1412.0595].
- **Scalability**: A single GPU with GeNN can emulate the performance of small CPU clusters for SNN tasks, reducing hardware cost and power consumption [1412.0595].

The GeNN simulator combines code-generation strategies with advanced data structures, delivering an extensible platform for high-throughput investigation of sparse, plastic, and structurally adaptive spiking neural networks at large scale [2510.19764][1412.0595].

Source: https://www.emergentmind.com/topics/genn-simulator