Spiking Neural Networks Overview

• Spiking Neural Networks are the third-generation neural models featuring event-driven, spike-based communication and precise temporal coding.
• They employ diverse neuron models, learning rules like STDP and surrogate gradients, achieving near-ANN performance with significant energy savings.
• SNNs are applied in neuromorphic hardware, edge computing, and robotics, with ongoing research targeting scalability, stability, and unified training methods.

Spiking Neural Networks (SNNs) constitute the third generation of neural network models, uniquely characterized by event-driven, sparse, and temporally-coded spike-based communication. Unlike conventional artificial neural networks (ANNs) with continuous-valued signals, SNN neurons propagate discrete spikes whose timing, order, and synchrony encode information. This paradigm provides potential advantages in energy efficiency, temporal processing capabilities, and biological plausibility, especially for edge and neuromorphic hardware. SNNs employ diverse neuron models, learning rules, encoding schemes, and architectures, with a variety of training strategies, ranging from local plasticity to gradient-based optimization and evolutionary algorithms. Recent advances demonstrate SNNs achieving near-parity with deep ANNs on core benchmarks, and ongoing research addresses theoretical foundations, hardware deployment, and unified training methodologies.

1. Neuron Models and Information Encoding

Spiking neuron models can be classified according to their biophysical realism and computational structure. The Hodgkin–Huxley (HH) model captures ionic channel gating and membrane voltage evolution via coupled nonlinear ODEs, serving as a gold standard for biophysical accuracy but is computationally intensive. More tractable abstractions include the Leaky Integrate-and-Fire (LIF) model, where the membrane potential $u(t)$ integrates input currents with an exponential leak; a spike is emitted when $u(t)$ crosses a threshold $\vartheta$, followed by a reset and refractory period. The LIF equation is

$$\tau \frac{\mathrm{d}u}{\mathrm{d}t} = - (u - u_{rest}) + R I_{in}(t),$$

and synaptic currents are typically composed by weighted, delayed spike trains or conductance-based terms (Shen et al., 18 Jun 2024, Jang et al., 2020).

Advanced neuron models such as the Izhikevich system reconcile biological firing patterns with computational efficiency:

$$\frac{dv}{dt} = 0.04v^2 + 5v + 140 - u + I(t), \quad \frac{du}{dt} = a (b v - u),$$

resetting when $v \geq 30$ mV (Shen et al., 18 Jun 2024).

Spike timing and patterns encode information in several ways: rate coding (average firing rates over time), time-to-first-spike (TTFS) latency coding (spike time inversely represents input strength), rank-order coding, and phase-based population codes (Paul et al., 27 Mar 2024, Sakemi et al., 2020, Oh et al., 2020). TTFS and phase codes offer rapid inference and minimal energy usage due to sparse spikes.

2. Learning Rules and Training Algorithms

Learning paradigms for SNNs include local, biologically-plausible mechanisms and global optimization strategies. Spike-Timing-Dependent Plasticity (STDP) adjusts synaptic weights according to the relative timing of pre- and postsynaptic spikes:

$$\Delta w_{ij} = \begin{cases} A_+ \exp(-\Delta t / \tau_+), & \Delta t > 0,\ -A_- \exp(\Delta t / \tau_-), & \Delta t < 0, \end{cases}$$

supporting unsupervised, online adaptation (Paul et al., 27 Mar 2024, Hussaini et al., 2021, Dang et al., 2020).

Global, supervised learning faces the non-differentiability of the spike firing function. Surrogate gradient (SG) methods resolve this by employing smooth approximations (e.g., piecewise linear, sigmoid, arctan) for the derivative of the Heaviside or thresholding function in backpropagation-through-time (BPTT):

$$\frac{\partial S(V)}{\partial V} \approx \sigma(V) = \gamma \max(0, 1 - |V - V_{th}|),$$

allowing gradient-based optimization with high accuracy and convergence rates (Skatchkovsky et al., 2020, Jr, 31 Oct 2025, Guo et al., 2023).

Direct temporal coding approaches, including TTFS and first-to-spike schemes, use spike timing as the score for cross-entropy loss, propagating timing gradients through the network (Oh et al., 2020, Sakemi et al., 2020, Jiang et al., 26 Apr 2024). These permit significant latency and energy savings, although robustness may be affected by device or input variability.

Hybrid/composite strategies leverage knowledge distillation from ANNs to SNNs, aligning spike-based outputs or internal features to ANN teacher distributions (Xu et al., 2023, Guo et al., 2023). Evolutionary algorithms, including genetic algorithms, evolution strategies, and NEAT, optimize SNN weights, synaptic delays, and even architectural/topological characteristics without gradients (Shen et al., 18 Jun 2024).

3. Architectural Design and Optimization

SNN architectures span from dense, feedforward designs to more biologically-plausible network topologies. Early SNNs often repurpose established ANN architectures (VGG, ResNet) into the spiking domain. Modern work increasingly focuses on architecture search tailored to spiking dynamics and temporal sparsity.

Neural Architecture Search (NAS) for SNNs incorporates both forward (feedforward) and backward (temporal feedback) connections, discovering cell/block arrangements that maximize spike diversity and representation power at initialization (Kim et al., 2022). Metrics such as sparsity-aware Hamming distance and kernel determinant scores guide architecture selection without the need for expensive training.

Evolutionary optimization applies population-based algorithms for topology and parameter selection, encoding genotypes as connection matrices and real-valued vectors for delays and weights. Fitness functions penalize energy (spike count, SynOps) or latency alongside accuracy (Shen et al., 18 Jun 2024).

Recent advances include random-feature SNNs and random network architectures (RanSNN), where hidden weights are randomly sampled and fixed, and only output/readout layers are trained. These strategies provide pronounced gains in training efficiency and stability, with performance comparable to fully trained spiking models in convex tasks (Dai et al., 19 May 2025, Gollwitzer et al., 1 Oct 2025).

4. Hardware Implementation and Energy Considerations

SNNs are well-suited for neuromorphic and edge hardware due to their event-driven and sparse computational structure. Device-level implementations exploit analog resistive memory (floating-gate transistors, crossbar arrays) and custom integrate-and-fire circuits to minimize power (Oh et al., 2020, Sakemi et al., 2020). Integration of TTFS coding with refractory circuits yields energy reductions up to $15\times$ versus rate-based SNNs, and decision latency improvements of $5-6\times$ (Oh et al., 2020).

Energy benchmarks in modern SNNs indicate single-inference energy of 5–20 mJ, orders-of-magnitude lower than conventional ANNs (typically $>\!200$ mJ), and spike counts in high-accuracy networks are often below $10^4$ per sample (Jr, 31 Oct 2025, Shen et al., 18 Jun 2024). ASIC, FPGA, and neuromorphic platforms (Intel Loihi, IBM TrueNorth, SpiNNaker, BrainScaleS) provide programmable event-driven cores supporting SNN inference and (in some cases) on-chip learning (Paul et al., 27 Mar 2024, Dang et al., 2020).

Software frameworks such as Brian2, BindsNET, NEST, SLAYER, and Norse support rapid prototyping, surrogate gradient training, and hardware deployment.

5. Performance and Comparative Evaluation

SNN performance is rapidly converging toward that of deep ANNs, particularly on vision benchmarks. Surrogate-gradient trained SNNs achieve accuracy within $1-2\%$ of ANNs (e.g., $97-98\%$ on MNIST, $76-77\%$ on CIFAR-100), often with just $2-5$ simulation timesteps (Guo et al., 2023, Jr, 31 Oct 2025, Xu et al., 2023).

Knowledge-distilled SNNs from complex teacher models yield further accuracy gains and notable robustness to input noise (e.g., $+1.5\%$ for CIFAR-10 under Gaussian perturbation) (Xu et al., 2023). Evolutionary SNNs and random-feature approaches also surpass manually-tuned architectures on key tasks, providing efficient trade-offs between accuracy, latency, and energy (Shen et al., 18 Jun 2024, Gollwitzer et al., 1 Oct 2025).

Comparative latency and spike statistics suggest surrogate-gradient SNNs operate at $10$ ms latency and $10^4$ spikes or fewer per sample; TTFS/FTS models can reduce latency to single-digit ms and spike count to $O($network depth$)$ (Jiang et al., 26 Apr 2024, Oh et al., 2020, Bybee et al., 2022). STDP-based SNNs, while slower to converge, offer the lowest energy consumption and spike sparsity, making them optimal for online unsupervised tasks and edge use cases (Jr, 31 Oct 2025, Dang et al., 2020, Hussaini et al., 2021).

6. Theoretical Foundations and Open Problems

Recent theoretical work established universal approximation theorems for SNNs with LIF neurons, showing that spike-timing parameterizations and threshold–reset dynamics suffice for arbitrary continuous function approximation on compact domains (Biccari, 26 Sep 2025). Constructive encoding via delta-driven and Gaussian-mollified circuits enables both expressivity and practical trainability.

Spike-count dynamics across layers are quantitatively mapped, with rigorous bounds on conditions for stability, monotonicity, and resonance-induced increases in spike counts. Well-posedness of hybrid dynamics is demonstrated, and surrogate gradient mollification is proposed for stable training adjoints.

Open challenges remain in scaling training schemes to deep architectures, hybridizing local plasticity and global credit assignment, hardware-software co-design, coding scheme optimization for real-world signals, biologically-constrained SNN learning, and standardization of benchmarks and APIs (Shen et al., 18 Jun 2024, Paul et al., 27 Mar 2024, Jr, 31 Oct 2025). Future work toward multi-objective EC, hardware-in-the-loop optimization, and integrating heterogeneous neuron types is underway.

7. Applications and Future Directions

SNNs are well-suited for energy-constrained edge AI, low-latency robotics, event-based neuromorphic vision, adaptive control, and spatiotemporal signal processing. Associative memory SNNs perform pattern completion and prototype extraction robustly in sparse, unsupervised regimes (Ravichandran et al., 5 Jun 2024).

Advances in neural architecture search, stochastic/temporal coding, phase-domain spike representations, and knowledge-distilled deep SNNs propel the field toward practical deployment and brain-inspired intelligence (Kim et al., 2022, Jiang et al., 26 Apr 2024, Bybee et al., 2022).

The trajectory involves unification of SNN frameworks, consolidation of performance metrics, scalable hardware integration, and further exploration of mathematical foundations—establishing SNNs as a central paradigm for brain-inspired, real-time, and sustainable computing.