Spike-Driven Hybrid State Space Models

• Spike-Driven Hybrid State Space (SHS) models are frameworks combining continuous state dynamics with discrete spike generation for bio-inspired, energy-efficient computation.
• They implement architectures such as stacked SSM-SNN hierarchies, delay-augmented networks, and probabilistic spike mechanisms to facilitate long-range memory and scalable sequence modeling.
• SHS models have demonstrated competitive accuracy (84–95%) on benchmarks while reducing energy consumption up to 30× compared to traditional dense neural networks.

Spike-Driven Hybrid State Space (SHS) models constitute a framework that integrates continuous-time or discrete-time state-space dynamics—central to modern sequence models—with event-driven, non-linear spike generation, as found in spiking neural networks (SNNs). SHS models enable energy-efficient, sparse, and bio-inspired computation while maintaining the expressivity and long-range memory properties of state-space approaches. This architecture is now foundational across advanced SNNs, hybrid neuromorphic controllers, and practical low-power sequence modeling and pattern recognition systems.

1. Fundamental Principles and Mathematical Formulation

SHS models combine two principal components:

1. State Evolution: Continuous or discrete state variables evolve linearly (or in structured nonlinear fashion) following state-space model (SSM) equations. For a generic neuron or network,

$$x[t+1] = A x[t] + B u[t], \quad y[t]=C x[t] + D u[t],$$

where $x$ is the hidden state, $u$ the (possibly spiking) input, $y$ the subthreshold pre-activation, and $A,B,C,D$ are (possibly learnable) parameter matrices (Karilanova et al., 3 Apr 2025).

2. Spike Generation: A nonlinear function (deterministic e.g., Heaviside, or stochastic, e.g., Bernoulli sampling) discretizes $y[t]$ into binary (or signed) spikes,

$s[t] = \sigma(y[t])$

with potential resets in $x[t]$ depending on $s[t]$ to mimic biological reset behaviors.

The SHS framework generalizes to multiple-input multiple-output (MIMO) neurons, delay-augmented state-variables, and second-order dynamics, supporting architectures beyond classic single-membrane-potential models. Hybrid ("hybrid" in the mathematical sense) dynamics emerge as the state undergoes continuous flows punctuated by discrete spike-driven jumps, as in neuromorphic control setups (Petri et al., 2024).

2. Architectural Variants and Implementations

SHS models are highly modular and support several instantiations:

• Stacked SHS/SNN Hierarchies: Each network layer may consist of an SSM sub-block (for extended memory and recurrent signal mixing) followed by a spiking neuron block (e.g., LIF/adLIF/resonate-and-fire) (Shen et al., 2024, Fabre et al., 4 Jun 2025, Agrawal et al., 16 Oct 2025).
• Hybrid Convolutional/State-Space Designs: For vision, SHS blocks alternate spike-driven convolutions with 1D state-space operations for global context capture, often with residual and attention-like connections (Chen et al., 22 Dec 2025).
• Delay-Augmented SHS: Additional linear shift-register states enable neurons to integrate a temporal buffer of past inputs, allowing efficient modeling of axonal/synaptic delays (Karilanova et al., 1 Dec 2025).
• Probabilistic and Stochastic SHS: SHS models replace hard thresholding with stochastic spike generation and surrogate gradients to facilitate learning and parallelization (Bal et al., 2024).

A concise schematic table of several canonical SHS model types is below:

Model Family	State Update	Spike Mechanism
SSM+LIF	$x_{t+1}=A x_t+B u_t$, $y_t=C x_t$	$s_t=H(y_t-\theta)$
SHaRe-SSM	2nd-order ODE (oscillatory)	$z_t=\Theta(v_t-\theta)$
Delayed SHS	Buffer: $v_d$ shift+sum; $x_{t+1}$ depends on $v_d$	$s_t=H(y_t)$
Probabilistic SHS	SSM state, probabilistic spike draw	$S_t\sim \mathrm{Bern}(p_s[t])$

3. Surrogate Gradients, Sparsity, and Parallel Training

The SHS paradigm leverages binary, stochastic, or non-differentiable spike outputs, which challenge direct gradient-based training. To address this:

• Surrogate Gradients: Hard threshold (Heaviside) functions are replaced by smooth approximations (e.g., triangular, fast sigmoid) for backprop (Shen et al., 2024, Fabre et al., 4 Jun 2025).
• Stochastic Spiking and Surrogate Networks: Some SHS models sample spikes probabilistically and use the expectation in the backward pass (Bal et al., 2024). Surrogate dynamic networks (small parallel 1D convnets) can be trained to mimic the spike reset dynamics, decoupling forward-pass event simulation from backward gradient flow and enabling orders-of-magnitude faster parallelized training for long sequences (Shen et al., 2024).
• Sparsity and Energy Efficiency: The explicit spike-driven computation ensures that only a small fraction of synaptic operations are executed (typical firing rates 4–15%), yielding up to $30\times$ energy reduction compared to dense ANNs (Shen et al., 2024, Agrawal et al., 16 Oct 2025).

4. Extensions: Oscillatory Dynamics, Delays, and MIMO Spiking

Recent advances have extended the SHS framework:

• Second-order / Oscillatory Dynamics: SHaRe-SSM introduces resonate-and-fire neuron blocks, supporting long-range memory with multiplicative-free, event-driven computation and stability via energy-conserving IMEX discretization. Parallel prefix-scan algorithms allow $O(\log L)$ inference complexity (Agrawal et al., 16 Oct 2025).
• State-Space with Delays: SHS neurons may include explicit input history buffers, allowing delayed influences via engineered or learned weightings. This provides enhanced performance, especially in size-constrained settings (Karilanova et al., 1 Dec 2025).
• Multiple-Input Multiple-Output (MIMO) Spiking Neurons: SHS models generalize the neuron abstraction, allowing each unit to process multiple input channels and emit multi-channel spike outputs, improving representational efficiency and reducing parameter counts at similar accuracy (Karilanova et al., 3 Apr 2025).

5. Applications: Event-Based Sensing, Sequence Modeling, and Control

SHS architectures are foundational in:

• Event-Based Perception: In high-resolution, asynchronous sensor tasks (e.g., DVGL, HAR-DVS), SHS blocks provide efficient, latency-minimizing processing by leveraging both sparse event arrivals and continuous memory evolution (Chen et al., 22 Dec 2025, Chakraborty et al., 2 Apr 2025).
• Long-Range Sequence Modeling: SHS models match or surpass prior SNNs on benchmarks such as Long Range Arena, sMNIST/psMNIST, and SHD. They achieve accuracy competitive with SSMs while retaining high sparsity and low compute (Shen et al., 2024, Fabre et al., 4 Jun 2025).
• Neuromorphic Control: Formal SHS hybrid system frameworks enable guarantees of practical stability for spike-driven controllers in closed-loop control applications (Petri et al., 2024).
• Real-Time Neural Decoding: SHS models (e.g., POSSM) with spike tokenization and SSM recurrence provide sub-millisecond inference latencies for online brain–computer interfaces, while enabling multi-dataset pretraining and cross-species transfer (Ryoo et al., 5 Jun 2025).

6. Comparative Evaluation and Design Trade-offs

Quantitative assessments on sequence classification tasks consistently show that SHS models, especially those with hybrid structure and integration of SSM principles, outperform both conventional non-SSM SNNs and earlier spiking RNN architectures (Shen et al., 2024, Agrawal et al., 16 Oct 2025). Design trade-offs arise between the number of internal state variables, neuron count, and channel bandwidth (SIMO/MIMO) (Karilanova et al., 3 Apr 2025). Delay-augmented and second-order dynamics further boost expressivity at modest memory overhead (Karilanova et al., 1 Dec 2025, Fabre et al., 4 Jun 2025).

Empirically:

• SHS models can achieve 84–95% accuracy on benchmarks at 90%+ sparsity, closely approaching or matching full-precision SSMs with far lower energy per inference (Shen et al., 2024, Fabre et al., 4 Jun 2025).
• SHS with kernel-based spike regression attains competitive performance on long-range regression tasks with $73\times$ energy reduction over dense ANN SSMs (Agrawal et al., 16 Oct 2025).
• Control-theoretic SHS guarantee strict lower bounds on inter-spike intervals and uniform practical stability in hybrid feedback schemes (Petri et al., 2024).

7. Outlook and Unifying Perspective

The SHS framework unifies the event-driven, sparse, hardware-friendly properties of spiking networks with the scalable memory and structure of advanced state-space modeling. Embedding SSM kernels, memory-adaptive HiPPO transitions, and multi-branch/delay dynamics into spike-driven infrastructures yields models suitable for sequence, control, and neuromorphic inference at scale, while preserving provable stability and efficient parallelization (Chakraborty et al., 2 Apr 2025, Karilanova et al., 1 Dec 2025, Agrawal et al., 16 Oct 2025).

A plausible implication is that future SNNs, event-based transformers, and neuromorphic sequence models will increasingly adopt the SHS paradigm to balance biological realism, computational tractability, scalability, and cross-modal generalization.

References:

• (Bal et al., 2024, Petri et al., 2024, Chen et al., 22 Dec 2025, Agrawal et al., 16 Oct 2025, Shen et al., 2024, Fabre et al., 4 Jun 2025, Chakraborty et al., 2 Apr 2025, Ryoo et al., 5 Jun 2025, Karilanova et al., 1 Dec 2025, Karilanova et al., 3 Apr 2025)