---
title: Spike-Driven Hybrid State Space Models
url: https://www.emergentmind.com/topics/spike-driven-hybrid-state-space-shs
type: topic
---

# Spike-Driven Hybrid State Space Models

Spike-Driven Hybrid State Space (SHS) models constitute a framework that integrates continuous-time or discrete-time state-space dynamics—central to modern sequence models—with event-driven, non-linear spike generation, as found in spiking neural networks (SNNs). SHS models enable energy-efficient, sparse, and bio-inspired computation while maintaining the expressivity and long-range memory properties of state-space approaches. This architecture is now foundational across advanced SNNs, hybrid neuromorphic controllers, and practical low-power sequence modeling and pattern recognition systems.

## 1. Fundamental Principles and Mathematical Formulation

SHS models combine two principal components:

1. **State Evolution:** Continuous or discrete state variables evolve linearly (or in structured nonlinear fashion) following state-space model (SSM) equations. For a generic neuron or network,
   $$
   x[t+1] = A x[t] + B u[t], \quad y[t]=C x[t] + D u[t],
   $$
   where $x$ is the hidden state, $u$ the (possibly spiking) input, $y$ the subthreshold pre-activation, and $A,B,C,D$ are (possibly learnable) parameter matrices [2504.02591].

2. **Spike Generation:** A nonlinear function (deterministic e.g., Heaviside, or stochastic, e.g., Bernoulli sampling) discretizes $y[t]$ into binary (or signed) spikes,
   $$
   s[t] = \sigma(y[t])
   $$
   with potential resets in $x[t]$ depending on $s[t]$ to mimic biological reset behaviors.

The SHS framework generalizes to multiple-input multiple-output (MIMO) neurons, delay-augmented state-variables, and second-order dynamics, supporting architectures beyond classic single-membrane-potential models. Hybrid ("hybrid" in the mathematical sense) dynamics emerge as the state undergoes continuous flows punctuated by discrete spike-driven jumps, as in neuromorphic control setups [2409.06353].

## 2. Architectural Variants and Implementations

SHS models are highly modular and support several instantiations:

- **Stacked SHS/SNN Hierarchies:** Each network layer may consist of an SSM sub-block (for extended memory and recurrent signal mixing) followed by a spiking neuron block (e.g., LIF/adLIF/resonate-and-fire) [2408.14909, 2506.06374, 2510.14386].
- **Hybrid Convolutional/State-Space Designs:** For vision, SHS blocks alternate spike-driven convolutions with 1D state-space operations for global context capture, often with residual and attention-like connections [2512.19365].
- **Delay-Augmented SHS:** Additional linear shift-register states enable neurons to integrate a temporal buffer of past inputs, allowing efficient modeling of axonal/synaptic delays [2512.01906].
- **Probabilistic and Stochastic SHS:** SHS models replace hard thresholding with stochastic spike generation and surrogate gradients to facilitate learning and parallelization [2406.02923].

A concise schematic table of several canonical SHS model types is below:

| Model Family           | State Update                       | Spike Mechanism        |
|----------------------- |------------------------------------|-----------------------|
| SSM+LIF               | $x_{t+1}=A x_t+B u_t$, $y_t=C x_t$ | $s_t=H(y_t-\theta)$   |
| SHaRe-SSM             | 2nd-order ODE (oscillatory)         | $z_t=\Theta(v_t-\theta)$ |
| Delayed SHS           | Buffer: $v_d$ shift+sum; $x_{t+1}$ depends on $v_d$ | $s_t=H(y_t)$ |
| Probabilistic SHS     | SSM state, probabilistic spike draw | $S_t\sim \mathrm{Bern}(p_s[t])$ |

## 3. Surrogate Gradients, Sparsity, and Parallel Training

The SHS paradigm leverages binary, stochastic, or non-differentiable spike outputs, which challenge direct gradient-based training. To address this:

- **Surrogate Gradients:** Hard threshold (Heaviside) functions are replaced by smooth approximations (e.g., triangular, fast sigmoid) for backprop [2408.14909, 2506.06374].
- **Stochastic Spiking and Surrogate Networks:** Some SHS models sample spikes probabilistically and use the expectation in the backward pass [2406.02923]. Surrogate dynamic networks (small parallel 1D convnets) can be trained to mimic the spike reset dynamics, decoupling forward-pass event simulation from backward gradient flow and enabling orders-of-magnitude faster parallelized training for long sequences [2408.14909].
- **Sparsity and Energy Efficiency:** The explicit spike-driven computation ensures that only a small fraction of synaptic operations are executed (typical firing rates 4–15%), yielding up to $30\times$ energy reduction compared to dense ANNs [2408.14909, 2510.14386].

## 4. Extensions: Oscillatory Dynamics, Delays, and MIMO Spiking

Recent advances have extended the SHS framework:

- **Second-order / Oscillatory Dynamics:** SHaRe-SSM introduces resonate-and-fire neuron blocks, supporting long-range memory with multiplicative-free, event-driven computation and stability via energy-conserving IMEX discretization. Parallel prefix-scan algorithms allow $O(\log L)$ inference complexity [2510.14386].
- **State-Space with Delays:** SHS neurons may include explicit input history buffers, allowing delayed influences via engineered or learned weightings. This provides enhanced performance, especially in size-constrained settings [2512.01906].
- **Multiple-Input Multiple-Output (MIMO) Spiking Neurons:** SHS models generalize the neuron abstraction, allowing each unit to process multiple input channels and emit multi-channel spike outputs, improving representational efficiency and reducing parameter counts at similar accuracy [2504.02591].

## 5. Applications: Event-Based Sensing, Sequence Modeling, and Control

SHS architectures are foundational in:

- **Event-Based Perception:** In high-resolution, asynchronous sensor tasks (e.g., DVGL, HAR-DVS), SHS blocks provide efficient, latency-minimizing processing by leveraging both sparse event arrivals and continuous memory evolution [2512.19365, 2504.01257].
- **Long-Range Sequence Modeling:** SHS models match or surpass prior SNNs on benchmarks such as Long Range Arena, sMNIST/psMNIST, and SHD. They achieve accuracy competitive with SSMs while retaining high sparsity and low compute [2408.14909, 2506.06374].
- **Neuromorphic Control:** Formal SHS hybrid system frameworks enable guarantees of practical stability for spike-driven controllers in closed-loop control applications [2409.06353].
- **Real-Time Neural Decoding:** SHS models (e.g., POSSM) with spike tokenization and SSM recurrence provide sub-millisecond inference latencies for online brain–computer interfaces, while enabling multi-dataset pretraining and cross-species transfer [2506.05320].

## 6. Comparative Evaluation and Design Trade-offs

Quantitative assessments on sequence classification tasks consistently show that SHS models, especially those with hybrid structure and integration of SSM principles, outperform both conventional non-SSM SNNs and earlier spiking RNN architectures [2408.14909, 2510.14386]. Design trade-offs arise between the number of internal state variables, neuron count, and channel bandwidth (SIMO/MIMO) [2504.02591]. Delay-augmented and second-order dynamics further boost expressivity at modest memory overhead [2512.01906, 2506.06374].

Empirically:

- SHS models can achieve 84–95% accuracy on benchmarks at 90%+ sparsity, closely approaching or matching full-precision SSMs with far lower energy per inference [2408.14909, 2506.06374].
- SHS with kernel-based spike regression attains competitive performance on long-range regression tasks with $73\times$ energy reduction over dense ANN SSMs [2510.14386].
- Control-theoretic SHS guarantee strict lower bounds on inter-spike intervals and uniform practical stability in hybrid feedback schemes [2409.06353].

## 7. Outlook and Unifying Perspective

The SHS framework unifies the event-driven, sparse, hardware-friendly properties of spiking networks with the scalable memory and structure of advanced state-space modeling. Embedding SSM kernels, memory-adaptive HiPPO transitions, and multi-branch/delay dynamics into spike-driven infrastructures yields models suitable for sequence, control, and neuromorphic inference at scale, while preserving provable stability and efficient parallelization [2504.01257, 2512.01906, 2510.14386].

A plausible implication is that future SNNs, event-based transformers, and neuromorphic sequence models will increasingly adopt the SHS paradigm to balance biological realism, computational tractability, scalability, and cross-modal generalization.

**References:**
- [2406.02923], [2409.06353], [2512.19365], [2510.14386], [2408.14909], [2506.06374], [2504.01257], [2506.05320], [2512.01906], [2504.02591]

Source: https://www.emergentmind.com/topics/spike-driven-hybrid-state-space-shs