Selective Spike Dropping in SNNs

• Selective spike dropping is an algorithmic strategy in spiking neural networks that suppresses non-salient spikes based on programmable conditions.
• It employs both neuron-level dynamics and event-queue mechanisms to optimize computational efficiency, reducing latency, memory, and energy usage with minimal accuracy loss.
• Empirical evaluations on datasets like Fashion-MNIST and CIFAR demonstrate substantial efficiency gains, with notable reductions in energy consumption and memory demands.

Selective spike dropping denotes a class of algorithmic and architectural strategies in spiking neural networks (SNNs) and spiking simulation systems where candidate spikes—either neuron-generated or event-queued—are explicitly suppressed according to programmable conditions. The method enables reductions in inference latency, memory and energy consumption, and system complexity while minimally affecting the information content of spike trains or network function. Recent work distinguishes two principal operational contexts: (1) neuron-intrinsic spike suppression based on membrane dynamics, exemplified by the Self-Dropping Neuron in single-timestep SNNs, and (2) event-queue-level selective dropping to limit queue growth on accelerators and simulators. Both contexts implement a defined, loss-tolerant trade-off between compute efficiency and signal fidelity (Xu et al., 1 Aug 2025, Landsmeer et al., 5 Dec 2025).

1. Mechanisms of Selective Spike Dropping in SNN Architectures

Neuron-level spike dropping is exemplified by the Self-Dropping Neuron (SD neuron) mechanism, which augments the classic Leaky Integrate-and-Fire (LIF) dynamics. The SD neuron operates over an accumulation window of $t_\ell$ sub-steps per layer $\ell$, but only emits spikes under selective conditions. The membrane potential update employs a soft reset: $$u^{(t,\ell)} = \gamma\,u^{(t-1,\ell)} - V_{\rm th}^{(\ell)}\,o^{(t-1,\ell)} + x^{(t,\ell-1)},$$ where $\gamma$ is the leak constant and $V_{\rm th}^{(\ell)}$ a dynamic threshold. A spike is emitted if both $u^{(t,\ell)} > V_{\rm th}^{(\ell)}$ (membrane above threshold) and $u^{(t,\ell)} < u^{(t-1,\ell)}$ (potential decaying), with the spike amplitude given by $\lfloor u^{(t,\ell)}/V_{\rm th}^{(\ell)}\rfloor$, capped at $o_{\max}$. Otherwise, the spike is suppressed—this operation is termed selective spike dropping (Xu et al., 1 Aug 2025).

At the system level, only the final output of $t_\ell$ mini-steps is propagated. Thus, only a neuron’s peak information-carrying event per window is transmitted, enhancing per-spike saliency and reducing redundant bursts.

2. Selective Dropping in Spike Event Queues

In large-scale brain simulation and SNN training frameworks, event queues accumulate delayed spike events. Unbounded delays or high firing rates can induce memory overflow on AI accelerators; thus, “lossy” queue designs implement per-slot or per-queue spike caps, enacting selective spike dropping when capacity is reached (Landsmeer et al., 5 Dec 2025).

For a Poisson spike process with rate $\lambda$ per timestep and delay window $d$, the expected in-flight spike count is $\mu = \lambda d$. Capping the queue size at $C$ enforces a drop-rate

$$\alpha(\lambda, d; C) = 1 - \sum_{k=0}^C e^{-\lambda d}\frac{(\lambda d)^k}{k!}$$

for $K \sim \text{Poisson}(\lambda d)$. Empirically, under typical network parameters $\alpha$ remains $\ll1\%$ for modest $C$, with minimal error in aggregate synaptic currents. Two prevalent queue structures, LossyRingDelay and FIFORing, implement these strategies in JAX for autodifferentiable simulation (see Section 4).

3. Dynamic Thresholds and Condition Scheduling

The SD neuron employs a dynamic threshold schedule over its window, $V_{\rm th}^{(\ell)} = V_{\rm th,0}/t_\ell$, regulating sensitivity and suppressing spikes during low-input or ascending-potential phases. This mechanism prevents quiescence (no spiking) at short $t_\ell$ and mitigates excess firing at larger $t_\ell$. In event queues, a static drop cap $C$ is standard, but future extensions envision adaptive or saliency-weighted capacities (Xu et al., 1 Aug 2025, Landsmeer et al., 5 Dec 2025).

4. Integration with Training Paradigms and Gradient Propagation

Selective spike dropping can complicate gradient calculation. In SDSNN, backpropagation employs a one-step surrogate gradient: $$h(u) = \frac{1}{a}\,\mathrm{sign}(|u - V_{\rm th}|<\tfrac{a}{2}), \quad a=1$$ applied only at the terminal sub-step, thus sidestepping backpropagation-through-time (BPTT) and reducing memory overhead from $O(\sum t_\ell)$ to $O(1)$ (Xu et al., 1 Aug 2025).

Queue-based lossy dropping in autodifferentiable simulation (e.g., JAX custom-JVPs) tracks primal and tangent paths. When a spike is dropped, neither membrane state nor gradient path is updated; the JVP remains static for that event, i.e. $x^+ = x^-$ and $$\partial x^+/\partial\theta = \partial x^-/\partial\theta$$. This ensures consistent forward and tangent computation, with potential for divergence in buffer data-structure choices (bitmask for primal, float buffer for tangent) in future frameworks (Landsmeer et al., 5 Dec 2025).

5. Quantitative Impact and Trade-offs

Selective spike dropping yields measurable reductions in memory, latency, and energy with negligible or modest loss of accuracy:

Dataset	Accuracy: SDSNN (SD neuron, 1 step)	Accuracy: LIF SNN (multi-step)	Energy Reduction
Fashion-MNIST	93.72%	93.51% (3 steps)	56%
CIFAR-10	92.20%	92.01% (2 steps)	21%
CIFAR-100	69.45%	70.42% (2 steps)	22%

Memory demands (ResNet-18 backbone, CIFAR-10) for SDSNN are 0.91 GB, compared to 3.00 GB for full BPTT (Xu et al., 1 Aug 2025). In event-queue simulations (10,000 neurons, H100 GPU), FIFORing($C=4$) achieves $\sim$1.7× speedup with $2\%$ drop-rate for $\lambda=0.01$, $d=20$, and negligible spike-train RMS error ($\sim$1% at $\alpha=2\%$) (Landsmeer et al., 5 Dec 2025). Only at $\lambda d \gg C$ does drop-induced error exceed $5\%$ RMS.

6. Hardware-Specific Design and Optimization

Selective spike dropping interacts closely with the characteristics of hardware backends. On CPUs, conventional heap-based or FIFO queues suffice. GPUs favor ring buffers for moderate queue depths, but small-capacity FIFORing or LossyRing outperform under memory constraint. TPUs leverage sort-based queues for tiny $C$, while dataflow architectures (e.g., Groq LPU) benefit from simple, fixed-capacity FIFORing/LossyRing implementations. These choices reflect cache behaviors, memory bandwidth, and branch prediction penalties (Landsmeer et al., 5 Dec 2025).

7. Future Directions and Prospects

Potential developments include diverging primal/tangent memory structures, enabling further reduction of simulation memory without loss of gradient exactness. Stochastic or learned importance dropping, where spikes are dropped or kept based on context (e.g., surrogate gradient magnitude), could selectively retain high-saliency spikes. Adaptive per-neuron $C$ settings, responsive to real-time spike statistics, promise improved memory-accuracy trade-offs. Hardware support for explicit delay lines may eventually render some software-level dropping unnecessary (Landsmeer et al., 5 Dec 2025). The SD neuron's framework is extensible to alternative dynamic-threshold or context-aware gating mechanisms, potentially amplifying signal efficiency.

Selective spike dropping consolidates the principled culling of non-salient or redundant spikes into both neuron dynamics and system simulation infrastructure, delivering a spectrum of efficiency gains validated at both the device and algorithmic scales (Xu et al., 1 Aug 2025, Landsmeer et al., 5 Dec 2025).