Spike Residual Basic Block (SRBB)

• SRBB is a residual module for spiking neural networks that adapts shortcut connections to maintain spike-based communication and temporal membrane dynamics.
• It integrates convolution, normalization, and spiking neuron operations in diverse formulations (SEW, PSP-normalized, spike-driven) to mitigate degradation and improve trainability.
• Empirical studies show SRBB enhances depth scalability, energy efficiency, and hardware compatibility across classification, transformer, and image restoration tasks.

Searching arXiv for the cited papers to ground the article in the research record. to=arxiv_search.search 微信的天天中彩票_json code {"query":"id:(Hu et al., 2021) OR id:(Zhou et al., 2023) OR id:(Fang et al., 2021) OR id:(Ikegawa et al., 2022) OR id:(Xu et al., 17 Aug 2025)","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} Spike Residual Basic Block (SRBB) denotes a class of residual modules for spiking neural networks (SNNs) in which the shortcut mechanism is redesigned to respect spike-based communication, temporal membrane dynamics, and the optimization constraints induced by binary activation. In the literature, SRBB is associated with several closely related formulations rather than a single universally fixed block. These formulations share a common objective: to preserve information flow and trainability in deep SNNs while retaining the event-driven computational model. Across directly trained deep SNNs, transformer-based SNNs, and spike-driven image restoration systems, SRBB-style designs are used to mitigate degradation, facilitate identity mapping, reduce non-spike computations, and improve depth scalability (Hu et al., 2021, Fang et al., 2021, Zhou et al., 2023).

1. Origins and problem setting

Residual learning is central to deep artificial neural networks, but its direct transplantation to SNNs is nontrivial because SNNs combine discrete binary activation with complex spatial-temporal dynamics. Deep residual learning in SNNs therefore confronts a distinct degradation problem: previous Spiking ResNet formulations “simply replace ReLU activation layers with spiking neurons,” which “suffers the degradation problem and can hardly implement residual learning” (Fang et al., 2021). A related diagnosis appears in later work identifying that prior residual designs neglected “spike-based communication and spatiotemporal dynamics,” leading to “impeded information flow and the accompanying degradation problem” (Hu et al., 2021).

This line of work established the practical stakes of residual-block design for SNNs. One study reported that a novel residual block could extend the depth of directly trained SNNs “up to 482 layers on CIFAR-10 and 104 layers on ImageNet, without observing any slight degradation problem,” with “76.02% accuracy on ImageNet” for SRM-ResNet104, described as “the first time in the domain of directly trained SNNs” (Hu et al., 2021). Earlier work on SEW ResNet stated that “directly training deep SNNs with more than 100 layers becomes possible” and that performance can improve “by simply adding more layers” (Fang et al., 2021). A complementary normalization-oriented study similarly reported that, “through the pre-activation residual blocks,” an SNN “can train with more than 100 layers without other special techniques dedicated to SNNs” (Ikegawa et al., 2022).

Taken together, these results place SRBB at the center of a broader research effort to make deep SNNs trainable without abandoning spike-driven computation.

2. Core architectural pattern

At a high level, SRBBs retain the two-path logic of residual networks: a learned residual branch and a shortcut branch. What changes is the placement of spiking neurons, the representation that traverses the shortcut, and the operator used to fuse the two paths.

A widely used formulation is the spike-element-wise residual block introduced in SEW ResNet. At time-step $t$, the block takes an input spike tensor $S^l[t] \in \{0,1\}^{C \times H \times W}$ and produces an output spike tensor $O^l[t] \in \{0,1\}^{C \times H' \times W'}$. Its residual path is

$$S^l[t] \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{SN} \rightarrow A^l[t],$$

while the shortcut path is either identity or a $1 \times 1$ strided $$\mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{SN}$$ when downsampling is required. The two paths are fused by a binary function $g$:

$O^l[t] = g(A^l[t], S^l[t]).$

The common choices reported are ADD, AND, and IAND (Fang et al., 2021).

A second major formulation, used in Spikingformer, makes the residual path explicitly spike-driven and hardware-oriented. For block $l$, the summary form is

$$O_l = \mathrm{ConvBN}_l(\mathrm{SN}_l(O_{l-1})) + O_{l-1}.$$

Here, $S^l[t] \in \{0,1\}^{C \times H \times W}$0 is a multi-step LIF neuron layer, $S^l[t] \in \{0,1\}^{C \times H \times W}$1 is convolution with fused BatchNorm, and the residual fusion is an element-wise add (Zhou et al., 2023). The paper emphasizes that the next block immediately re-spikes this floating output, so non-binary signals do not enter a subsequent weight-multiply stage.

A third variant appears in SNNSIR, where SRBB is described as a “lightweight Spike Residual Basic Block” for fully spike-driven stereo image restoration. There, the block contains two stacked $S^l[t] \in \{0,1\}^{C \times H \times W}$2 Spike Convolution Units and a “Membrane Shortcut,” with shortcut fusion performed in membrane space before output firing. In that formulation, “the input’s membrane potential” is added “to the main branch’s membrane potential before firing” (Xu et al., 17 Aug 2025).

A normalization-centered formulation appears in work on postsynaptic potential normalization. There, the SRBB is the “pre-activation residual block” in which the residual branch applies Spike, PSP filter, normalization, and convolution twice before adding the identity:

$S^l[t] \in \{0,1\}^{C \times H \times W}$3

This version treats the block input and output as membrane-potential or feature-map tensors rather than purely binary spikes (Ikegawa et al., 2022).

3. Spiking dynamics and mathematical formulations

The defining mathematical feature of SRBB is that the residual mechanism is embedded in discrete-time spiking neuron dynamics rather than in static activations. Across multiple formulations, the neuron model is given in terms of membrane potential update, thresholding, and reset. In SEW ResNet, the unified update is

$S^l[t] \in \{0,1\}^{C \times H \times W}$4

For IF neurons, $S^l[t] \in \{0,1\}^{C \times H \times W}$5, and for LIF neurons,

$S^l[t] \in \{0,1\}^{C \times H \times W}$6

Backpropagation uses a surrogate in place of the derivative of the Heaviside step (Fang et al., 2021).

Spikingformer uses an analogous multi-step LIF formulation. For each spatial position or channel $S^l[t] \in \{0,1\}^{C \times H \times W}$7,

$S^l[t] \in \{0,1\}^{C \times H \times W}$8

$S^l[t] \in \{0,1\}^{C \times H \times W}$9

with reset to $O^l[t] \in \{0,1\}^{C \times H' \times W'}$0 on spike and retention of $O^l[t] \in \{0,1\}^{C \times H' \times W'}$1 otherwise (Zhou et al., 2023). The block then computes $O^l[t] \in \{0,1\}^{C \times H' \times W'}$2 and applies shortcut addition.

In the SRM-ResNet context, the paper abstract emphasizes “spatiotemporal dynamics” and “one spike per neuron” on average for classification, but the data available here does not provide the full-text exact block equations. A reconstructed summary describes a two-convolution spiking block with membrane accumulation and reset, but since that description is explicitly reconstructed, it should be treated as provisional rather than canonical (Hu et al., 2021).

For the PSP-normalized pre-activation residual block, the key mathematical contribution is not only the residual form but also the normalization of the postsynaptic potential:

$O^l[t] \in \{0,1\}^{C \times H' \times W'}$3

The paper argues that this normalization controls spike firing by removing the subtraction term from standard normalization and dividing by the second raw moment rather than the variance (Ikegawa et al., 2022).

4. Identity mapping, degradation, and gradient behavior

A central theoretical role of SRBB is to realize identity mapping in a form compatible with spike dynamics. The SEW ResNet paper states that its block “can easily implement identity mapping and overcome the vanishing/exploding gradient problems of Spiking ResNet” (Fang et al., 2021). The formal argument compares a naïve Spiking ResNet block,

$O^l[t] \in \{0,1\}^{C \times H' \times W'}$4

with the SEW formulation

$O^l[t] \in \{0,1\}^{C \times H' \times W'}$5

Under identity mapping, the SEW block yields

$O^l[t] \in \{0,1\}^{C \times H' \times W'}$6

for the appropriate settings of $O^l[t] \in \{0,1\}^{C \times H' \times W'}$7, whereas the naïve construction can exhibit vanishing or exploding behavior depending on $O^l[t] \in \{0,1\}^{C \times H' \times W'}$8 (Fang et al., 2021).

The degradation perspective is framed differently in SRM-ResNet. There, the “crux” is the mismatch between ordinary residual design and the “characteristics of spike-based communication and spatiotemporal dynamics,” which impedes information flow (Hu et al., 2021). This diagnosis is consistent with later hardware-oriented work showing that even when residual connections improve optimization, they may still introduce undesirable computation patterns. Spikingformer argues that state-of-the-art deep SNNs, “including Spikformer and SEW ResNet,” suffer from “non-spike computations (integer-float multiplications) caused by the structure of their residual connection,” increasing power consumption and reducing suitability for “mainstream neuromorphic hardware, which only supports spike operations” (Zhou et al., 2023).

The normalization literature addresses a different failure mode: excessive firing. “Deep SNNs are not easy to train due to the excessive firing of spiking neurons in the hidden layers,” and pre-activation residual blocks become effective when combined with postsynaptic potential normalization that controls hidden-layer firing (Ikegawa et al., 2022). This indicates that SRBB is not only an architectural motif for preserving gradients, but also a site where firing-rate control and residual topology interact.

A plausible implication is that SRBB research has gradually shifted from a single question—whether residual learning works in SNNs—to a more differentiated agenda involving identity realizability, spike sparsity, membrane-state transport, and hardware-valid residual arithmetic.

5. Major variants in the literature

The term SRBB is used across several sublines of SNN research with distinct emphases.

Variant	Defining mechanism	Representative paper
SEW residual block	Spike-element-wise fusion $O^l[t] \in \{0,1\}^{C \times H' \times W'}$9 with ADD, AND, or IAND	(Fang et al., 2021)
SRM-ResNet residual block	Residual learning redesigned for spike-based communication and spatiotemporal dynamics	(Hu et al., 2021)
Pre-activation PSP-normalized residual block	Spike-PSP-Norm before each convolution with identity shortcut	(Ikegawa et al., 2022)
Spike-driven residual block	$$S^l[t] \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{SN} \rightarrow A^l[t],$$0 skip-add, re-binarized before next weighted stage	(Zhou et al., 2023)
Membrane-shortcut SRBB	Shortcut adds membrane potentials before output spike generation	(Xu et al., 17 Aug 2025)

The SEW variant is the most explicit in treating the residual fusion itself as the core design variable. Its defining operation is the element-wise binary function $$S^l[t] \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{SN} \rightarrow A^l[t],$$1, and its theoretical claim is exact identity mapping under suitable conditions (Fang et al., 2021).

The SRM-ResNet variant emphasizes depth scalability in directly trained SNNs. The reported depths—482 layers on CIFAR-10 and 104 layers on ImageNet—position residual-block redesign as the key enabler of very deep spiking architectures (Hu et al., 2021).

The PSP-normalized pre-activation variant ties residual topology to normalization strategy. Its results distinguish between post-activation and pre-activation residual layouts: the “post-activation block” failed to converge at 10% accuracy, whereas the “pre-activation block (SRBB)” achieved “up to 86.8% on deep ResNet-106” in the reported setting (Ikegawa et al., 2022).

The Spikingformer variant treats residual learning as a hardware interface problem. The main novelty is that $$S^l[t] \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{SN} \rightarrow A^l[t],$$2 receives binary spikes, the skip path uses floating add only, and the next block’s spiking neuron re-binarizes the signal so that “no integer-float multiplies ever occur” in the weighted stage (Zhou et al., 2023).

The SNNSIR variant extends SRBB beyond classification to stereo image restoration. Its “Membrane Shortcut” differs conceptually from spike-element-wise fusion: residual identity is preserved in membrane potential and only then converted to spikes, which the paper associates with improved expressiveness in a fully spike-driven pipeline (Xu et al., 17 Aug 2025).

6. Empirical outcomes, efficiency, and scope

Empirical results associate SRBB-style blocks with both depth scaling and energy-related benefits. In the SRM-ResNet study, the proposed residual block enabled directly trained SNNs of substantial depth and yielded “76.02% accuracy on ImageNet” for SRM-ResNet104. The same abstract reports that the resulting networks “need on average only one spike per neuron for classifying an input sample,” with “great energy efficiency” estimated (Hu et al., 2021).

SEW ResNet reports that it “outperforms the state-of-the-art directly trained SNNs in both accuracy and time-steps” on ImageNet, DVS Gesture, and CIFAR10-DVS, and that “higher performance” can be achieved by “simply adding more layers” (Fang et al., 2021). In the detailed summary provided here, SEW ResNet with $$S^l[t] \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{SN} \rightarrow A^l[t],$$3 is described as already outperforming some alternatives using more time-steps, while ADD is reported as the best-performing $$S^l[t] \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{Conv} \rightarrow \mathrm{BN} \rightarrow \mathrm{SN} \rightarrow A^l[t],$$4 choice on DVS Gesture (Fang et al., 2021).

The hardware-friendly spike-driven residual design in Spikingformer is associated with “75.85% top-1 accuracy on ImageNet” and “57.34%” reduced energy consumption “compared with Spikformer on ImageNet,” while the detailed summary reports 7.46 mJ per image for Spikingformer-8-512 and 18.82 mJ for Spikformer-8-512 (Zhou et al., 2023). The same work presents SRBB as the residual template used in both the Spiking Self-Attention and Spiking MLP sub-layers of the transformer block.

For PSP-normalized pre-activation residual blocks, the empirical emphasis is on trainability and firing-rate control. The study reports that SNNs with postsynaptic potential normalization “outperformed other models using other normalizations,” that PSP-LN slightly exceeded PSP-BN in the cited settings, and that deep pre-activation residual networks could surpass 100 layers without “other special techniques dedicated to SNNs” (Ikegawa et al., 2022).

In SNNSIR, SRBB is applied to stereo image restoration tasks including “rain streak removal, raindrop removal, low-light enhancement, and super-resolution.” The abstract reports “competitive restoration performance while significantly reducing computational overhead,” and the detailed summary attributes a PSNR improvement from 23.03 dB to 24.96 dB over a “plain SEW-RBB,” while also reporting large energy reductions relative to ANN baselines (Xu et al., 17 Aug 2025).

These results indicate that SRBB is not confined to a single benchmark regime. It functions as a reusable design principle across image classification, event-based recognition, transformer backbones, and low-power restoration pipelines.

7. Conceptual significance and open interpretations

The significance of SRBB lies in its reformulation of residual learning under spike constraints. In ordinary residual networks, the shortcut is primarily an optimization device. In SNNs, the shortcut simultaneously affects optimization, temporal state propagation, spike sparsity, and hardware realizability. The literature therefore treats SRBB as an interface between learning dynamics and execution model.

One recurring misconception is that a residual block for SNNs can be obtained by merely replacing ReLU with spiking neurons. Multiple papers reject this assumption directly: naïve Spiking ResNet “suffers the degradation problem,” and residual connections designed without regard to spike dynamics can impede information flow or trigger non-spike computations (Fang et al., 2021, Hu et al., 2021, Zhou et al., 2023).

A second misconception is that all residual additions in SNNs are equally compatible with neuromorphic deployment. Spikingformer explicitly argues otherwise, identifying “integer-float multiplications” induced by residual structure as a deployment-relevant flaw in otherwise strong SNN backbones (Zhou et al., 2023). SNNSIR reinforces this hardware criterion by contrasting a “fully spike-driven architecture” with hybrid SNN-ANN models that rely on floating-point division or exponentiation (Xu et al., 17 Aug 2025).

A third point of contention concerns what should remain on the shortcut path: binary spikes, membrane potentials, or real-valued feature maps. The surveyed variants instantiate all three possibilities. This suggests that SRBB is best understood not as a single canonical block, but as a design space for residual transport in spiking systems. A plausible interpretation is that future work will continue to differentiate this space along three axes already visible in current research: exact identity realizability, firing-rate regulation, and strict event-driven hardware compliance.