Spiking Inception Module in SNNs

• Spiking Inception Module is an unsupervised spiking neural network architecture that leverages split-and-merge topology with multi-scale, parallel feature learning.
• It integrates independent excitatory pathways with diverse receptive fields and local lateral inhibition to enable robust, winner-take-all competition and Vote-for-All decoding.
• The design incorporates a stackable Pooling-Reshape-Activate layer that transforms sparse spike outputs into rich, trainable representations, boosting learning efficiency and accuracy.

Searching arXiv for the specified papers and closely related work. A Spiking Inception Module is an Inception-style building block for spiking neural networks trained by STDP-based competitive learning. In the 2019 formulation, it denotes a single-layer but high-parallelism, multi-pathway architecture that combines global and local receptive fields, winner-take-all competition, Vote-for-All decoding, and an adaptive repolarization mechanism (Meng et al., 2019). In the 2020 formulation, the concept is generalized into a stackable “Sp-Inception” module whose branch outputs are pooled, reshaped, and re-activated by a Pooling-Reshape-Activate layer so that multiple such modules can form a deep unsupervised SNN (Meng et al., 2020). Across both formulations, the defining idea is multi-scale parallel feature learning in the spike domain, without backpropagation, using local plasticity and inhibitory competition.

1. Conceptual lineage and defining properties

The module preserves three core ideas from classical ANN Inception architectures: split-and-merge topology, multi-scale processing, and parallelism. A single input is split into several branches with different receptive field sizes, and the resulting features are merged downstream. In the single-layer version, the merge occurs through a Vote-for-All decoding layer; in the stackable version, branch outputs are flattened, concatenated, and eventually decoded by VFA during evaluation (Meng et al., 2019).

The analogy to ANN Inception is limited in precise and important ways. The branches are not conventional convolutional paths with shared kernels, and the 2019 design explicitly has no explicit convolutions, no $1 \times 1$ reductions, and no pooling layers. Its locally connected branches use convolution-like receptive fields but without weight sharing, and its computation is driven by Poisson spike trains, spike-triggered STDP, and lateral inhibition rather than continuous activations and supervised gradients. The 2020 stackable formulation retains the split-and-merge logic and multi-scale receptive fields, but again operates with LIF neurons, STDP, and competition rather than backpropagation (Meng et al., 2020).

A recurrent misconception is to treat the module as a direct spiking transcription of GoogLeNet. The papers instead present it as an architectural borrowing of the Inception principle—parallel multi-scale paths—adapted to event-driven, competitive, unsupervised SNNs. In that sense, “Inception” identifies the topology and scale diversity rather than a full transfer of ANN implementation details.

2. Architectural organization

In the 2019 paper, the module is a complete high-parallelism output architecture built from three independent excitatory pathways, each connected directly to a $28 \times 28$ input layer of Poisson spike generators. The first pathway is fully connected, with topology $(28,1)$ and feature map number $F = 4 \times \text{Size}_{SA}$. The second is locally connected with $(k,s) = (24,4)$ and $F = 2 \times \text{Size}_{SA}$. The third is locally connected with $(k,s) = (16,6)$ and $F = \text{Size}_{SA}$. All output neurons are LIF neurons receiving spikes from their receptive fields through excitatory STDP synapses, and neurons within a competition area are linked by fixed inhibitory synapses. The three pathways do not inhibit or excite one another; they learn in parallel and send their spikes to a VFA decoding layer (Meng et al., 2019).

Competition is organized through receptive-field-defined competition areas. A competition area is the set of output neurons that share the same receptive field and inhibit one another. This makes each competition area an independent sub-network with local winner-take-all dynamics. The paper’s principal design move is to increase parallelism by subdividing these areas: the FC pathway is split into 4 sub-areas, the $(24,4)$ pathway yields 4 competition areas that are each split into 2 sub-areas for a total of 8, and the $(16,6)$ pathway yields 9 competition areas that are not split further. The final architecture therefore contains $28 \times 28$0 independent competition sub-networks (Meng et al., 2019).

The 2020 paper reframes the same family of ideas as a reusable module. A Sp-Inception module sits between an input spike layer and a spiking decoding layer, and consists of several parallel LC or FC branches. Each branch is defined by either a fully connected topology with neuron count $28 \times 28$1, or a locally connected topology $28 \times 28$2. Branch outputs are reshaped, flattened, and concatenated into a single one-dimensional output representation. The paper distinguishes between a naive Sp-Inception, in which branch sizes are chosen empirically, and a balanced Sp-Inception, in which large branches are divided into multiple smaller branches and branch feature-map counts are equalized to avoid a single slow branch bottlenecking training (Meng et al., 2020).

Configuration	Topology	Reported test accuracy
Sp-Inception I	FC, $28 \times 28$3; LC, $28 \times 28$4; LC, $28 \times 28$5	93.36%
Sp-Inception III	$28 \times 28$6; $28 \times 28$7; LC, $28 \times 28$8	94.86%
Sp-Inception VI	FC, $28 \times 28$9; LC, $(28,1)$0; LC, $(28,1)$1; LC, $(28,1)$2	95.85%

These designs make the module simultaneously multi-scale and highly parallel. The FC path captures global structure, while the LC paths capture almost-global or local structure at several scales. This suggests that the module’s central representational claim is not depth alone, but width combined with heterogeneous receptive fields.

3. Neuron models, plasticity, and competitive learning

The 2019 implementation uses conductance-based LIF output neurons with separate excitatory and inhibitory conductances, refractory behavior, and a homeostatic adaptive threshold. The threshold variable $(28,1)$3 increases by $(28,1)$4 whenever the neuron spikes and otherwise decays toward $(28,1)$5, preventing a few units from dominating the learning dynamics. Excitatory synapses are trained by a simplified triplet STDP rule in which potentiation occurs on postsynaptic spikes and depression on presynaptic spikes, with $(28,1)$6; the paper gives $(28,1)$7 and $(28,1)$8 as an example (Meng et al., 2019).

Within each competition area or sub-area, all neurons share the same presynaptic set and mutually inhibit one another via fixed inhibitory synapses. The winners spike strongly, and those spikes drive local STDP updates that move receptive fields toward the current input pattern. Because different competition areas are disjoint, each area effectively learns a distinct feature dictionary. Splitting competition areas into equal-size sub-areas increases both the number of features learned and the degree of parallelism (Meng et al., 2019).

The 2020 stackable version uses a current-based LIF neuron model. Voltage dynamics follow

$(28,1)$9

and the synaptic current obeys

$F = 4 \times \text{Size}_{SA}$0

with spike generation at threshold, reset to $F = 4 \times \text{Size}_{SA}$1, and a refractory period $F = 4 \times \text{Size}_{SA}$2. Its adaptive threshold again enforces homeostasis, and its STDP rule is additive rather than triplet-based:

$F = 4 \times \text{Size}_{SA}$3

The paper reports $F = 4 \times \text{Size}_{SA}$4, $F = 4 \times \text{Size}_{SA}$5, and $F = 4 \times \text{Size}_{SA}$6 (Meng et al., 2020).

The change in neuron and synapse models between the two papers is notable. This suggests that the Spiking Inception Module is primarily an architectural principle—parallel branches, multi-scale receptive fields, local competition, and unsupervised learning—rather than a construction tied to one unique spiking dynamical system.

4. Decoding and information aggregation

A major contribution of the 2019 work is the replacement of standard Vote-for-One decoding with Vote-for-All decoding. In VFO, each output neuron is assigned to the single class for which it fires most on labeled training data, and each later spike is treated as a vote only for that class. The paper argues that this assumption is approximately valid for FC architectures with global receptive fields, but becomes sub-optimal for LC and Inception-like architectures because local receptive fields often respond to patterns shared across multiple classes. If a neuron responds strongly to both “2” and “3” in a local patch, assigning it exclusively to one class discards evidence for the other (Meng et al., 2019).

The VFA layer therefore assigns one neuron per class and connects every excitatory output neuron to every class unit. After unsupervised training, the weight from output neuron $F = 4 \times \text{Size}_{SA}$7 to class neuron $F = 4 \times \text{Size}_{SA}$8 is set to

$F = 4 \times \text{Size}_{SA}$9

where $(k,s) = (24,4)$0 is the average spike count of neuron $(k,s) = (24,4)$1 on class $(k,s) = (24,4)$2, and $(k,s) = (24,4)$3. During inference, VFA neurons do not spike; they integrate incoming EPSCs, and the predicted label is the class with maximum membrane potential. This makes each output neuron vote for all classes in proportion to its empirical class-conditional activity, reducing information loss and increasing redundancy under damage (Meng et al., 2019).

The empirical role of VFA is sharply demonstrated by ablation. In the 2019 paper, the Inception architecture with VFO decoding (“Ours-noVFA”) reaches 94.88% on MNIST and 69.27% on EMNIST, versus 95.64% and 80.11% for the full method. The resulting drop is especially large on EMNIST, which is consistent with the paper’s argument that local and multi-scale branches violate the “one neuron–one class” assumption more severely than FC networks do (Meng et al., 2019).

The 2020 paper retains VFA for evaluation after branch outputs are concatenated. In that setting, the decoder serves as the terminal classifier for both single-module and multi-module networks, preserving the original logic that spike statistics rather than gradient-trained classifier weights should determine the class readout (Meng et al., 2020).

5. Stackability and the Pooling-Reshape-Activate layer

The 2020 paper’s distinctive advance is the Pooling-Reshape-Activate (PRA) layer, introduced to make Sp-Inception modules stackable. The motivation is twofold: first, branch concatenation produces high-dimensional outputs; second, raw output spike rates from a competitive STDP layer are too sparse to effectively drive another STDP layer. PRA addresses both issues by acting as the input layer to the next module (Meng et al., 2020).

Its pooling stage uses LIF neurons that collect spikes from local regions of the previous module’s output, with pooling connections sharing a common weight $(k,s) = (24,4)$4. This reduces dimensionality and increases local spiking by summating multiple weakly active inputs. The reshape stage maps pooled activations into a $(k,s) = (24,4)$5 tensor, preserving relative topological structure for the next module’s LC and FC branches. The activate stage adaptively adjusts either the initial Poisson encoding gain $(k,s) = (24,4)$6 for the first module or the pooling weight $(k,s) = (24,4)$7 for later modules. If output spiking intensity is too low, the gain is increased and the image is re-presented until sufficient spikes are produced (Meng et al., 2020).

The reported spike intensities illustrate the problem PRA solves. For a training iteration, the typical input spike intensity is approximately 3000 spikes per iteration. Output spike intensity is reported as 7.84 per iteration for Baseline-FC, 58.26 for Baseline-LC, 142.83 for Sp-Inception, and 2294.87 for Sp-Inception + PRA. The paper presents this increase as the key enabler for deep, purely unsupervised stacking (Meng et al., 2020).

With PRA, the network can be organized as

$(k,s) = (24,4)$8

and trained module-by-module in a forward direction, with each module’s synapses updated locally by STDP and no backpropagation across modules. The authors describe this as layer-wise unsupervised training and argue that it allows higher modules to learn increasingly abstract spike patterns (Meng et al., 2020).

Stacked architecture	Total neurons / synapses	Reported result
1st module only	778K / 778K	93.36%
+2nd module	3894K / 3894K	95.17%
+3rd module	11533K / 11533K	96.03%
+4th module	23818K / 23818K	96.48%

6. Empirical properties, limitations, and research significance

The 2019 paper reports 95.64% test accuracy on MNIST and 80.11% on EMNIST letters for its best configuration with $(k,s) = (24,4)$9. Against baselines, it reports 94.97% MNIST and 47.41% EMNIST for Diehl-FC, and 95.02% MNIST and 69.68% EMNIST for Saunders-LC. Ablations show that replacing the Inception-like architecture with FC yields 95.06% MNIST and 56.16% EMNIST, replacing it with pure LC yields 95.26% and 77.85%, removing VFA yields 94.88% and 69.27%, and removing adaptive reset yields 95.67% and 80.21%. The paper therefore attributes final accuracy primarily to the architecture and VFA, while presenting adaptive reset mainly as a learning-speed mechanism (Meng et al., 2019).

Learning efficiency is a central empirical claim. Under the 2019 training protocol, each image is presented for 350 ms followed by 150 ms of rest, and one pass through the training set corresponds to 60k iterations for MNIST and 124.8k for EMNIST. The paper contrasts its method with a Diehl-FC configuration that needs about 900,000 iterations to fully train and attains only about 42% accuracy at 10,000 iterations. By comparison, the Inception-like model with adaptive reset reaches nearly 80% accuracy at 500 iterations and nearly 90% at 2500 iterations, with one pass sufficient for full training (Meng et al., 2019).

Robustness is measured by random deletion of neurons or learnable synapses after training. For networks with similar original accuracy near 94.9%, the Inception-like model maintains about 80% accuracy even at $F = 2 \times \text{Size}_{SA}$0 neuron deletion, whereas at $F = 2 \times \text{Size}_{SA}$1 Diehl-FC is approximately 80% and Saunders-LC approximately 85%. Under synapse deletion, the Inception-like model maintains about 90% accuracy at $F = 2 \times \text{Size}_{SA}$2 and about 80% at $F = 2 \times \text{Size}_{SA}$3, while Diehl-FC and Saunders-LC fall to approximately 70% and 65% at $F = 2 \times \text{Size}_{SA}$4. The paper attributes this to the coexistence of many independent competition sub-networks, multi-scale redundancy, and VFA’s all-to-all voting structure (Meng et al., 2019).

The 2020 paper extends these claims from a single module to a multi-module architecture on MNIST. Single-module Sp-Inception VI reaches 95.85%, exceeding Baseline-FC IV at 94.88% and Baseline-LC IV at 95.02% under the reported synapse budgets. A four-module network reaches 96.48%. The paper compares this result with unsupervised SNNs including Diehl et al. at 95.0%, Saunders et al. at 95.07%, Panda et al. (ASP) at 96.80%, She et al. (stochastic STDP) at 96.10%, Meng et al. at 95.64%, and Lammie et al. at 94.0%, and characterizes the four-module result as among the best purely unsupervised STDP-based SNNs, while noting that ASP and stochastic STDP achieve slightly higher accuracy with more sophisticated synaptic mechanisms (Meng et al., 2020).

The limitations are explicit. The multi-layer study uses only MNIST and not more complex datasets. Its components remain deliberately simple: LIF neurons, basic additive STDP, and rate-based Poisson encoding. The four-module network uses tens of millions of synapses, and only four modules are tested because of resource limits. The paper therefore frames future work around more challenging datasets, more sophisticated neuron or synapse models such as Adaptive Synaptic Plasticity, stochastic STDP, or triplet STDP, hybrid supervised fine-tuning, and neuromorphic hardware implementation (Meng et al., 2020).

Taken together, the two papers define the Spiking Inception Module as a family of unsupervised, STDP-trained, multi-branch spiking architectures centered on heterogeneous receptive fields, branchwise competition, and non-gradient aggregation. In the single-layer setting, its distinctive elements are high parallelism, VFA decoding, and adaptive reset; in the deep setting, the decisive addition is PRA, which converts a sparse competitive representation into a stackable intermediate signal. The result is a specific line of research in which architectural design, rather than more elaborate plasticity alone, is treated as the primary lever for improving learning capability, learning efficiency, and robustness in unsupervised SNNs.