Bidirectional Spike-based Distillation (BSD)

• The paper introduces a novel spike-driven distillation method that jointly trains feedforward and backward spiking networks to achieve performance comparable to classical backpropagation.
• BSD leverages a bidirectional spiking architecture with asymmetric weights and localized error computation, satisfying five key biological constraints for neuromorphic learning.
• Empirical evaluations across image recognition, sequential regression, and image generation tasks show superior performance over alternative spiking approaches on standard benchmarks.

Searching arXiv for the named BSD paper and closely related distillation work to ground the article in current arXiv records. Bidirectional Spike-based Distillation (BSD) is a learning algorithm for biologically plausible deep learning in which a feedforward spiking network and a backward spiking network are trained jointly, using only discrete, unsigned spikes for communication in both inference and learning. BSD formulates learning as a transformation between two spiking representations—stimulus encoding and concept encoding—so that the feedforward pathway implements perception and decision-making by mapping stimuli to actions, while the backward pathway supports memory recall by reconstructing stimuli from concept representations. In the formulation reported for image recognition, image generation, and sequential regression, BSD is positioned as a spike-driven alternative to classical error backpropagation that preserves asymmetric weights, local learning, and non-dual-phase operation while achieving performance comparable to networks trained with backpropagation (Lv et al., 24 Sep 2025).

1. Motivation and defining criteria

BSD addresses a longstanding difficulty in biologically plausible learning: most alternatives to classical backpropagation either avoid using spikes for error propagation, rely on both positive and negative learning signals, or generalize poorly on complex tasks. A central issue identified in this line of work is that spikes are binary and do not directly represent negative values, which complicates the use of signed error transmission. BSD is proposed specifically to avoid that dependence on signed errors while retaining competitive task performance (Lv et al., 24 Sep 2025).

The method is organized around five biological constraints. These are: asymmetric synaptic weights for the forward and backward pathways; local error computation in which each weight update depends only on information locally available at the neuron; non-dual-phase learning without a strictly separated inference phase and error phase; a spiking neuron model based on discrete events; and unsigned error signals. In the reported formulation, BSD is described as the first biologically plausible learning algorithm to satisfy all five criteria simultaneously. This positioning is significant because the comparison class includes approaches such as Predictive Coding, DLL, CCL, and R-STDP, which are treated as biologically motivated but less complete with respect to this specific combination of constraints (Lv et al., 24 Sep 2025).

A useful way to interpret the contribution is that BSD does not attempt to make backpropagation more biologically acceptable by approximation alone. Instead, it reformulates learning itself as alignment between two spike-based pathways. This suggests a shift from global error transport to layer-wise representational consistency.

2. Dual-path spiking architecture and biological interpretation

BSD consists of two bidirectionally connected spiking neural networks. The feedforward path is associated with Type 1 neurons and is analogous to perception and decision-making: it transforms sensory input spikes into concept spikes. The feedback path is associated with Type 2 neurons and is analogous to memory recall: it reconstructs the original stimulus from concept-level spike encodings. The two pathways are trained together rather than sequentially, and both are implemented with populations of leaky integrate-and-fire (LIF) neurons (Lv et al., 24 Sep 2025).

The architecture is framed as an alignment between two kinds of spike trains. Stimulus encoding denotes spikes representing the sensory environment; concept encoding denotes spikes representing learned abstract structure such as labels, categories, or features. By construction, the forward network maps stimulus to concept, whereas the backward network maps concept back to stimulus. The authors explicitly relate this organization to bottom-up recognition and top-down imagination or recall. In that sense, BSD treats category formation and reconstruction as coupled computational primitives rather than as separate tasks (Lv et al., 24 Sep 2025).

Biological plausibility is further reinforced by the neuronal model. Each BSD neuron is described as a three-compartment pyramidal unit with basal dendrite, apical dendrite, and soma, which segregates incoming sensory and learning signals. The use of compartmental separation is not merely descriptive: it provides the substrate for comparing feedforward and feedback signals locally at each layer. A plausible implication is that BSD uses structural motifs associated with cortical circuitry to justify how local credit assignment can be performed without weight symmetry or global error vectors.

3. Formal learning rule and local objective

Let $L$ denote the number of layers. For the feedforward path, the layerwise dynamics are given as

$$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$

where $\mathbf{W}_{i-1}$ is the learned feedforward weight matrix and $\mathcal{SN}$ is the spike generator. For the backward path, the corresponding dynamics are

$$\mathbf{v}'_L = \hat{\mathbf{s}}, \qquad \mathbf{v}'_i = \boldsymbol{\Theta}_i \mathbf{s}'_{i+1}, \qquad \mathbf{s}'_i = \mathcal{SN}(\mathbf{v}'_i), \quad i=L-1,\dots,1,$$

where $\boldsymbol{\Theta}_i$ is the backward weight matrix and $\hat{\mathbf{s}}$ encodes the target (Lv et al., 24 Sep 2025).

The core local objective is a relaxed contrastive loss defined on the alignment between basal and apical compartment voltages. For a batch of $B$ samples, BSD first forms a cosine affinity matrix,

$$[\mathbf{C}_i]_{kj} = \frac{\mathbf{v}_{i,k} \cdot \hat{\mathbf{v}}_{i,j}} {\|\mathbf{v}_{i,k}\|\,\|\hat{\mathbf{v}}_{i,j}\|},$$

and then minimizes, at each layer $i$,

$$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$0

The first term aligns matched pairs; the second suppresses spurious overlap between non-matching samples but does not penalize negative correlation. The latter property is explicitly tied to the unsigned-error requirement. The total objective sums these layer-wise losses and includes a cross-entropy loss at the top layer (Lv et al., 24 Sep 2025).

The update rule is local. For example, the feedforward gradient at layer $$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$1 is written as

$$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$2

No gradient flow is allowed between compartments or across layers, and computational graphs are detached to preserve strict locality. Training proceeds in a single round consisting of a forward pass, a backward reconstruction pass, layer-wise comparison of basal and apical voltages, and local updates to both $$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$3 and $$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$4. The learning process is described as joint and asynchronous, without a mandatory separation between forward and backward phases (Lv et al., 24 Sep 2025).

4. Empirical scope, performance, and ablations

BSD is evaluated on image classification, sequential regression and natural language tasks, and image generation. For image classification, the reported datasets are MNIST, FashionMNIST, SVHN, CIFAR-10, and CIFAR-100. For sequential regression and natural language settings, the reported tasks include text character prediction on the Harry Potter corpus and time-series regression on Electricity, METR-LA, and PEMS-BAY. For image generation, the reported autoencoding benchmarks are MNIST, FashionMNIST, and CIFAR-10 (Lv et al., 24 Sep 2025).

Across these settings, BSD is reported to achieve performance comparable to backpropagation-trained artificial neural networks. On image classification, it is said to match BP-trained ANNs and to outperform Predictive Coding, DLL, CCL, and R-STDP by large margins, especially on more difficult datasets such as SVHN and CIFAR-10/100. On sequential regression and character prediction, BSD-trained recurrent models are reported to perform close to BP-trained ANNs while outperforming or matching Predictive Coding and DLL. On image generation, BSD autoencoders are reported as competitive in structure and FID score relative to BP-trained ANNs, and as substantially outperforming fully spiking variational autoencoders in FID (Lv et al., 24 Sep 2025).

The ablation results clarify the role of the local objective. Using mean squared error for voltage alignment is reported to cause poor convergence. Using InfoNCE is better, but still consistently worse than the relaxed contrastive loss used in BSD. Additional observations concern dynamics rather than final metrics: alignment of spikes between the forward and backward networks rises quickly and remains high during training, while cosine similarity between forward and backward weights remains low. The latter point is important because it indicates that good performance is not obtained by drifting toward weight symmetry, which would contradict one of BSD’s defining biological criteria (Lv et al., 24 Sep 2025).

5. Relation to other distillation paradigms

The term “BSD” can be misleading because related literatures use similar vocabulary for substantively different mechanisms. Earlier work on the spike distillation network in reinforcement learning transfers knowledge from a DNN teacher to an SNN student with STBP, uses an experience pool, and explicitly notes that bidirectional distillation is not discussed; only unidirectional distillation is considered (Zhang et al., 2021). In dialogue generation, bidirectional distillation refers to peer models that regularize one another through fused predictions and a Jensen–Shannon objective across subtasks (Feng et al., 2021). In semantic segmentation, “BSD” denotes Bidirectional Selective Distillation, where region-wise and pixel-wise knowledge transfer directions are selected dynamically between CNN and ViT students according to reliability (Zhu et al., 2023). In spiking language modeling, Spike-Aware Alignment Distillation aligns embeddings, attention maps, features, and logits from a frozen ANN teacher to an SNN student, but there is no reverse distillation from student to teacher (Guo et al., 14 Apr 2026).

Method	Distillation pattern	Domain
SDN with STBP (Zhang et al., 2021)	DNN $$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$5 SNN, unidirectional	Reinforcement learning
MRBD (Feng et al., 2021)	Student $$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$6 peers via JS divergence	Dialogue generation
Bidirectional Selective Distillation (Zhu et al., 2023)	Dynamic region-wise and pixel-wise transfer	Semantic segmentation
BSD (Lv et al., 24 Sep 2025)	Forward/backward spiking pathways trained jointly	Biologically plausible learning
SpAD (Guo et al., 14 Apr 2026)	ANN $$\mathbf{v}_1 = \mathbf{x}, \qquad \mathbf{v}_i = \mathbf{W}_{i-1}\mathbf{s}_{i-1}, \qquad \mathbf{s}_i = \mathcal{SN}(\mathbf{v}_i), \quad i=2,\dots,L,$$7 SNN, no reverse distillation	Spiking causal language modeling

Against this background, Bidirectional Spike-based Distillation is distinct in two respects. First, its bidirectionality is architectural and representational: the feedforward and backward pathways are both spiking networks, and learning aligns stimulus and concept encodings in both directions. Second, its local objective is designed around unsigned spike-based signals rather than around mutual KL/JS supervision between continuous-output peers. This suggests that BSD belongs as much to the literature on biologically grounded credit assignment as to the literature on knowledge distillation narrowly construed.

6. Limitations, misconceptions, and open directions

A common misconception is to treat BSD as simply another instance of mutual knowledge distillation. The available description does not support that simplification. BSD is presented as a local-learning algorithm in which forward and backward pathways support perception, decision-making, and memory recall by aligning internal spiking representations. Its novelty is therefore bound up with biological plausibility constraints—especially asymmetric weights, unsigned signals, and non-dual-phase learning—rather than only with the existence of two directions of information transfer (Lv et al., 24 Sep 2025).

The reported limitations are concrete. Architectural generality remains limited: the demonstrated models are standard MLPs, CNNs, RNNs, and autoencoders, and extension to residual, attention-based, or more complex architectures is not yet shown. In image generation, reconstructed images retain global structure but may lose some fine-grained texture compared with classical ANNs. Scalability and efficiency are also identified as open issues for very large-scale tasks or real-time hardware deployment. The proposed future directions are accordingly to broaden the architectural scope, improve fine-detail generation, and explore integration with more biologically detailed neural models (Lv et al., 24 Sep 2025).

Taken together, these caveats delimit the present scope of BSD without undercutting its conceptual position. Within the reported experiments, BSD is advanced as a unifying framework in which perception, action, and recall are all mediated by local alignment between spike-based representations. A plausible implication is that subsequent work will test whether this formulation can be retained when moving from relatively standard architectures to more contemporary residual or attention-dominated systems.