Extend BSD to Transformer architectures

Determine whether the bidirectional construction of Bidirectional Spike-Based Distillation can train larger model families such as Transformers, including the design of an attention-based reverse branch and suitable alignment points.

Background

Bidirectional Spike-Based Distillation is evaluated only on convolutional, recurrent, and gated-recurrent models. The paper identifies Transformer architectures as a substantially broader class whose attention mechanisms may require a dedicated target-driven reverse branch and a corresponding alignment strategy. Whether the BSD framework can be adapted to Transformers is therefore left unresolved.

References

Our evaluation covers convolutional, recurrent, and gated-recurrent models, and it remains to be explored whether the same bidirectional construction can train larger families such as Transformers, where the attention-based reverse branch and its alignment points require a dedicated design.

— Learning While Inferring: Local and Parallel Learning for Edge SNNs across Sensing Modalities  (2610.03149 - Zhang et al., 2 Oct 2026) in Appendix, Section “Limitations and Future Directions” (Appendix~\ref{app:limitations})