Papers
Topics
Authors
Recent
Search
2000 character limit reached

Scalable Superconductor Neuron with Ternary Synaptic Connections for Ultra-Fast SNN Hardware

Published 26 Feb 2024 in cond-mat.supr-con, cs.ET, and cs.NE | (2402.16384v2)

Abstract: A novel high-fan-in differential superconductor neuron structure designed for ultra-high-performance Spiking Neural Network (SNN) accelerators is presented. Utilizing a high-fan-in neuron structure allows us to design SNN accelerators with more synaptic connections, enhancing the overall network capabilities. The proposed neuron design is based on superconductor electronics fabric, incorporating multiple superconducting loops, each with two Josephson Junctions. This arrangement enables each input data branch to have positive and negative inductive coupling, supporting excitatory and inhibitory synaptic data. Compatibility with synaptic devices and thresholding operation is achieved using a single flux quantum (SFQ) pulse-based logic style. The neuron design, along with ternary synaptic connections, forms the foundation for a superconductor-based SNN inference. To demonstrate the capabilities of our design, we train the SNN using snnTorch, augmenting the PyTorch framework. After pruning, the demonstrated SNN inference achieves an impressive 96.1% accuracy on MNIST images. Notably, the network exhibits a remarkable throughput of 8.92 GHz while consuming only 1.5 nJ per inference, including the energy consumption associated with cooling to 4K. These results underscore the potential of superconductor electronics in developing high-performance and ultra-energy-efficient neural network accelerator architectures.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. D. S. Holmes, “Keys needs in the technology roadmap for superconductor electronics,” IEEE-CSC & ESAS SUPERCONDUCTIVITY NEWS FORUM, March 2023, presentation 1EOr2B-01 given at ASC, Honolulu, HI, USA, October 24, 2022.
  2. Razmkhah, Sasan and Febvre, Pascal, “Superconducting Quantum Electronics,” in Beyond-CMOS, 2023, ch. 8, pp. 295–391.
  3. K. Likharev and V. Semenov, “RSFQ logic/memory family: a new Josephson-junction technology for sub-terahertz-clock-frequency digital systems,” IEEE TAS, vol. 1, March 1991.
  4. T. Hirose et al., “Spiking neuron devices consisting of single-flux-quantum circuits,” Physica C: Superconductivity and its Applications, vol. 445-448, 2006, proc. of the 18th ISS.
  5. P. Crotty et al., “Josephson junction simulation of neurons,” Phys. Rev. E, vol. 82, Jul 2010.
  6. M. A. Jardine and C. J. Fourie, “Hybrid RSFQ-QFP Superconducting Neuron,” IEEE TAS, vol. 33, no. 4, 2023.
  7. M. A. Karamuftuoglu and M. Pedram, “αα\upalpharoman_α-Soma: Single Flux Quantum Threshold Cell for Spiking Neural Network Implementations,” IEEE TAS, vol. 33, no. 5, 2023.
  8. M. A. Karamuftuoglu et al., “JJ-Soma: Towards a Spiking Neuromorphic Processor Architecture,” IEEE TAS, 2023.
  9. A. J. Edwards, G. Krylov, J. S. Friedman, and E. G. Friedman, “Harnessing stochasticity for superconductive multi-layer spike-rate-coded neuromorphic networks,” Neuromorphic Computing and Engineering, vol. 4, no. 1, p. 014005, Feb 2024.
  10. M. L. Schneider and K. Segall, “Fan-out and fan-in properties of superconducting neuromorphic circuits,” Jnl. of Applied Physics, vol. 128, no. 21, 12 2020.
  11. J. K. Eshraghian et al., “Training spiking neural networks using lessons from deep learning,” arXiv preprint arXiv:2109.12894, 2021.
  12. Y. Lecun et al., “Gradient-based learning applied to document recognition,” Proc. of the IEEE, vol. 86, no. 11, 1998.
  13. X. Chen et al., “CMOS-based area-and-power-efficient neuron and synapse circuits for time-domain analog spiking neural networks,” Applied Physics Letters, vol. 122, no. 7, 02 2023.
  14. S. A. Aamir et al., “A Mixed-Signal Structured AdEx Neuron for Accelerated Neuromorphic Cores,” IEEE Transactions on Biomedical Circuits and Systems, vol. 12, no. 5, 2018.
  15. R. Kleiner et al., “Superconducting quantum interference devices: State of the art and applications,” Proc. of the IEEE, vol. 92, no. 10, Oct 2004.
  16. E. Toomey et al., “Design of a Power Efficient Artificial Neuron Using Superconducting Nanowires,” Frontiers in Neuroscience, 2019.
  17. H. Gao et al., “High-accuracy deep ANN-to-SNN conversion using quantization-aware training framework and calcium-gated bipolar leaky integrate and fire neuron,” Frontiers in Neuroscience, March 2023.
  18. S. Kim et al., “Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, Apr. 2020.
  19. M. L. Schneider et al., “Energy-efficient single-flux-quantum based neuromorphic computing,” in IEEE Int. Conf. on Rebooting Computing (ICRC), 2017.
  20. A. Bozbey et al., “Single flux quantum based ultrahigh speed spiking neuromorphic processor architecture,” arXiv.org, Jul 2020. [Online]. Available: https://arxiv.org/abs/1812.10354
  21. K. Ishida et al., “Superconductor Computing for Neural Networks,” IEEE Micro, vol. 41, no. 3, 2021.
  22. R. Cai et al., “A stochastic-computing based deep learning framework using adiabatic quantum-flux-parametron superconducting technology,” in Proc. of the 46th Int. Symposium on Computer Architecture, 2019.
  23. H. Zhang et al., “Brain-inspired spiking neural network using superconducting devices,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 7, 2023.
  24. D. E. Kirichenko et al., “Zero Static Power Dissipation Biasing of RSFQ Circuits,” IEEE TAS, vol. 21, no. 3, 2011.
  25. M. Schneider et al., “Synaptic weighting in single flux quantum neuromorphic computing,” Scientific Reports, vol. 10, no. 1, 2020.
  26. S. Razmkhah, M. A. Karamuftuoglu, and A. Bozbey, “Hybrid synaptic structure for spiking neural network realization,” arXiv preprint arXiv:2311.07787, 2023.
  27. C. J. Fourie et al., “Results from the ColdFlux Superconductor Integrated Circuit Design Tool Project,” IEEE Transactions on Applied Superconductivity, pp. 1–27, 2023.
  28. B. Z. Ucpinar et al., “Design of a Superconducting Multiflux Non-Destructive Readout Memory Unit,” arXiv:2309.14613, 2023.
  29. D. Heeger et al., “Poisson model of spike generation,” Handout, University of Standford, vol. 5, no. 1-13, 2000.
  30. S. K. Tolpygo et al., “Fabrication Process and Properties of Fully-Planarized Deep-Submicron Nb/Al–AlOx⁢/NbsubscriptAlOx/Nb\hbox{AlO}_{\rm x}\hbox{/Nb}AlO start_POSTSUBSCRIPT roman_x end_POSTSUBSCRIPT /Nb Josephson Junctions for VLSI Circuits,” IEEE TAS, vol. 25, no. 3, pp. 1–12, June 2015.
  31. J. A. Delport et al., “JoSIM—Superconductor SPICE Simulator,” IEEE TAS, vol. 29, no. 5, 2019.
  32. S. Bang et al., “An Energy-Efficient SNN Processor Design based on Sparse Direct Feedback and Spike Prediction,” in 2021 Int. Joint Conf. on Neural Networks (IJCNN), 2021.
  33. M. S. Asghar et al., “A compact and low-power soc design for spiking neural network based on current multiplier charge injector synapse,” MDPI, vol. 23, 2023.
Citations (4)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up