Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Darwin3: A large-scale neuromorphic chip with a Novel ISA and On-Chip Learning (2312.17582v1)

Published 29 Dec 2023 in cs.NE and cs.AR

Abstract: Spiking Neural Networks (SNNs) are gaining increasing attention for their biological plausibility and potential for improved computational efficiency. To match the high spatial-temporal dynamics in SNNs, neuromorphic chips are highly desired to execute SNNs in hardware-based neuron and synapse circuits directly. This paper presents a large-scale neuromorphic chip named Darwin3 with a novel instruction set architecture(ISA), which comprises 10 primary instructions and a few extended instructions. It supports flexible neuron model programming and local learning rule designs. The Darwin3 chip architecture is designed in a mesh of computing nodes with an innovative routing algorithm. We used a compression mechanism to represent synaptic connections, significantly reducing memory usage. The Darwin3 chip supports up to 2.35 million neurons, making it the largest of its kind in neuron scale. The experimental results showed that code density was improved up to 28.3x in Darwin3, and neuron core fan-in and fan-out were improved up to 4096x and 3072x by connection compression compared to the physical memory depth. Our Darwin3 chip also provided memory saving between 6.8X and 200.8X when mapping convolutional spiking neural networks (CSNN) onto the chip, demonstrating state-of-the-art performance in accuracy and latency compared to other neuromorphic chips.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Brian 2, an intuitive and efficient neural simulator. Elife, 8:e47314, 2019.
  2. Nest: An environment for neural systems simulations. Forschung und wisschenschaftliches Rechnen, Beiträge zum Heinz-Billing-Preis, 58:43–70, 2001.
  3. Spaic: A spike-based artificial intelligence computing framework. IEEE Computational Intelligence, 2022.
  4. Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proceedings of the IEEE, 102(5):699–716, 2014.
  5. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38(1):82–99, 2018.
  6. Flexlearn: fast and highly efficient brain simulations using flexible on-chip learning. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, pages 304–318, 2019.
  7. Spinnaker: A 1-w 18-core system-on-chip for massively-parallel neural network simulation. IEEE Journal of Solid-State Circuits, 48(8):1943–1953, 2013.
  8. Efficient neuromorphic signal processing with loihi 2. In 2021 IEEE Workshop on Signal Processing Systems (SiPS), pages 254–259. IEEE, 2021.
  9. Mapping very large scale spiking neuron network to neuromorphic hardware. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, pages 419–432, 2023.
  10. Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE transactions on computer-aided design of integrated circuits and systems, 34(10):1537–1557, 2015.
  11. Unicorn: A multicore neuromorphic processor with flexible fan-in and unconstrained fan-out for neurons. In Proceedings of the 59th ACM/IEEE Design Automation Conference, pages 943–948, 2022.
  12. The brainscales-2 accelerated neuromorphic system with hybrid plasticity. Frontiers in Neuroscience, 16:795876, 2022.
  13. Open-loop analog programmable electrochemical memory array. Nature Communications, 14(1):6184, 2023.
  14. Darwin: A neuromorphic hardware co-processor based on spiking neural networks. Journal of systems architecture, 77:43–51, 2017.
  15. Anthony N Burkitt. A review of the integrate-and-fire neuron model: I. homogeneous synaptic input. Biological cybernetics, 95:1–19, 2006.
  16. Eugene M Izhikevich. Simple model of spiking neurons. IEEE Transactions on neural networks, 14(6):1569–1572, 2003.
  17. Spike timing–dependent plasticity: a hebbian learning rule. Annu. Rev. Neurosci., 31:25–46, 2008.
  18. Fourier analysis of sinusoidally driven thalamocortical relay neurons and a minimal integrate-and-fire-or-burst model. Journal of neurophysiology, 83(1):588–610, 2000.
  19. Bard Ermentrout. Type i membranes, phase resetting curves, and synchrony. Neural computation, 8(5):979–1001, 1996.
  20. Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. Journal of neurophysiology, 94(5):3637–3642, 2005.
  21. A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of physiology, 117(4):500, 1952.
  22. Alan L Hodgkin. The local electric changes associated with repetitive action in a non-medullated axon. The Journal of physiology, 107(2):165, 1948.
  23. The dynamic synapse. Neuron, 80(3):691–703, 2013.
  24. Modeling synapses. Computational modeling methods for neuroscientists, 6:139–160, 2009.
  25. Synaptic plasticity: multiple forms, functions, and mechanisms. Neuropsychopharmacology, 33(1):18–41, 2008.
  26. The hebb rule for synaptic plasticity: algorithms and implementations. In Neural models of plasticity, pages 94–103. Elsevier, 1989.
  27. Neuronal synapse as a memristor: Modeling pair-and triplet-based stdp rule. IEEE transactions on biomedical circuits and systems, 9(1):87–95, 2014.
  28. Bio-plausible digital implementation of a reward modulated stdp synapse. Neural Computing and Applications, 34(18):15649–15660, 2022.
  29. Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural computation, 19(11):2881–2912, 2007.
  30. Anp-i: A 28nm 1.5 pj/sop asynchronous spiking neural network processor enabling sub-o. 1 μ𝜇\muitalic_μj/sample on-chip learning for edge-ai applications. In 2023 IEEE International Solid-State Circuits Conference (ISSCC), pages 21–23. IEEE, 2023.
  31. Low latency network-on-chip router microarchitecture using request masking technique. International Journal of Reconfigurable Computing, 2015:2–2, 2015.
  32. Process variation delay and congestion aware routing algorithm for asynchronous noc design. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 24(3):909–919, 2015.
  33. A fair arbitration for network-on-chip routing with odd-even turn model. Microelectronics Journal, 64:1–8, 2017.
  34. A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (dynaps). IEEE transactions on biomedical circuits and systems, 12(1):106–122, 2017.
  35. The spinnaker 2 processing element architecture for hybrid digital neuromorphic computing. arXiv preprint arXiv:2103.08392, 2021.
  36. A 65nm 236.5 nj/classification neuromorphic processor with 7.5% energy overhead on-chip learning using direct spike-only feedback. In 2019 IEEE International Solid-State Circuits Conference-(ISSCC), pages 140–142. IEEE, 2019.
  37. A 0.086-mm22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT 12.7-pj/sop 64k-synapse 256-neuron online-learning digital spiking neuromorphic processor in 28-nm cmos. IEEE transactions on biomedical circuits and systems, 13(1):145–158, 2018.
  38. Neuromorphic architectures for spiking deep neural networks. In 2015 IEEE International Electron Devices Meeting (IEDM), pages 4–2. IEEE, 2015.
  39. Scalable energy-efficient, low-latency implementations of trained spiking deep belief networks on spinnaker. In 2015 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2015.
  40. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197):668–673, 2014.
  41. Mapping convolutional neural networks onto neuromorphic chip for spike-based computation. In 2021 China Semiconductor Technology International Conference (CSTIC), pages 1–3. IEEE, 2021.
  42. Convolutional networks for fast, energy-efficient neuromorphic computing. Proc. Natl. Acad. Sci. USA, 27:201604850, 2016.
  43. Energy-efficient deployment of machine learning workloads on neuromorphic hardware. In 2022 IEEE 13th International Green and Sustainable Computing Conference (IGSC), pages 1–7. IEEE, 2022.
  44. An efficient spiking neural network for recognizing gestures with a dvs camera on the loihi neuromorphic processor. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–9. IEEE, 2020.
  45. Reckon: A 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales. In 2022 IEEE International Solid-State Circuits Conference (ISSCC), volume 65, pages 1–3. IEEE, 2022.
  46. In-hardware learning of multilayer spiking neural networks on a neuromorphic processor. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 367–372. IEEE, 2021.
  47. spynnaker: a software package for running pynn simulations on spinnaker. Frontiers in neuroscience, 12:816, 2018.
  48. Click elements: An implementation style for data-driven compilation. In 2010 IEEE Symposium on Asynchronous Circuits and Systems, pages 3–14. IEEE, 2010.
  49. Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in computational neuroscience, 9:99, 2015.
  50. Multi-level firing with spiking ds-resnet: Enabling better and deeper directly-trained spiking neural networks. Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22), 14(1):2471–2477, 2022.
  51. Fast-snn: Fast spiking neural network by converting quantized ann. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):14546–14562, 2023.
  52. Intrusion detection system using voting-based neural network. Tsinghua Science and Technology, 26(4):484–495, 2021.
Citations (10)
View on Semantic Scholar

Summary

  • The paper introduces Darwin3, a neuromorphic chip supporting up to 2.35M neurons through a novel instruction set architecture that enables flexible neuron modeling and local learning.
  • The chip employs an innovative topology compression mechanism that reduces memory usage and improves code density by up to 28.3x compared to previous designs.
  • The design integrates on-chip learning for real-time adaptation in spiking neural networks, marking a significant leap in neuromorphic computing performance.

Introduction

Emerging as a new frontier in computing, Spiking Neural Networks (SNNs) offer a closer simulation of biological brains compared to traditional artificial neural networks. This similarity may lead to more power-efficient processing of spatio-temporal data. To fully exploit the advantages of SNNs, dedicated hardware, known as neuromorphic chips, has become an area of active research. These chips execute SNNs via specialized neuron and synapse circuits, offering a different computational paradigm from traditional CPUs and GPUs.

Novel Neuromorphic Chip Design

Darwin3 is a recently designed large-scale neuromorphic chip that represents a leap in neuron scale, supporting up to 2.35 million neurons. A standout feature of Darwin3 is its newly developed instruction set architecture (ISA), offering a set of 10 primary instructions and several extended instructions. This ISA enables flexible programming of various neuron models and the design of local learning rules, which is crucial for simulating different biological neuron types and synaptic behaviors.

Topology Compression and On-Chip Learning

The Darwin3 chip architecture employs an innovative compression mechanism for synaptic connections, which dramatically reduces memory usage for storing these connections. This advancement leads to improved memory efficiency, particularly evident when mapping models converted from Convolutional Neural Networks (CSNNs) to the chip.

Moreover, Darwin3 includes on-chip learning capabilities, supporting the potential for SNNs to learn and adapt in real-time. On-chip learning is a pivotal aspect of biological neural networks and a desired feature in neuromorphic chips, though often limited in scope on existing platforms.

Performance and Scalability

Experimental results reveal that Darwin3 significantly enhances code density, with improvements of up to 28.3x, and shows substantial increases in neuron core fan-in and fan-out due to its connection compression technique compared to its predecessors. The chip demonstrates state-of-the-art performance in both accuracy and latency when benchmarked against other neuromorphic chips in typical SNN applications, highlighting its effective use for both inference and learning tasks. The mesh architecture featuring a large array of computing nodes also positions Darwin3 to scale up effectively, opening the door to more complex and expansive neural network simulations.

In summary, Darwin3's novel ISA, efficient topology representation, large-scale support, and advanced on-chip learning make it a significant advancement in neuromorphic computing, potentially shaping future research and applications in this progressive domain.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets