Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning (2401.04491v1)

Published 9 Jan 2024 in cs.ET, cs.LG, and cs.NE

Abstract: The joint progress of artificial neural networks (ANNs) and domain specific hardware accelerators such as GPUs and TPUs took over many domains of machine learning research. This development is accompanied by a rapid growth of the required computational demands for larger models and more data. Concurrently, emerging properties of foundation models such as in-context learning drive new opportunities for machine learning applications. However, the computational cost of such applications is a limiting factor of the technology in data centers, and more importantly in mobile devices and edge systems. To mediate the energy footprint and non-trivial latency of contemporary systems, neuromorphic computing systems deeply integrate computational principles of neurobiological systems by leveraging low-power analog and digital technologies. SpiNNaker2 is a digital neuromorphic chip developed for scalable machine learning. The event-based and asynchronous design of SpiNNaker2 allows the composition of large-scale systems involving thousands of chips. This work features the operating principles of SpiNNaker2 systems, outlining the prototype of novel machine learning applications. These applications range from ANNs over bio-inspired spiking neural networks to generalized event-based neural networks. With the successful development and deployment of SpiNNaker2, we aim to facilitate the advancement of event-based and asynchronous algorithms for future generations of machine learning systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. SpiNNcloud Systems GmbH, 2023. URL https://spinncloud.com/. Accessed on October 3, 2023.
  2. SpiNNworld: SpiNNaker presence worldwide, 2023. URL https://www.google.com/maps/d/u/0/edit?mid=1jrbV2OVaBFqGlVYMxerSh0Pcexd_wznQ&ll=1.5170674512027844%2C0&z=2. Accessed on October 3, 2023.
  3. A Low Power, Fully Event-Based Gesture Recognition System. page 10.
  4. Machine learning systems are stuck in a rut. In Proceedings of the Workshop on Hot Topics in Operating Systems, HotOS ’19, page 177–183, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450367271. doi: 10.1145/3317550.3321441. URL https://doi.org/10.1145/3317550.3321441.
  5. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11(1):3625, July 2020. ISSN 2041-1723. doi: 10.1038/s41467-020-17236-y.
  6. Spinnaker—programming model. IEEE Transactions on Computers, 64(6):1769–1782, 2014.
  7. Hungry hungry hippos: Towards language modeling with state space models. arXiv preprint arXiv:2212.14052, 2022.
  8. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38(1):82–99, 2018.
  9. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 111(9):1016–1054, 2023.
  10. Steve Furber and Petrut Bogdan, editors. SpiNNaker: A Spiking Neural Network Architecture. now publishers, Boston-Delft, 2020. doi: 10.1561/9781680836523. URL http://dx.doi.org/10.1561/9781680836523.
  11. The spinnaker project. Proceedings of the IEEE, 102(5):652–665, 2014. doi: 10.1109/JPROC.2014.2304638.
  12. Ai and memory wall. RiseLab Medium Post, 2021. URL https://github.com/amirgholami/ai_and_memory_wall.
  13. P Hammer. Reasoning-learning systems based on non-axiomatic reasoning system theory, vol. 192. 2022.
  14. Sara Hooker. The hardware lottery. Commun. ACM, 64(12):58–65, nov 2021. ISSN 0001-0782. doi: 10.1145/3467017. URL https://doi.org/10.1145/3467017.
  15. Spiking neural network based real-time radar gesture recognition live demonstration. In 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 500–500, 2022a. doi: 10.1109/AICAS54282.2022.9869943.
  16. Real-time radar gesture classification with spiking neural network on spinnaker 2 prototype. In 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 362–365, 2022b. doi: 10.1109/AICAS54282.2022.9869987.
  17. Efficient snn multi-cores mac array acceleration on spinnaker 2. Frontiers in Neuroscience, 17, 2023a. ISSN 1662-453X. doi: 10.3389/fnins.2023.1223262. URL https://www.frontiersin.org/articles/10.3389/fnins.2023.1223262.
  18. Efficient algorithms for accelerating spiking neural networks on mac array of spinnaker 2. In 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 1–5, 2023b. doi: 10.1109/AICAS57966.2023.10168559.
  19. Live demonstration: Dynamic voltage and frequency scaling for neuromorphic many-core systems. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1–1, 2017. doi: 10.1109/ISCAS.2017.8050396.
  20. Dynamic power management for neuromorphic many-core systems. IEEE Transactions on Circuits and Systems I: Regular Papers, 66(8):2973–2986, 2019. doi: 10.1109/TCSI.2019.2911898.
  21. The spinnaker 2 processing element architecture for hybrid digital neuromorphic computing, 2022.
  22. The wormhole ai training processor. In 2022 IEEE International Solid- State Circuits Conference (ISSCC), volume 65, pages 356–358, 2022. doi: 10.1109/ISSCC42614.2022.9731633.
  23. A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning. eNeuro, 5(2), 2018. doi: 10.1523/ENEURO.0301-17.2018. URL https://www.eneuro.org/content/5/2/ENEURO.0301-17.2018.
  24. Mapping deep neural networks on spinnaker2. In Proceedings of the 2020 Annual Neuro-Inspired Computational Elements Workshop, NICE ’20, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450377188. doi: 10.1145/3381755.3381778. URL https://doi.org/10.1145/3381755.3381778.
  25. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
  26. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. URL https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf.
  27. Memory-efficient deep learning on a spinnaker 2 prototype. Frontiers in Neuroscience, 12, 2018. ISSN 1662-453X. doi: 10.3389/fnins.2018.00840. URL https://www.frontiersin.org/articles/10.3389/fnins.2018.00840.
  28. Spinnaker: A 1-w 18-core system-on-chip for massively-parallel neural network simulation. IEEE Journal of Solid-State Circuits, 48(8):1943–1953, 2013. doi: 10.1109/JSSC.2013.2259038.
  29. Neuromorphic intermediate representation: A unified instruction set for interoperable brain-inspired computing, 2023.
  30. Norse - A deep learning library for spiking neural networks, January 2021. URL https://doi.org/10.5281/zenodo.4422025. Documentation: https://norse.ai/docs/.
  31. The brainscales-2 accelerated neuromorphic system with hybrid plasticity. Frontiers in Neuroscience, 16:795876, 2022.
  32. Rwkv: Reinventing rnns for the transformer era, 2023.
  33. Sambanova sn10 rdu: A 7nm dataflow architecture to accelerate software 2.0. In 2022 IEEE International Solid- State Circuits Conference (ISSCC), volume 65, pages 350–352, 2022. doi: 10.1109/ISSCC42614.2022.9731612.
  34. E-prop on spinnaker 2: Exploring online learning in spiking rnns on neuromorphic hardware. Frontiers in Neuroscience, 16, 2022. ISSN 1662-453X. doi: 10.3389/fnins.2022.1018006. URL https://www.frontiersin.org/articles/10.3389/fnins.2022.1018006.
  35. Spinntools: the execution engine for the spinnaker platform. Frontiers in neuroscience, 13:231, 2019.
  36. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in Neuroscience, 11, 2017. ISSN 1662-453X. doi: 10.3389/fnins.2017.00682. URL https://www.frontiersin.org/articles/10.3389/fnins.2017.00682.
  37. Compute trends across three eras of machine learning. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2022. doi: 10.1109/IJCNN55064.2022.9891914.
  38. Efficient recurrent architectures through activity sparsity and sparse back-propagation through time. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=lJdOlWg8td.
  39. Retentive network: A successor to transformer for large language models, 2023.
  40. A review of learning in biologically plausible spiking neural networks. Neural Networks, 122:253–272, 2020. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2019.09.036. URL https://www.sciencedirect.com/science/article/pii/S0893608019303181.
  41. Deep learning in spiking neural networks. Neural Networks, 111:47–63, 2019. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2018.12.002. URL https://www.sciencedirect.com/science/article/pii/S0893608018303332.
  42. The computational limits of deep learning, 2022.
  43. py-spinnaker2, November 2023. URL https://doi.org/10.5281/zenodo.10202110.
  44. Deep learning incorporating biologically inspired neural dynamics and in-memory computing. Nature Machine Intelligence, 2(6):325–336, Jun 2020. ISSN 2522-5839. doi: 10.1038/s42256-020-0187-0. URL https://doi.org/10.1038/s42256-020-0187-0.
  45. Event-based backpropagation can compute exact gradients for spiking neural networks. Scientific Reports, 11(1):12829, Jun 2021. ISSN 2045-2322. doi: 10.1038/s41598-021-91786-z.
  46. Spiking neural networks and their applications: A review. Brain Sciences, 12(7), 2022. ISSN 2076-3425. doi: 10.3390/brainsci12070863. URL https://www.mdpi.com/2076-3425/12/7/863.
  47. Yexin Yan. Implementation of bioinspired algorithms on the neuromorphic VLSI system SpiNNaker 2. PhD thesis, TU Dresden, 2022.
  48. Efficient reward-based structural plasticity on a spinnaker 2 prototype. IEEE Transactions on Biomedical Circuits and Systems, 13(3):579–591, 2019. doi: 10.1109/TBCAS.2019.2906401.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (14)
  1. Hector A. Gonzalez (5 papers)
  2. Jiaxin Huang (48 papers)
  3. Florian Kelber (5 papers)
  4. Khaleelulla Khan Nazeer (8 papers)
  5. Tim Langer (1 paper)
  6. Chen Liu (206 papers)
  7. Matthias Lohrmann (2 papers)
  8. Amirhossein Rostami (2 papers)
  9. Mark Schöne (8 papers)
  10. Bernhard Vogginger (17 papers)
  11. Timo C. Wunderlich (5 papers)
  12. Yexin Yan (5 papers)
  13. Mahmoud Akl (4 papers)
  14. Christian Mayr (35 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube