Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Neuromorphic on-chip reservoir computing with spiking neural network architectures (2407.20547v1)

Published 30 Jul 2024 in cs.NE, cs.ET, and cs.LG

Abstract: Reservoir computing is a promising approach for harnessing the computational power of recurrent neural networks while dramatically simplifying training. This paper investigates the application of integrate-and-fire neurons within reservoir computing frameworks for two distinct tasks: capturing chaotic dynamics of the H\'enon map and forecasting the Mackey-Glass time series. Integrate-and-fire neurons can be implemented in low-power neuromorphic architectures such as Intel Loihi. We explore the impact of network topologies created through random interactions on the reservoir's performance. Our study reveals task-specific variations in network effectiveness, highlighting the importance of tailored architectures for distinct computational tasks. To identify optimal network configurations, we employ a meta-learning approach combined with simulated annealing. This method efficiently explores the space of possible network structures, identifying architectures that excel in different scenarios. The resulting networks demonstrate a range of behaviors, showcasing how inherent architectural features influence task-specific capabilities. We study the reservoir computing performance using a custom integrate-and-fire code, Intel's Lava neuromorphic computing software framework, and via an on-chip implementation in Loihi. We conclude with an analysis of the energy performance of the Loihi architecture.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks. Nano Express, 5(1):015021, Mar. 2024.
  2. Structural plasticity on an accelerated analog neuromorphic hardware system. Neural Networks, 133:11–20, Jan. 2021.
  3. F. Caravelli and J. P. Carbajal. Memristors for the curious outsiders. Technologies, 6(4):118, 2018.
  4. T. L. Carroll. Do reservoir computers work best at the edge of chaos? Chaos: An Interdisciplinary Journal of Nonlinear Science, 30(12), 12 2020. 121109.
  5. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1):82–99, 2018.
  6. Extended liquid state machines for speech recognition. Frontiers in Neuroscience, 16:1023470, October 2022.
  7. P. Erdős and A. Rényi. On random graphs i. Publicationes Mathematicae Debrecen, 6:290–297, 1959.
  8. Reservoir based spiking models for univariate time series classification. Frontiers in Computational Neuroscience, 17, June 2023.
  9. Spiking reservoir computing for temporal edge intelligence on loihi. In 2022 IEEE/ACM 7th Symposium on Edge Computing (SEC). IEEE, Dec. 2022.
  10. Neuronal Dynamics. Cambridge University Press, July 2014.
  11. S. Greengard. Neuromorphic chips take shape. Communications of the ACM, 63(8):9–11, July 2020.
  12. S. Haykin. Neural Networks and Learning Machines. Parson Press, 2016.
  13. Learning to learn using gradient descent. Lecture Notes in Computer Science, Proceedings of the International Conference on Artificial Neural Networks, 2015.
  14. D. Ielmini and H.-S. P. Wong. In-memory computing with resistive switching devices. Nature Ele., 1(6):333–343, 2018.
  15. Spike-based learning with a generalized integrate and fire silicon neuron. In Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pages 1951–1954, 2010.
  16. Intel Labs. Lava. https://github.com/lava-nc, 2023.
  17. Increasing liquid state machine performance with edge-of-chaos dynamics organized by astrocyte-modulated plasticity. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021.
  18. E. M. Izhikevich. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17(10):2443–2452, Jan. 2007.
  19. H. Jaeger and H. Haas. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304(5667):78–80, Apr. 2004.
  20. A. Johannson and J. Zou. A slime mold solver for linear programming problems. In Conference on Computability in Europe, pages 344–354. Springer, 2012.
  21. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural computation, 14(11):2531–2560, 2002.
  22. C. A. Mead. Analog vlsi and neural systems. Analog Integrated Circuits and Signal Processing, 1(1):3–14, 1989.
  23. SpiNNaker: A multi-core system-on-chip for massively-parallel neural net simulation. In Proceedings of the IEEE 2012 Custom Integrated Circuits Conference. IEEE, Sept. 2012.
  24. Liquid state machine on loihi: Memory metric for performance prediction. In E. Pimenidis, P. Angelov, C. Jayne, A. Papaleonidas, and M. Aydin, editors, Artificial Neural Networks and Machine Learning – ICANN 2022, volume 13531 of Lecture Notes in Computer Science. Springer, 2022.
  25. Liquid state machine on spinnaker for spatio-temporal classification tasks. Frontiers in Neuroscience, 16, 2022.
  26. The backpropagation algorithm implemented on spiking neuromorphic hardware. arXiv:arXiv:2106.07030, 2021.
  27. spynnaker: a software package for running pynn simulations on spinnaker. Frontiers in Neuroscience, 12:816, 2018.
  28. F. Salam. On the analysis and design of neural nets with an implementation via cmos vlsi. In 1988., IEEE International Symposium on Circuits and Systems, pages 1001–1004 vol.2, 1988.
  29. A. Sebastian and et. al. Memory devices and applications for in-memory computing. Nature Nano., 15(7):529–544, 2020.
  30. Going deeper in spiking neural networks: VGG and residual architectures. Frontiers in Neuroscience, 13, Mar. 2019.
  31. Computational capacity of l⁢r⁢c𝑙𝑟𝑐lrcitalic_l italic_r italic_c, memristive, and hybrid reservoirs. Phys. Rev. E, 106:045310, Oct 2022.
  32. Backpropagation with biologically plausible spatiotemporal adjustment for training deep spiking neural networks. Patterns, 3(6):100522, June 2022.
  33. Implementation of a liquid state machine with temporal dynamics on a novel spiking neuromorphic architecture. 10 2016.
  34. N. Soures and D. Kudithipudi. Spiking reservoir networks: Brain-inspired recurrent algorithms that use random, fixed synaptic strengths. IEEE Signal Processing Magazine, 36(6):78–87, Nov. 2019.
  35. A neural architecture search based framework for liquid state machine design. Neurocomputing, 443:174–182, 2021.
  36. Collective dynamics of ’small-world’ networks. Nature, 393(6684):440–442, 1998.
  37. Editorial: Physical neuromorphic computing and its industrial applications. Frontiers in Neuroinformatics, 17, July 2023.
  38. Memristive reservoirs learn to learn. arXiv:2306.12676, to appear in Proceedings of the ICONS conference, 2023.

Summary

We haven't generated a summary for this paper yet.