Genetic Motifs as a Blueprint for Mismatch-Tolerant Neuromorphic Computing (2410.19403v1)
Abstract: Mixed-signal implementations of SNNs offer a promising solution to edge computing applications that require low-power and compact embedded processing systems. However, device mismatch in the analog circuits of these neuromorphic processors poses a significant challenge to the deployment of robust processing in these systems. Here we introduce a novel architectural solution inspired by biological development to address this issue. Specifically we propose to implement architectures that incorporate network motifs found in developed brains through a differentiable re-parameterization of weight matrices based on gene expression patterns and genetic rules. Thanks to the gradient descent optimization compatibility of the method proposed, we can apply the robustness of biological neural development to neuromorphic computing. To validate this approach we benchmark it using the Yin-Yang classification dataset, and compare its performance with that of standard multilayer perceptrons trained with state-of-the-art hardware-aware training method. Our results demonstrate that the proposed method mitigates mismatch-induced noise without requiring precise device mismatch measurements, effectively outperforming alternative hardware-aware techniques proposed in the literature, and providing a more general solution for improving the robustness of SNNs in neuromorphic hardware.
- E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for deep learning in NLP,” CoRR, vol. abs/1906.02243, 2019.
- G. Indiveri and Y. Sandamirskaya, “The importance of space and time for signal processing in neuromorphic agents,” IEEE Signal Processing Magazine, vol. 36, no. 6, pp. 16–28, 2019.
- O. Richter, C. Wu, A. M. Whatley, G. Köstinger, C. Nielsen, N. Qiao, and G. Indiveri, “DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processor,” Neuromorphic Computing and Engineering, vol. 4, p. 014003, Jan. 2024.
- S. Moradi, N. Qiao, F. Stefanini, and G. Indiveri, “A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs),” IEEE Transactions on Biomedical Circuits and Systems, vol. 12, pp. 106–122, Feb. 2018.
- C. Pehle, S. Billaudelle, B. Cramer, J. Kaiser, K. Schreiber, Y. Stradmann, J. Weis, A. Leibfried, E. Müller, and J. Schemmel, “The brainscales-2 accelerated neuromorphic system with hybrid plasticity,” Frontiers in Neuroscience, vol. 16, 2022.
- A. Neckar, S. Fok, B. V. Benjamin, T. C. Stewart, N. N. Oza, A. R. Voelker, C. Eliasmith, R. Manohar, and K. Boahen, “Braindrop: A mixed-signal neuromorphic architecture with a dynamical systems-based programming model,” Proceedings of the IEEE, vol. 107, pp. 144–164, Jan. 2019.
- M. Giulioni, P. Camilleri, V. Dante, D. Badoni, G. Indiveri, J. Braun, and P. Del Giudice, “A VLSI network of spiking neurons with plastic fully configurable “stop-learning” synapses,” in International Conference on Electronics, Circuits, and Systems, ICECS 2008, pp. 678–681, IEEE, 2008.
- E. Chicca, D. Badoni, V. Dante, M. D’Andreagiovanni, G. Salina, L. Carota, S. Fusi, and P. Del Giudice, “A VLSI recurrent network of integrate–and–fire neurons connected by plastic synapses with long term memory,” IEEE Transactions on Neural Networks, vol. 14, pp. 1297–1307, Sept. 2003.
- MIT Press, 2002.
- Wiley, 2014.
- K. Cameron and A. Murray, “Minimizing the effect of process mismatch in a neuromorphic system using spike-timing-dependent adaptation,” Neural Networks, IEEE Transactions on, vol. 19, pp. 899–913, May 2008.
- E. Neftci and G. Indiveri, “A device mismatch compensation method for VLSI spiking neural networks,” in Biomedical Circuits and Systems Conference (BioCAS), pp. 262–265, IEEE, 2010.
- S.-A. Bamford, A.-F. Murray, and D.-J. Willshaw, “Silicon synapses self-correct for both mismatch and design inhomogeneities,” Electronics letters, vol. 18, pp. 360–361, 2013.
- J. Büchel, D. Zendrikov, S. Solinas, G. Indiveri, and D. R. Muir, “Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors,” Scientific Reports, vol. 11, Dec. 2021.
- U. Cakal, Maryada, C. Wu, I. Ulusoy, and D. R. Muir, “Gradient-descent hardware-aware training and deployment for mixed-signal neuromorphic processors,” Neuromorphic Computing and Engineering, vol. 4, p. 014011, mar 2024.
- A. Gaier and D. Ha, “Weight agnostic neural networks,” in Advances in Neural Information Processing Systems (H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, eds.), vol. 32, Curran Associates, Inc., 2019.
- T. Boccato, M. Ferrante, and N. Toschi, “Optimizing genetically-driven synaptogenesis,” in ICLR 2024 Workshop on Machine Learning for Genomics Explorations, 2024.
- C. Stöckl, D. Lang, and W. Maass, “Structure induces computational function in networks with diverse types of spiking neurons,” bioRxiv, 2022.
- A. M. Zador, “A critique of pure learning and what artificial neural networks can learn from animal brains,” Nature Communications, vol. 10, p. 3770, Aug. 2019.
- I. A. Kovács, D. L. Barabási, and A.-L. Barabási, “Uncovering the genetic blueprint of the ¡i¿c. elegans¡/i¿ nervous system,” Proceedings of the National Academy of Sciences, vol. 117, no. 52, pp. 33570–33577, 2020.
- D. L. Barabási, T. Beynon, A. Katona, and N. Perez-Nieves, “Complex computation from developmental priors,” Nature Communications, vol. 14, p. 2226, Apr. 2023.
- E. O. Neftci, H. Mostafa, and F. Zenke, “Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks,” IEEE Signal Processing Magazine, vol. 36, no. 6, pp. 51–63, 2019.
- L. Kriener, J. Göltz, and M. A. Petrovici, “The yin-yang dataset,” in Proceedings of the 2022 Annual Neuro-Inspired Computational Elements Conference, NICE ’22, (New York, NY, USA), p. 107–111, Association for Computing Machinery, 2022.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (Y. Bengio and Y. LeCun, eds.), 2015.
- D. Zendrikov, S. Solinas, and G. Indiveri, “Brain-inspired methods for achieving robust computation in heterogeneous mixed-signal neuromorphic processing systems,” Neuromorphic Computing and Engineering, vol. 3, p. 034002, jul 2023.