Transductive Spiking Graph Neural Networks for Loihi (2404.17048v1)
Abstract: Graph neural networks have emerged as a specialized branch of deep learning, designed to address problems where pairwise relations between objects are crucial. Recent advancements utilize graph convolutional neural networks to extract features within graph structures. Despite promising results, these methods face challenges in real-world applications due to sparse features, resulting in inefficient resource utilization. Recent studies draw inspiration from the mammalian brain and employ spiking neural networks to model and learn graph structures. However, these approaches are limited to traditional Von Neumann-based computing systems, which still face hardware inefficiencies. In this study, we present a fully neuromorphic implementation of spiking graph neural networks designed for Loihi 2. We optimize network parameters using Lava Bayesian Optimization, a novel hyperparameter optimization system compatible with neuromorphic computing architectures. We showcase the performance benefits of combining neuromorphic Bayesian optimization with our approach for citation graph classification using fixed-precision spiking neurons. Our results demonstrate the capability of integer-precision, Loihi 2 compatible spiking neural networks in performing citation graph classification with comparable accuracy to existing floating point implementations.
- Watch Your Step: Learning Graph Embeddings Through Attention. CoRR abs/1710.09599 (2017). arXiv:1710.09599 http://arxiv.org/abs/1710.09599
- Provable Neuromorphic Advantages for Computing Shortest Paths. Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (2020). https://api.semanticscholar.org/CorpusID:220434243
- TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 34, 10 (2015), 1537–1557. https://doi.org/10.1109/TCAD.2015.2474396
- Hyperparameter Optimization and Feature Inclusion in Graph Neural Networks for Spiking Implementation. In 2023 International Conference on Machine Learning and Applications (ICMLA). 1541–1546. https://doi.org/10.1109/ICMLA58977.2023.00232
- Guojing Cong and Seung-Hwan Lim. 2021. Versatile feature learning with graph convolutions and graph structures. In 2021 International Conference on Data Mining Workshops (ICDMW). 669–677. https://doi.org/10.1109/ICDMW53433.2021.00088
- Semi-supervised graph structure learning on neuromorphic computers. In Proceedings of the International Conference on Neuromorphic Systems 2022. 1–4.
- Semi-Supervised Graph Structure Learning on Neuromorphic Computers. In Proceedings of the International Conference on Neuromorphic Systems 2022 (Knoxville, TN, USA) (ICONS ’22). Association for Computing Machinery, New York, NY, USA, Article 28, 4 pages. https://doi.org/10.1145/3546790.3546821
- Loihi: A Neuromorphic Manycore Processor with On-Chip Learning. IEEE Micro 38, 1 (2018), 82–99. https://doi.org/10.1109/MM.2018.112130359
- Graph Random Neural Network. CoRR abs/2005.11079 (2020). arXiv:2005.11079 https://arxiv.org/abs/2005.11079
- A neuronal learning rule for sub-millisecond temporal coding. Nature 383 (1996), 76–78. https://api.semanticscholar.org/CorpusID:4319500
- Marc-Oliver Gewaltig and Markus Diesmann. 2007. NEST (NEural Simulation Tool). Scholarpedia 2, 4 (2007), 1430.
- Self-Paced Co-Training of Graph Neural Networks for Semi-Supervised Node Classification. IEEE Transactions on Neural Networks and Learning Systems 34, 11 (2023), 9234–9247. https://doi.org/10.1109/TNNLS.2022.3157688
- Neuromorphic Graph Algorithms: Extracting Longest Shortest Paths and Minimum Spanning Trees. Proceedings of the 2020 Annual Neuro-Inspired Computational Elements Workshop (2020). https://api.semanticscholar.org/CorpusID:218593937
- Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs/1609.02907 (2016). arXiv:1609.02907 http://arxiv.org/abs/1609.02907
- Intel Labs. 2024. Lava Software Framework. https://lava-nc.org Accessed: 2024-3-16.
- Convolutional networks for images, speech, and time series. (1995).
- Automating the Construction of Internet Portals with Machine Learning. Information Retrieval 3(2) (11 2000). https://doi.org/10.1023/A:1009953814988
- Shortest Path and Neighborhood Subgraph Extraction on a Spiking Memristive Neuromorphic Implementation. In Proceedings of the 7th Annual Neuro-Inspired Computational Elements Workshop (Albany, NY, USA) (NICE ’19). Association for Computing Machinery, New York, NY, USA, Article 3, 6 pages. https://doi.org/10.1145/3320288.3320290
- Collective Classification in Network Data. AI Magazine 29, 3 (Sep. 2008), 93. https://doi.org/10.1609/aimag.v29i3.2157
- Neuromorphic Bayesian Optimization in Lava. In Proceedings of the 2023 International Conference on Neuromorphic Systems (Santa Fe, NM USA) (ICONS ’23). Association for Computing Machinery, New York, NY, USA, Article 9, 5 pages. https://doi.org/10.1145/3589737.3605998
- Graph convolutional networks for computational drug development and discovery. Briefings in Bioinformatics 21, 3 (06 2019), 919–935. https://doi.org/10.1093/bib/bbz042 arXiv:https://academic.oup.com/bib/article-pdf/21/3/919/33227266/bbz042.pdf
- Tian Xie and Jeffrey C. Grossman. 2018. Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. Physical Review Letters 120, 14 (April 2018). https://doi.org/10.1103/physrevlett.120.145301
- Graph Convolutional Networks for Text Classification. arXiv:1809.05679 [cs.CL]
- Graph convolutional networks: a comprehensive review. Computational Social Networks 6, 1 (2019), 1–23.