Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Continuous Spiking Graph Neural Networks (2404.01897v1)

Published 2 Apr 2024 in cs.NE, cs.AI, and cs.LG

Abstract: Continuous graph neural networks (CGNNs) have garnered significant attention due to their ability to generalize existing discrete graph neural networks (GNNs) by introducing continuous dynamics. They typically draw inspiration from diffusion-based methods to introduce a novel propagation scheme, which is analyzed using ordinary differential equations (ODE). However, the implementation of CGNNs requires significant computational power, making them challenging to deploy on battery-powered devices. Inspired by recent spiking neural networks (SNNs), which emulate a biological inference process and provide an energy-efficient neural architecture, we incorporate the SNNs with CGNNs in a unified framework, named Continuous Spiking Graph Neural Networks (COS-GNN). We employ SNNs for graph node representation at each time step, which are further integrated into the ODE process along with time. To enhance information preservation and mitigate information loss in SNNs, we introduce the high-order structure of COS-GNN, which utilizes the second-order ODE for spiking representation and continuous propagation. Moreover, we provide the theoretical proof that COS-GNN effectively mitigates the issues of exploding and vanishing gradients, enabling us to capture long-range dependencies between nodes. Experimental results on graph-based learning tasks demonstrate the effectiveness of the proposed COS-GNN over competitive baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. Interaction networks for learning about objects, relations and physics. Proceedings of the Conference on Neural Information Processing Systems, 2016.
  2. Long short-term memory and learning-to-learn in networks of spiking neurons. In Proceedings of the Conference on Neural Information Processing Systems, pp.  795–805, 2018.
  3. Spikeprop: Backpropagation for networks of spiking neurons error-backpropagation in a network of spik-ing neurons. In ESANN, 2000.
  4. Residual gated graph convnets. arXiv preprint arXiv:1711.07553, 2017.
  5. Simulation of networks of spiking neurons: a review of tools and strategies. Journal of computational neuroscience, 23:349–398, 2007.
  6. Grand: Graph neural diffusion. In Proceedings of the International Conference on Machine Learning, pp.  1407–1418, 2021.
  7. Simple and deep graph convolutional networks. In Proceedings of the International Conference on Machine Learning, pp.  1725–1735, 2020.
  8. Neural ordinary differential equations. In Proceedings of the Conference on Neural Information Processing Systems, 2018.
  9. Adaptive universal generalized pagerank graph neural network. In Proceedings of the International Conference on Learning Representations, 2020.
  10. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the Conference on Neural Information Processing Systems, 2016.
  11. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 2023.
  12. Backpropagation for energy-efficient neuromorphic computing. In Proceedings of the Conference on Neural Information Processing Systems, pp.  1117–1125, 2015.
  13. Splinecnn: Fast geometric deep learning with continuous b-spline kernels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  869–877, 2018.
  14. Probabilistic numeric convolutional neural networks. In Proceedings of the International Conference on Learning Representations, 2021.
  15. Learning modular simulations for homogeneous systems. In Proceedings of the Conference on Neural Information Processing Systems, pp.  14852–14864, 2022.
  16. Influential nodes detection in dynamic social networks: A survey. Expert Systems with Applications, 159:113642, 2020.
  17. Solving Ordinary Differential Equations I: Nonstiff Problems, volume 8. 01 1993. ISBN 978-3-540-56670-0. doi: 10.1007/978-3-540-78862-1.
  18. Inductive representation learning on large graphs. In Proceedings of the Conference on Neural Information Processing Systems, 2017.
  19. Explainable multivariate time series classification: a deep neural network which learns to attend to important variables as well as time intervals. In Proceedings of the International ACM Conference on Web Search & Data Mining, pp.  607–615, 2021.
  20. Spiking deep residual networks. IEEE Transactions on Neural Networks and Learning Systems, 2021.
  21. Learning continuous system dynamics from irregularly-sampled partial observations. In Proceedings of the Conference on Neural Information Processing Systems, volume 33, pp.  16177–16187, 2020.
  22. Coupled graph ode for learning interacting system dynamics. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp.  705–715, 2021.
  23. A survey of graph neural networks in real world: Imbalance, noise, privacy and ood challenges. arXiv preprint arXiv:2403.04468, 2024.
  24. Spiking-yolo: spiking neural network for energy-efficient object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.  11270–11277, 2020.
  25. Neural relational inference for interacting systems. In Proceedings of the International Conference on Machine Learning, pp.  2688–2697, 2018.
  26. Semi-supervised classification with graph convolutional networks. In Proceedings of the International Conference on Learning Representations, 2017.
  27. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  28. Scaling up dynamic graph representation learning via spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.  8588–8596, 2023.
  29. Learning dynamic embeddings for temporal knowledge graphs. In Proceedings of the International ACM Conference on Web Search & Data Mining, pp.  535–543, 2021.
  30. Hope: High-order graph ode for modeling interacting dynamics. In Proceedings of the International Conference on Machine Learning, pp.  23124–23139, 2023.
  31. Maass, W. Networks of spiking neurons: the third generation of neural network models. Neural networks, 10(9):1659–1671, 1997.
  32. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127–163, 2000.
  33. Training high-performance low-latency spiking neural networks by differentiation on spike representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  12444–12453, 2022.
  34. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  5115–5124, 2017.
  35. Query-driven active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs, volume 8, pp.  1, 2012.
  36. Sa-gda: Spectral augmentation for graph domain adaptation. In Proceedings of the ACM International Conference on Multimedia, pp.  309–318, 2023.
  37. Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287, 2020.
  38. Graph neural ordinary differential equations. arXiv preprint arXiv:1911.07532, 2019.
  39. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. In Proceedings of the International Conference on Machine Learning, 2020.
  40. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in neuroscience, 11:682, 2017.
  41. Graph-coupled oscillator networks. In Proceedings of the International Conference on Machine Learning, pp.  18888–18909, 2022.
  42. Collective classification in network data. AI magazine, 29(3):93–93, 2008.
  43. Graph attention networks. In Proceedings of the International Conference on Learning Representations, 2017.
  44. Simplifying graph convolutional networks. In Proceedings of the International Conference on Machine Learning, pp.  6861–6871, 2019.
  45. Continuous graph neural networks. In Proceedings of the International Conference on Machine Learning, 2020.
  46. How powerful are graph neural networks? In Proceedings of the International Conference on Learning Representations, 2019.
  47. Dynamic hypergraph convolutional network. In Proceedings of the IEEE Conference on Data Engineering, pp.  1621–1634, 2022a.
  48. Deal: An unsupervised domain adaptive framework for graph-level classification. In Proceedings of the ACM International Conference on Multimedia, pp.  3470–3479, 2022b.
  49. Coco: A coupled contrastive framework for unsupervised domain adaptive graph classification. In Proceedings of the International Conference on Machine Learning, pp.  40040–40053, 2023a.
  50. Messages are never propagated alone: Collaborative hypergraph neural network for time-series forecasting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023b.
  51. Dynamic spiking graph neural networks. arXiv preprint arXiv:2401.05373, 2023c.
  52. Dynamic spiking graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.  16495–16503, 2024a.
  53. DREAM: Dual structured exploration with mixup for open-set graph domain adaption. In Proceedings of the International Conference on Learning Representations, 2024b.
  54. Recent advances and new frontiers in spiking neural networks. In Proceedings of the International Joint Conference on Artificial Intelligence, pp.  5670–5677, 2022a.
  55. Improving social network embedding via new second-order continuous graph neural networks. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp.  2515–2523, 2022b.
  56. Pairnorm: Tackling oversmoothing in gnns. arXiv preprint arXiv:1909.12223, 2019.
  57. Generalizing graph neural networks beyond homophily. In Proceedings of the Conference on Neural Information Processing Systems, 2020.
  58. Spiking graph convolutional networks. In Proceedings of the International Joint Conference on Artificial Intelligence, pp.  2434–2440, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Nan Yin (33 papers)
  2. Mengzhu Wan (1 paper)
  3. Li Shen (363 papers)
  4. Hitesh Laxmichand Patel (9 papers)
  5. Baopu Li (45 papers)
  6. Bin Gu (86 papers)
  7. Huan Xiong (42 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now