Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardware (2306.14052v1)

Published 24 Jun 2023 in cs.LG, cs.AR, and cs.DC

Abstract: Graph neural networks (GNNs) are emerging for machine learning research on graph-structured data. GNNs achieve state-of-the-art performance on many tasks, but they face scalability challenges when it comes to real-world applications that have numerous data and strict latency requirements. Many studies have been conducted on how to accelerate GNNs in an effort to address these challenges. These acceleration techniques touch on various aspects of the GNN pipeline, from smart training and inference algorithms to efficient systems and customized hardware. As the amount of research on GNN acceleration has grown rapidly, there lacks a systematic treatment to provide a unified view and address the complexity of relevant works. In this survey, we provide a taxonomy of GNN acceleration, review the existing approaches, and suggest future research directions. Our taxonomic treatment of GNN acceleration connects the existing works and sets the stage for further development in this area.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (177)
  1. Computing graph neural networks: A survey from algorithms to accelerators. ACM Computing Surveys (CSUR) 54, 9 (2021), 1–38.
  2. Uri Alon and Eran Yahav. 2021. On the Bottleneck of Graph Neural Networks and its Practical Implications. In International Conference on Learning Representations. https://openreview.net/forum?id=i80OPhOCVH2
  3. Binary graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9492–9501.
  4. Ripple walk training: A subgraph-based training framework for large and deep graph neural network. In 2021 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
  5. Directional graph networks. In International Conference on Machine Learning. PMLR, 748–758.
  6. András A Benczúr and David R Karger. 1996. Approximating st minimum cuts in Õ (n 2) time. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing. 47–55.
  7. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 (2013).
  8. Gecia Bravo Hermsdorff and Lee Gunderson. 2019. A unifying framework for spectrum-preserving graph sparsification and coarsening. Advances in Neural Information Processing Systems 32 (2019).
  9. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. 535–541.
  10. Graph Coarsening with Neural Networks. In International Conference on Learning Representations. https://openreview.net/forum?id=uxpzitPEooJ
  11. DSP: Efficient GNN Training with Multiple GPUs. In Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming. 392–404.
  12. Fastgcn: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:1801.10247 (2018).
  13. Stochastic training of graph convolutional networks with variance reduction. arXiv preprint arXiv:1710.10568 (2017).
  14. Simple and Deep Graph Convolutional Networks. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daumé III and Aarti Singh (Eds.). PMLR, 1725–1735. https://proceedings.mlr.press/v119/chen20v.html
  15. {{\{{TVM}}\}}: An automated {{\{{End-to-End}}\}} optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 578–594.
  16. A unified lottery ticket hypothesis for graph neural networks. In International Conference on Machine Learning. PMLR, 1695–1706.
  17. Rubik: A hierarchical architecture for efficient graph neural network training. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2021).
  18. On Self-Distilling Graph Neural Network. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Zhi-Hua Zhou (Ed.). International Joint Conferences on Artificial Intelligence Organization, 2278–2284. https://doi.org/10.24963/ijcai.2021/314 Main Track.
  19. Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks. ACM SIGARCH Computer Architecture News 44, 3 (2016), 367–379.
  20. fuseGNN: accelerating graph convolutional neural network training on GPGPU. In 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 1–9.
  21. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 257–266.
  22. One trillion edges: Graph processing at facebook-scale. Proceedings of the VLDB Endowment 8, 12 (2015), 1804–1815.
  23. Minimal variance sampling with provable guarantees for fast training of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1393–1403.
  24. Principal neighbourhood aggregation for graph nets. Advances in Neural Information Processing Systems 33 (2020), 13260–13271.
  25. Language modeling with gated convolutional networks. In International conference on machine learning. PMLR, 933–941.
  26. GraphZoom: A Multi-level Spectral Approach for Accurate and Scalable Graph Embedding. In International Conference on Learning Representations. https://openreview.net/forum?id=r1lGO0EKDH
  27. Xiang Deng and Zhongfei Zhang. 2021. Graph-Free Knowledge Distillation for Graph Neural Networks. arXiv:2105.07519 [cs.LG]
  28. Eta prediction with graph neural networks in google maps. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3767–3776.
  29. Global neighbor sampling for mixed CPU-GPU training on giant graphs. arXiv preprint arXiv:2106.06150 (2021).
  30. Heterogeneous Network Representation Learning.. In IJCAI, Vol. 20. 4861–4867.
  31. Enhancing Graph Neural Network-Based Fraud Detectors against Camouflaged Fraudsters. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (Virtual Event, Ireland) (CIKM ’20). Association for Computing Machinery, New York, NY, USA, 315–324. https://doi.org/10.1145/3340531.3411903
  32. Learned step size quantization. arXiv preprint arXiv:1902.08153 (2019).
  33. Sgquant: Squeezing the last bit on graph neural networks with specialized quantization. In 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 1044–1052.
  34. Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
  35. Gnnautoscale: Scalable and expressive graph neural networks via historical embeddings. In International Conference on Machine Learning. PMLR, 3294–3304.
  36. Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018).
  37. CompactETA: A Fast Inference System for Travel Time Prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3337–3345.
  38. TLPGNN: A Lightweight Two-Level Parallelism Paradigm for Graph Neural Network Computation on GPU. In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing. 122–134.
  39. Swapnil Gandhi and Anand Padmanabha Iyer. 2021. P3: Distributed deep graph learning at scale. In 15th {normal-{\{{USENIX}normal-}\}} Symposium on Operating Systems Design and Implementation ({normal-{\{{OSDI}normal-}\}} 21). 551–568.
  40. AWB-GCN: A graph convolutional network accelerator with runtime workload rebalancing. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 922–936.
  41. I-GCN: A graph convolutional network accelerator with runtime locality enhancement through islandization. In MICRO-54: 54th annual IEEE/ACM international symposium on microarchitecture. 1051–1063.
  42. Neural message passing for quantum chemistry. In International conference on machine learning. PMLR, 1263–1272.
  43. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855–864.
  44. Deep learning with limited numerical precision. In International conference on machine learning. PMLR, 1737–1746.
  45. Inductive representation learning on large graphs. In Advances in neural information processing systems. 1024–1034.
  46. Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017).
  47. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015).
  48. Learning Both Weights and Connections for Efficient Neural Networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (Montreal, Canada) (NIPS’15). MIT Press, Cambridge, MA, USA, 1135–1143.
  49. Distilling the Knowledge in a Neural Network. arXiv:1503.02531 [stat.ML]
  50. OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs. arXiv preprint arXiv:2103.09430 (2021).
  51. Open Graph Benchmark: Datasets for Machine Learning on Graphs. CoRR abs/2005.00687 (2020). arXiv:2005.00687 https://arxiv.org/abs/2005.00687
  52. Featgraph: A flexible and efficient backend for graph neural network systems. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–13.
  53. Heterogeneous graph transformer. In Proceedings of The Web Conference 2020. 2704–2710.
  54. Adaptive sampling towards fast graph representation learning. Advances in neural information processing systems 31 (2018).
  55. Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment. In Annual Meeting of the Association for Computational Linguistics (ACL).
  56. Learning Continuous System Dynamics from Irregularly-Sampled Partial Observations. In Advances in Neural Information Processing Systems.
  57. Coupled Graph ODE for Learning Interacting System Dynamics. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining.
  58. Scaling Up Graph Neural Networks Via Graph Coarsening. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery amp; Data Mining (Virtual Event, Singapore) (KDD ’21). Association for Computing Machinery, New York, NY, USA, 675–684. https://doi.org/10.1145/3447548.3467256
  59. Huawei. 2020. MindSpore. https://e.huawei.com/en/products/cloud-computing-dc/atlas/mindspore
  60. Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations. Journal of Machine Learning Research 18, 187 (2018), 1–30. http://jmlr.org/papers/v18/16-456.html
  61. Weakly Supervised Attention for Hashtag Recommendation using Graph Data. Proceedings of The Web Conference 2020 (2020).
  62. Improving the accuracy, scalability, and performance of graph neural networks with roc. Proceedings of Machine Learning and Systems 2 (2020), 187–198.
  63. Redundancy-free computation for graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 997–1005.
  64. Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models. Journal of cheminformatics 13, 1 (2021), 1–23.
  65. Weiwei Jiang and Jiayun Luo. 2022. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications (2022), 117921.
  66. Condensing Graphs via One-Step Gradient Matching. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 720–730.
  67. Graph Condensation for Graph Neural Networks. arXiv preprint arXiv:2110.07580 (2021).
  68. Accelerating training and inference of graph neural networks with fast sampling and pipelining. Proceedings of Machine Learning and Systems 4 (2022), 172–189.
  69. George Karypis and Vipin Kumar. 1998. A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs. SIAM J. Sci. Comput. 20, 1 (1998), 359–392. https://doi.org/10.1137/S1064827595287997
  70. Brian W Kernighan and Shen Lin. 1970. An efficient heuristic procedure for partitioning graphs. The Bell system technical journal 49, 2 (1970), 291–307.
  71. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  72. Raghuraman Krishnamoorthi. 2018. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342 (2018).
  73. Training graph neural networks with 1000 layers. In International conference on machine learning. PMLR, 6437–6449.
  74. Sgcn: A graph sparsifier based on graph convolutional networks. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 275–287.
  75. Deepburning-gl: an automated framework for generating graph neural network accelerators. In 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 1–9.
  76. Engn: A high-throughput and energy-efficient accelerator for large graph neural networks. IEEE Trans. Comput. 70, 9 (2020), 1511–1525.
  77. A Comprehensive Survey on Distributed Training of Graph Neural Networks. arXiv preprint arXiv:2211.05368 (2022).
  78. Naas: Neural accelerator architecture search. In 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, 1051–1056.
  79. Pagraph: Scaling gnn training on large graphs via computation-aware caching. In Proceedings of the 11th ACM Symposium on Cloud Computing. 401–415.
  80. Bgl: Gpu-efficient gnn training by optimizing graph data i/o and preprocessing. arXiv preprint arXiv:2112.08541 (2021).
  81. Sampling methods for efficient training of graph convolutional networks: A survey. IEEE/CAA Journal of Automatica Sinica 9, 2 (2021), 205–234.
  82. Bandit samplers for training graph neural networks. Advances in Neural Information Processing Systems 33 (2020), 6878–6888.
  83. HGK-GNN: Heterogeneous Graph Kernel Based Graph Neural Networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (Virtual Event, Singapore) (KDD ’21). Association for Computing Machinery, New York, NY, USA, 1129–1138. https://doi.org/10.1145/3447548.3467429
  84. Andreas Loukas. 2019. Graph Reduction with Spectral and Cut Guarantees. J. Mach. Learn. Res. 20, 116 (2019), 1–42.
  85. Andreas Loukas and Pierre Vandergheynst. 2018. Spectrally Approximating Large Graphs with Smaller Graphs. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 3237–3246. https://proceedings.mlr.press/v80/loukas18a.html
  86. {{\{{NeuGraph}}\}}: Parallel Deep Neural Network Computation on Large Graphs. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). 443–458.
  87. Christopher D Manning. 2009. An introduction to information retrieval. Cambridge university press.
  88. UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 1253–1262.
  89. Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. arXiv preprint arXiv:1703.04826 (2017).
  90. A graph placement methodology for fast chip design. Nature 594, 7862 (2021), 207–212.
  91. Measurement and analysis of online social networks. In Proceedings of the 7th ACM SIGCOMM conference on Internet measurement. 29–42.
  92. Marius: Learning Massive Graph Embeddings on a Single Machine. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21). USENIX Association, 533–549. https://www.usenix.org/conference/osdi21/presentation/mohoney
  93. Hesham Mostafa. 2022. Sequential aggregation and rematerialization: Distributed full-batch training of graph neural networks on large graphs. Proceedings of Machine Learning and Systems 4 (2022), 265–275.
  94. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295 (2021).
  95. Sancus: staleness-aware communication-avoiding full-graph decentralized training in large-scale graph neural networks. Proceedings of the VLDB Endowment 15, 9 (2022), 1937–1950.
  96. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543.
  97. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701–710.
  98. Fusedmm: A unified sddmm-spmm kernel for graph embedding and graph neural networks. In 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 256–266.
  99. Learn Locally, Correct Globally: A Distributed Algorithm for Training Graph Neural Networks. In International Conference on Learning Representations. https://openreview.net/forum?id=FndDxSz3LxQ
  100. Russell Reed. 1993. Pruning algorithms-a survey. IEEE transactions on Neural Networks 4, 5 (1993), 740–747.
  101. Kaspar Riesen and Horst Bunke. 2008. IAM graph database repository for graph based pattern recognition and machine learning. In Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR). Springer, 287–297.
  102. Yousef Saad. 2003. Iterative methods for sparse linear systems. SIAM.
  103. Recent advances in recurrent neural networks. arXiv preprint arXiv:1801.01078 (2017).
  104. Learning to simulate complex physics with graph networks. In International Conference on Machine Learning. PMLR, 8459–8468.
  105. FlowGNN: A Dataflow Architecture for Universal Graph Neural Network Inference via Multi-Queue Streaming. arXiv preprint arXiv:2204.13103 (2022).
  106. Modeling relational data with graph convolutional networks. In European semantic web conference. Springer, 593–607.
  107. Collective classification in network data. AI magazine 29, 3 (2008), 93–93.
  108. Marco Serafini and Hui Guan. 2021. Scalable Graph Neural Network Training: The Case for Sampling. ACM SIGOPS Operating Systems Review 55, 1 (2021), 68–76.
  109. Extracting the multiscale backbone of complex weighted networks. Proceedings of the national academy of sciences 106, 16 (2009), 6483–6488.
  110. Distributed Graph Neural Network Training: A Survey. arXiv preprint arXiv:2211.00216 (2022).
  111. Automated Accelerator Optimization Aided by Graph Neural Networks. In 2022 59th ACM/IEEE Design Automation Conference (DAC).
  112. StreamGCN: Accelerating Graph Convolutional Networks with Streaming Processing. In 2022 IEEE Custom Integrated Circuits Conference (CICC). IEEE, 1–8.
  113. End-to-End Optimization of Deep Learning Applications. In FPGA. 133–139.
  114. AutoDSE: Enabling Software Programmers to Design Efficient FPGA Accelerators. ACM Transactions on Design Automation of Electronic Systems (TODAES) 27, 4 (2022), 1–27.
  115. Sextans: A streaming accelerator for general-purpose sparse-matrix dense-matrix multiplication. In Proceedings of the 2022 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 65–77.
  116. Daniel A Spielman and Nikhil Srivastava. 2011. Graph sparsification by effective resistances. SIAM J. Comput. 40, 6 (2011), 1913–1926.
  117. Daniel A Spielman and Shang-Hua Teng. 2014. Nearly linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. SIAM J. Matrix Anal. Appl. 35, 3 (2014), 835–885.
  118. Matraptor: A sparse-sparse matrix multiplication accelerator based on row-wise product. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 766–780.
  119. Degree-quant: Quantization-aware training for graph neural networks. arXiv preprint arXiv:2008.05000 (2020).
  120. Dorylus: Affordable, Scalable, and Accurate {{\{{GNN}}\}} Training with Distributed {{\{{CPU}}\}} Servers and Serverless Threads. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21). 495–514.
  121. Pcgcn: Partition-centric processing for accelerating graph convolutional network. In 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 936–945.
  122. Reducing communication in graph neural network training. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–14.
  123. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
  124. Ulrike Von Luxburg. 2007. A tutorial on spectral clustering. Statistics and computing 17, 4 (2007), 395–416.
  125. MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks. arXiv preprint arXiv:2202.02365 (2022).
  126. BNS-GCN: Efficient full-graph training of graph convolutional networks with partition-parallelism and random boundary node sampling. Proceedings of Machine Learning and Systems 4 (2022).
  127. PipeGCN: Efficient full-graph training of graph convolutional networks with pipelined feature communication. arXiv preprint arXiv:2203.10428 (2022).
  128. Autosa: A polyhedral compiler for high-performance systolic arrays on fpga. In The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 93–104.
  129. Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks. arXiv preprint arXiv:1909.01315 (2019).
  130. Dydiff-vae: A dynamic variational framework for information diffusion prediction. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 163–172.
  131. {{\{{GNNAdvisor}}\}}: An Adaptive and Efficient Runtime System for {{\{{GNN}}\}} Acceleration on {{\{{GPUs}}\}}. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21). 515–531.
  132. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) 38, 5 (2019), 1–12.
  133. PIGEON: Optimizing CUDA Code Generator for End-to-End Training and Inference of Relational Graph Neural Networks. arXiv preprint arXiv:2301.06284 (2023).
  134. Graph neural networks in recommender systems: a survey. ACM Computing Surveys (CSUR) (2020).
  135. Seastar: vertex-centric programming for graph neural networks. In Proceedings of the Sixteenth European Conference on Computer Systems. 359–375.
  136. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
  137. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10687–10698.
  138. Graphiler: Optimizing Graph Neural Networks with Message Passing Data Flow Graph. Proceedings of Machine Learning and Systems 4 (2022), 515–528.
  139. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).
  140. Meta-gradient reinforcement learning. Advances in neural information processing systems 31 (2018).
  141. TinyGNN: Learning Efficient Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 1848–1856.
  142. Hygcn: A gcn accelerator with hybrid architecture. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 15–29.
  143. Extract the knowledge of graph neural networks and go beyond it: An effective knowledge distillation framework. In Proceedings of the Web Conference 2021. 1227–1237.
  144. GNNLab: a factored system for sample-based GNN training over GPUs. In Proceedings of the Seventeenth European Conference on Computer Systems. 417–434.
  145. Deep Molecular Representation Learning via Fusing Physical and Chemical Information. Advances in Neural Information Processing Systems 34 (2021), 16346–16357.
  146. Distilling knowledge from graph convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7074–7083.
  147. Kai-Lang Yao and Wu-Jun Li. 2021. Blocking-based Neighbor Sampling for Large-scale Graph Neural Networks. In International Joint Conference on Artificial Intelligence.
  148. SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning. arXiv preprint arXiv:2207.04606 (2022).
  149. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 974–983.
  150. Hierarchical graph representation learning with differentiable pooling. Advances in neural information processing systems 31 (2018).
  151. Gcod: Graph convolutional network acceleration via dedicated algorithm and accelerator co-design. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 460–474.
  152. Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets. arXiv preprint arXiv:2103.00794 (2021).
  153. Graphfm: Improving large-scale gnn training via feature momentum. In International Conference on Machine Learning. PMLR, 25684–25701.
  154. Hanqing Zeng and Viktor Prasanna. 2020. GraphACT: Accelerating GCN training on CPU-FPGA heterogeneous platforms. In Proceedings of the 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 255–265.
  155. Decoupling the Depth and Scope of Graph Neural Networks. Advances in Neural Information Processing Systems 34 (2021).
  156. Graphsaint: Graph sampling based inductive learning method. arXiv preprint arXiv:1907.04931 (2019).
  157. Accurate, efficient and scalable training of Graph Neural Networks. J. Parallel and Distrib. Comput. 147 (2021), 166–183.
  158. BoostGCN: A framework for optimizing GCN inference on FPGA. In 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, 29–39.
  159. Hardware acceleration of large scale gcn inference. In 2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP). IEEE, 61–68.
  160. Heterogeneous Graph Neural Network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’19). 793–803.
  161. Agl: a scalable system for industrial-purpose graph machine learning. arXiv preprint arXiv:2003.02454 (2020).
  162. Understanding gnn computational graph: A coordinated computation, io, and memory perspective. Proceedings of Machine Learning and Systems 4 (2022), 467–484.
  163. Star-gcn: Stacked and reconstructed graph convolutional networks for recommender systems. arXiv preprint arXiv:1905.13129 (2019).
  164. Graph-less Neural Networks: Teaching Old MLPs New Tricks Via Distillation. In International Conference on Learning Representations. https://openreview.net/forum?id=4p6_5HBWPCw
  165. DNNBuilder: an automated tool for building high-performance DNN hardware accelerators for FPGAs. In ICCAD. IEEE, 1–8.
  166. G-CoS: Gnn-accelerator co-search towards both better accuracy and efficiency. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 1–9.
  167. Learned low precision graph neural networks. arXiv preprint arXiv:2009.09232 (2020).
  168. Robust graph representation learning via neural sparsification. In International Conference on Machine Learning. PMLR, 11458–11468.
  169. Distdgl: distributed graph neural network training for billion-scale graphs. In 2020 IEEE/ACM 10th Workshop on Irregular Applications: Architectures and Algorithms (IA3). IEEE, 36–44.
  170. Distributed hybrid CPU and GPU training for graph neural networks on billion-scale graphs. arXiv preprint arXiv:2112.15345 (2021).
  171. Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods. arXiv preprint arXiv:2111.04840 (2021).
  172. Accelerating large scale real-time GNN inference using channel pruning. arXiv preprint arXiv:2105.04528 (2021).
  173. TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs. arXiv preprint arXiv:2203.14883 (2022).
  174. BlockGNN: Towards Efficient GNN Acceleration Using Block-Circulant Weight Matrices. In 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, 1009–1014.
  175. Aligraph: a comprehensive graph neural network platform. arXiv preprint arXiv:1902.08730 (2019).
  176. Marinka Zitnik and Jure Leskovec. 2017. Predicting multicellular function through multi-layer tissue networks. Bioinformatics 33, 14 (2017), i190–i198.
  177. Layer-dependent importance sampling for training deep and large graph convolutional networks. Advances in neural information processing systems 32 (2019).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (10)
  1. Shichang Zhang (21 papers)
  2. Atefeh Sohrabizadeh (11 papers)
  3. Cheng Wan (48 papers)
  4. Zijie Huang (29 papers)
  5. Ziniu Hu (51 papers)
  6. Yewen Wang (4 papers)
  7. Yingyan (7 papers)
  8. Lin (16 papers)
  9. Jason Cong (62 papers)
  10. Yizhou Sun (149 papers)
Citations (19)
View on Semantic Scholar