Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DPGNN: Dual-Perception Graph Neural Network for Representation Learning (2110.07869v3)

Published 15 Oct 2021 in cs.LG

Abstract: Graph neural networks (GNNs) have drawn increasing attention in recent years and achieved remarkable performance in many graph-based tasks, especially in semi-supervised learning on graphs. However, most existing GNNs are based on the message-passing paradigm to iteratively aggregate neighborhood information in a single topology space. Despite their success, the expressive power of GNNs is limited by some drawbacks, such as inflexibility of message source expansion, negligence of node-level message output discrepancy, and restriction of single message space. To address these drawbacks, we present a novel message-passing paradigm, based on the properties of multi-step message source, node-specific message output, and multi-space message interaction. To verify its validity, we instantiate the new message-passing paradigm as a Dual-Perception Graph Neural Network (DPGNN), which applies a node-to-step attention mechanism to aggregate node-specific multi-step neighborhood information adaptively. Our proposed DPGNN can capture the structural neighborhood information and the feature-related information simultaneously for graph representation learning. Experimental results on six benchmark datasets with different topological structures demonstrate that our method outperforms the latest state-of-the-art models, which proves the superiority and versatility of our method. To our knowledge, we are the first to consider node-specific message passing in the GNNs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. MixHop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In Proceedings of the 36th International Conference on Machine Learning (pp. 21–29). PMLR volume 97. URL: https://proceedings.mlr.press/v97/abu-el-haija19a.html.
  2. A multilayered informative random walk for attributed social network embedding. In ECAI 2020 (pp. 1738–1745). IOS Press. doi:https://doi.org/10.3233/FAIA200287.
  3. Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems. Curran Associates, Inc. volume 32. URL: https://proceedings.neurips.cc/paper/2019/file/1cd138d0499a68f4bb72bee04bbec2d7-Paper.pdf.
  4. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In International Conference on Learning Representations. URL: https://openreview.net/forum?id=r1ZdKJ-0W.
  5. Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations.
  6. Multi-view clustering via deep concept factorization. Knowledge-Based Systems, 217, 106807. doi:https://doi.org/10.1016/j.knosys.2021.106807.
  7. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 3438–3445). volume 34. doi:https://doi.org/10.1609/aaai.v34i04.5747.
  8. Neighbor enhanced graph convolutional networks for node classification and recommendation. Knowledge-Based Systems, 246, 108594. doi:https://doi.org/10.1016/j.knosys.2022.108594.
  9. Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning (pp. 1725–1735). PMLR volume 119. URL: https://proceedings.mlr.press/v119/chen20v.html.
  10. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. In Advances in Neural Information Processing Systems (pp. 19314–19326). Curran Associates, Inc. volume 33. URL: https://proceedings.neurips.cc/paper/2020/file/e05c7ba4e087beea9410929698dc41a6-Paper.pdf.
  11. Explainable link prediction in knowledge hypergraphs. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (pp. 262–271). New York, NY, USA: Association for Computing Machinery. URL: https://doi.org/10.1145/3511808.3557316. doi:10.1145/3511808.3557316.
  12. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems. Curran Associates, Inc. volume 29. URL: https://proceedings.neurips.cc/paper/2016/file/04df4d434d481c5bb723be1b6df1ee65-Paper.pdf.
  13. Graph random neural networks for semi-supervised learning on graphs. In Advances in Neural Information Processing Systems (pp. 22092–22103). Curran Associates, Inc. volume 33. URL: https://proceedings.neurips.cc/paper/2020/file/fb4c835feb0a65cc39739320d7a51c02-Paper.pdf.
  14. Learning discrete structures for graph neural networks. In Proceedings of the 36th International Conference on Machine Learning (pp. 1972–1982). PMLR volume 97. URL: https://proceedings.mlr.press/v97/franceschi19a.html.
  15. Graph drawing by force-directed placement. Software: Practice and experience, 21, 1129–1164. doi:https://doi.org/10.1002/spe.4380211102.
  16. Large-scale learnable graph convolutional networks. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1416–1424). doi:https://doi.org/10.1145/3219819.3219947.
  17. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning (pp. 1263–1272). PMLR volume 70. URL: https://proceedings.mlr.press/v70/gilmer17a.html.
  18. Exploring network structure, dynamics, and function using networkx. Technical Report. URL: https://www.osti.gov/biblio/960616.
  19. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).
  20. Semi-supervised learning with graph learning-convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 11313–11320). doi:https://doi.org/10.1109/CVPR.2019.01157.
  21. Structured graph learning for clustering and semi-supervised classification. Pattern Recognition, 110, 107627. doi:https://doi.org/10.1016/j.patcog.2020.107627.
  22. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 1725–1732).
  23. Adam: A method for stochastic optimization, .
  24. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations.
  25. Predict then propagate: Graph neural networks meet personalized pagerank. In International Conference on Learning Representations.
  26. A knowledge graph completion model based on contrastive learning and relation enhancement method. Knowledge-Based Systems, 256, 109889. doi:https://doi.org/10.1016/j.knosys.2022.109889.
  27. Deeper insights into graph convolutional networks for semi-supervised learning. In Thirty-Second AAAI conference on artificial intelligence (p. 3538–3545).
  28. Collaborative representation learning for nodes and relations via heterogeneous graph neural network. Knowledge-Based Systems, 255, 109673. doi:https://doi.org/10.1016/j.knosys.2022.109673.
  29. Transo: a knowledge-driven representation learning method with ontology information constraints. World Wide Web, (pp. 1–23).
  30. Deep attributed network representation learning of complex coupling and interaction. Knowledge-Based Systems, 212, 106618.
  31. Deep linear graph attention model for attributed graph clustering. Knowledge-Based Systems, 246, 108665. doi:https://doi.org/10.1016/j.knosys.2022.108665.
  32. Self-supervised consensus representation learning for attributed graph. In Proceedings of the 29th ACM International Conference on Multimedia (pp. 2654–2662). doi:https://doi.org/10.1145/3474085.3475416.
  33. Towards deeper graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 338–348). doi:https://doi.org/10.1145/3394486.3403076.
  34. Item relationship graph neural networks for e-commerce. IEEE Transactions on Neural Networks and Learning Systems, 33, 4785–4799. doi:https://doi.org/10.1109/TNNLS.2021.3060872.
  35. Visualizing data using t-sne. Journal of machine learning research, 9.
  36. Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Networks, 16, 555–559. doi:https://doi.org/10.1016/S0893-6080(03)00115-1.
  37. Co-embedding attributed networks. In Proceedings of the twelfth ACM international conference on web search and data mining (pp. 393–401). doi:https://doi.org/10.1145/3289600.3291015.
  38. Graph neural networks exponentially lose expressive power for node classification. In International Conference on Learning Representations.
  39. Netprobe: a fast and scalable system for fraud detection in online auction networks. In Proceedings of the 16th international conference on World Wide Web (pp. 201–210). doi:https://doi.org/10.1145/1242572.1242600.
  40. Geom-gcn: Geometric graph convolutional networks. In International Conference on Learning Representations.
  41. Dropedge: Towards deep graph convolutional networks on node classification. In International Conference on Learning Representations.
  42. Bi-clkt: Bi-graph contrastive learning based knowledge tracing. Knowledge-Based Systems, 241, 108274. doi:https://doi.org/10.1016/j.knosys.2022.108274.
  43. Jkt: A joint graph convolutional network based deep knowledge tracing. Information Sciences, 580, 510–523. doi:https://doi.org/10.1016/j.ins.2021.08.100.
  44. Adaptive spatio-temporal graph neural network for traffic forecasting. Knowledge-Based Systems, 242, 108199. doi:https://doi.org/10.1016/j.knosys.2022.108199.
  45. Tang, Q. (2021). Ultraopt : Distributed asynchronous hyperparameter optimization better than hyperopt. doi:https://doi.org/10.5281/zenodo.4430148.
  46. Attention is all you need. In Advances in Neural Information Processing Systems. volume 30. URL: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
  47. Graph attention networks. In International Conference on Learning Representations.
  48. Graphmix: Improved training of gnns for semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 35. doi:https://doi.org/10.1609/aaai.v35i11.17203.
  49. A unified weakly supervised framework for community detection and semantic matching. In Advances in Knowledge Discovery and Data Mining (pp. 218–230). Springer International Publishing.
  50. Heterogeneous graph attention network. In The World Wide Web Conference (pp. 2022–2032). doi:https://doi.org/10.1145/3308558.3313562.
  51. Am-gcn: Adaptive multi-channel graph convolutional networks. In Proceedings of the 26th ACM SIGKDD International conference on knowledge discovery & data mining (pp. 1243–1253). doi:https://doi.org/10.1145/3394486.3403177.
  52. Simplifying graph convolutional networks. In International conference on machine learning (pp. 6861–6871). PMLR.
  53. Dual-view hypergraph neural networks for attributed graph learning. Knowledge-Based Systems, 227, 107185. doi:https://doi.org/10.1016/j.knosys.2021.107185.
  54. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32, 4–24. doi:https://doi.org/10.1109/TNNLS.2020.2978386.
  55. Graph convolutional networks with multi-level coarsening for graph classification. Knowledge-Based Systems, 194, 105578. doi:https://doi.org/10.1016/j.knosys.2020.105578.
  56. Adversarial incomplete multi-view clustering. In International Joint Conferences on Artificial Intelligence Organization (pp. 3933–3939). doi:https://doi.org/10.24963/ijcai.2019/546.
  57. How powerful are graph neural networks? In International Conference on Learning Representations.
  58. Representation learning on graphs with jumping knowledge networks. In Proceedings of the 35th International Conference on Machine Learning (pp. 5453–5462). PMLR. URL: https://proceedings.mlr.press/v80/xu18c.html.
  59. Semi-supervised classification via full-graph attention neural networks. Neurocomputing, 476, 63–74. doi:https://doi.org/10.1016/j.neucom.2021.12.077.
  60. Graph transformer networks. In Advances in Neural Information Processing Systems (pp. 11983–11993). Curran Associates, Inc. volume 32. URL: https://proceedings.neurips.cc/paper/2019/file/9d63484abb477c97640154d40595a3bb-Paper.pdf.
  61. A feature-importance-aware and robust aggregator for gcn. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (pp. 1813–1822). doi:https://doi.org/10.1145/3340531.3411983.
  62. Heterogeneous graph structure learning for graph neural networks. In 35th AAAI Conference on Artificial Intelligence. doi:https://doi.org/10.1609/aaai.v35i5.16600.
  63. A weighted gcn with logical adjacency matrix for relation extraction. In ECAI 2020 (pp. 2314–2321). IOS Press. doi:https://doi.org/10.3233/FAIA200360.
  64. Beyond homophily in graph neural networks: Current limitations and effective designs, . 33, 7793–7804. URL: https://proceedings.neurips.cc/paper/2020/file/58ae23d878a47004366189884c2f8440-Paper.pdf.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Li Zhou (216 papers)
  2. Wenyu Chen (49 papers)
  3. Dingyi Zeng (8 papers)
  4. Shaohuan Cheng (6 papers)
  5. Wanlong Liu (13 papers)
  6. Malu Zhang (43 papers)
  7. Hong Qu (13 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now