Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding Expressivity of GNN in Rule Learning (2303.12306v2)

Published 22 Mar 2023 in cs.LG and cs.AI

Abstract: Rule learning is critical to improving knowledge graph (KG) reasoning due to their ability to provide logical and interpretable explanations. Recently, Graph Neural Networks (GNNs) with tail entity scoring achieve the state-of-the-art performance on KG reasoning. However, the theoretical understandings for these GNNs are either lacking or focusing on single-relational graphs, leaving what the kind of rules these GNNs can learn an open problem. We propose to fill the above gap in this paper. Specifically, GNNs with tail entity scoring are unified into a common framework. Then, we analyze their expressivity by formally describing the rule structures they can learn and theoretically demonstrating their superiority. These results further inspire us to propose a novel labeling strategy to learn more rules in KG reasoning. Experimental results are consistent with our theoretical findings and verify the effectiveness of our proposed method. The code is publicly available at https://github.com/LARS-research/Rule-learning-expressivity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. The surprising power of graph neural networks with random node initialization. In International Joint Conference on Artificial Intelligence, 2021.
  2. Complex query answering with neural link predictors. In International Conference on Learning Representations, 2021.
  3. The logical expressiveness of graph neural networks. In International Conference on Learning Representations, 2020.
  4. Weisfeiler and leman go relational. In The First Learning on Graphs Conference, 2022. URL https://openreview.net/forum?id=wY_IYhh6pqj.
  5. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018.
  6. Translating embeddings for modeling multi-relational data. Advances in Neural Information Processing Systems, 2013.
  7. Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences. In International World Wide Web Conference, 2019.
  8. On the correspondence between monotonic max-sum gnns and datalog. arXiv preprint arXiv:2305.18015, 2023.
  9. Maarten De Rijke. A note on graded modal logic. Studia Logica, 64(2):271–283, 2000.
  10. Convolutional 2D knowledge graph embeddings. In AAAI conference on Artificial Intelligence, 2017.
  11. Neural message passing for quantum chemistry. In International Conference on Machine Learning, 2017.
  12. Embedding logical queries on knowledge graphs. Advances in neural information processing systems, 31, 2018.
  13. Geoffrey E Hinton et al. Learning distributed representations of concepts. In Annual Conference of the Cognitive Science Society, 1986.
  14. A theory of link prediction via relational weisfeiler-leman. arXiv preprint arXiv:2302.02209, 2023.
  15. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33(2):494–514, 2021.
  16. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2016.
  17. Statistical predicate invention. In International Conference on Machine Learning, 2007.
  18. Inductive logic programming. In WLP, pp.  146–160. Springer, 1994.
  19. Tune: A research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118, 2018.
  20. Discovering protein drug targets using knowledge graph embeddings. Bioinformatics, 2019.
  21. Weisfeiler and leman go neural: Higher-order graph neural networks. In AAAI conference on Artificial Intelligence, 2019.
  22. Weisfeiler and leman go sparse: Towards scalable higher-order graph embeddings. Advances in Neural Information Processing Systems, 2020.
  23. Martin Otto. Graded modal logic and counting bisimulation. arXiv preprint arXiv:1910.00039, 2019.
  24. Query2box: Reasoning over knowledge graphs in vector space using box embeddings. In International Conference on Learning Representations, 2019.
  25. Drum: End-to-end differentiable rule mining on knowledge graphs. Advances in Neural Information Processing Systems, 2019.
  26. Random features strengthen graph neural networks. In SIAM International Conference on Data Mining, 2021.
  27. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, 2018.
  28. On the equivalence between positional node embeddings and structural graph representations. ICLR, 2020.
  29. Rotate: Knowledge graph embedding by relational rotation in complex space. In International Conference on Learning Representations, 2019.
  30. Explainable gnn-based models over knowledge graphs. 2022.
  31. Inductive relation prediction by subgraph reasoning. In International Conference on Machine Learning, 2020.
  32. Observed versus latent features for knowledge base and text inference. In Alexandre Allauzen, Edward Grefenstette, Karl Moritz Hermann, Hugo Larochelle, and Scott Wen-tau Yih (eds.), Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp. 57–66, Beijing, China, July 2015. Association for Computational Linguistics. doi: 10.18653/v1/W15-4007. URL https://aclanthology.org/W15-4007.
  33. Complex embeddings for simple link prediction. In International Conference on Machine Learning, 2016.
  34. Composition-based multi-relational graph convolutional networks. In International Conference on Learning Representations, 2020.
  35. How powerful are graph neural networks? In International Conference on Learning Representations, 2019.
  36. Differentiable learning of logical rules for knowledge base reasoning. Advances in Neural Information Processing Systems, 2017.
  37. Identity-aware graph neural networks. In AAAI Conference on Artificial Intelligence, 2021.
  38. Labeling trick: A theory of using graph neural networks for multi-node representation learning. Advances in Neural Information Processing Systems, 2021.
  39. Quaternion knowledge graph embeddings. Advances in Neural Information Processing Systems, 32, 2019.
  40. Knowledge graph reasoning with relational digraph. In International World Wide Web Conference, 2022.
  41. Adaprop: Learning adaptive propagation for graph neural network based knowledge graph reasoning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  3446–3457, 2023.
  42. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems, 2021.
  43. A*net: A scalable path-based reasoning approach for knowledge graphs. arXiv preprint arXiv:2206.04798, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Haiquan Qiu (4 papers)
  2. Yongqi Zhang (33 papers)
  3. Yong Li (628 papers)
  4. Quanming Yao (102 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now