Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RulE: Knowledge Graph Reasoning with Rule Embedding (2210.14905v3)

Published 24 Oct 2022 in cs.AI and cs.LG

Abstract: Knowledge graph (KG) reasoning is an important problem for knowledge graphs. In this paper, we propose a novel and principled framework called \textbf{RulE} (stands for {Rul}e {E}mbedding) to effectively leverage logical rules to enhance KG reasoning. Unlike knowledge graph embedding (KGE) methods, RulE learns rule embeddings from existing triplets and first-order {rules} by jointly representing \textbf{entities}, \textbf{relations} and \textbf{logical rules} in a unified embedding space. Based on the learned rule embeddings, a confidence score can be calculated for each rule, reflecting its consistency with the observed triplets. This allows us to perform logical rule inference in a soft way, thus alleviating the brittleness of logic. On the other hand, RulE injects prior logical rule information into the embedding space, enriching and regularizing the entity/relation embeddings. This makes KGE alone perform better too. RulE is conceptually simple and empirically effective. We conduct extensive experiments to verify each component of RulE. Results on multiple benchmarks reveal that our model outperforms the majority of existing embedding-based and rule-based approaches.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Boxe: A box embedding model for knowledge base completion. Advances in Neural Information Processing Systems, 33:9649–9661.
  2. Tucker: Tensor factorization for knowledge graph completion. arXiv preprint arXiv:1901.09590.
  3. Islam Beltagy and Raymond J Mooney. 2014. Efficient markov logic inference for natural language semantics. In Workshops at the Twenty-Eighth AAAI Conference on Artificial Intelligence.
  4. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD Conference.
  5. Question answering with subgraph embeddings. arXiv preprint arXiv:1406.3676.
  6. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26.
  7. Probabilistic similarity logic. arXiv preprint arXiv:1203.3469.
  8. Liwei Cai and William Yang Wang. 2017. Kbgan: Adversarial learning for knowledge graph embeddings. arXiv preprint arXiv:1711.04071.
  9. Neural compositional rule learning for knowledge graph reasoning. arXiv preprint arXiv:2303.03581.
  10. Tensorlog: Deep learning meets probabilistic dbs. arXiv preprint arXiv:1707.05390.
  11. Principal neighbourhood aggregation for graph nets. Advances in Neural Information Processing Systems, 33:13260–13271.
  12. Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
  13. Improving knowledge graph embedding using simple constraints. arXiv preprint arXiv:1805.02408.
  14. Fast rule mining in ontological knowledge bases with amie++. The VLDB Journal, 24(6):707–730.
  15. Amie: association rule mining under incomplete evidence in ontological knowledge bases. In Proceedings of the 22nd international conference on World Wide Web, pages 413–422.
  16. Compounding geometric operations for knowledge graph completion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6947–6965.
  17. Knowledge graph embedding preserving soft logical regularity. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 425–434.
  18. Jointly embedding knowledge graphs and logical rules. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 192–202.
  19. Knowledge graph embedding with iterative guidance from soft rules. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
  20. Pascal Hitzler. 2022. Neuro-symbolic artificial intelligence: The state of the art.
  21. Patrick Hohenecker and Thomas Lukasiewicz. 2020. Ontology reasoning with deep neural networks. Journal of Artificial Intelligence Research, 68:503–540.
  22. Stanley Kok and Pedro Domingos. 2005. Learning the structure of markov logic networks. In Proceedings of the 22nd international conference on Machine learning, pages 441–448.
  23. Stanley Kok and Pedro Domingos. 2007. Statistical predicate invention. In Proceedings of the 24th international conference on Machine learning, pages 433–440.
  24. Relational retrieval using a combination of path-constrained random walks. Machine learning, 81(1):53–67.
  25. Are message passing neural networks really helpful for knowledge graph completion? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10696–10711.
  26. Yago3: A knowledge base from multilingual wikipedias. In 7th biennial conference on innovative data systems research. CIDR Conference.
  27. Deepproblog: Neural probabilistic logic programming. Advances in Neural Information Processing Systems, 31.
  28. Why a naive way to combine symbolic and latent knowledge base completion works surprisingly well. In 3rd Conference on Automated Knowledge Base Construction.
  29. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41.
  30. Simple augmentations of logical rules for neuro-symbolic knowledge graph completion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 256–269.
  31. Neuro-symbolic program synthesis. In ICLR.
  32. Rnnlogic: Learning logic rules for reasoning on knowledge graphs. arXiv preprint arXiv:2010.04029.
  33. J. Ross Quinlan. 1990. Learning logical definitions from relations. Machine learning, 5(3):239–266.
  34. Hongyu Ren and Jure Leskovec. 2020. Beta embeddings for multi-hop logical reasoning in knowledge graphs. Advances in Neural Information Processing Systems, 33:19716–19726.
  35. Matthew Richardson and Pedro Domingos. 2006. Markov logic networks. Machine learning, 62(1):107–136.
  36. Drum: End-to-end differentiable rule mining on knowledge graphs. Advances in Neural Information Processing Systems, 32.
  37. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593–607. Springer.
  38. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, pages 697–706.
  39. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197.
  40. Inductive relation prediction by subgraph reasoning. In International Conference on Machine Learning, pages 9448–9457. PMLR.
  41. Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality, pages 57–66.
  42. Complex embeddings for simple link prediction. In International conference on machine learning, pages 2071–2080. PMLR.
  43. Composition-based multi-relational graph convolutional networks. arXiv preprint arXiv:1911.03082.
  44. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the 27th ACM international conference on information and knowledge management, pages 417–426.
  45. Knowledge base completion using embeddings and rules. In Twenty-fourth international joint conference on artificial intelligence.
  46. Explicit semantic ranking for academic search via knowledge graph embedding. In Proceedings of the 26th international conference on world wide web, pages 1271–1279.
  47. A semantic loss function for deep learning with symbolic knowledge. In International conference on machine learning.
  48. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575.
  49. Differentiable learning of logical rules for knowledge base reasoning. Advances in neural information processing systems, 30.
  50. Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. Advances in neural information processing systems, 31.
  51. Interstellar: searching recurrent architecture for knowledge graph embedding. Advances in Neural Information Processing Systems, 33:10030–10040.
  52. Efficient probabilistic logic reasoning with graph neural networks. arXiv preprint arXiv:2001.11850.
  53. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems, 34:29476–29490.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Xiaojuan Tang (5 papers)
  2. Song-Chun Zhu (216 papers)
  3. Yitao Liang (53 papers)
  4. Muhan Zhang (89 papers)
Citations (1)
View on Semantic Scholar