Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning (2309.01538v3)

Published 4 Sep 2023 in cs.AI and cs.CL

Abstract: Logical rules are essential for uncovering the logical connections between relations, which could improve reasoning performance and provide interpretable results on knowledge graphs (KGs). Although there have been many efforts to mine meaningful logical rules over KGs, existing methods suffer from computationally intensive searches over the rule space and a lack of scalability for large-scale KGs. Besides, they often ignore the semantics of relations which is crucial for uncovering logical connections. Recently, LLMs have shown impressive performance in the field of natural language processing and various applications, owing to their emergent ability and generalizability. In this paper, we propose a novel framework, ChatRule, unleashing the power of LLMs for mining logical rules over knowledge graphs. Specifically, the framework is initiated with an LLM-based rule generator, leveraging both the semantic and structural information of KGs to prompt LLMs to generate logical rules. To refine the generated rules, a rule ranking module estimates the rule quality by incorporating facts from existing KGs. Last, the ranked rules can be used to conduct reasoning over KGs. ChatRule is evaluated on four large-scale KGs, w.r.t. different rule quality metrics and downstream tasks, showing the effectiveness and scalability of our method.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Beamqa: Multi-hop knowledge graph question answering with sequence-to-sequence prediction and beam search. In SIGIR, pages 781–790, 2023.
  2. Jon Barwise. An introduction to first-order logic. In Studies in Logic and the Foundations of Mathematics, volume 90, pages 5–46. Elsevier, 1977.
  3. Language models are few-shot learners. NeurIPS, 33:1877–1901, 2020.
  4. Ontological pathfinding. In SIGMOD, pages 835–846, 2016.
  5. Neural compositional rule learning for knowledge graph reasoning. In ICLR, 2022.
  6. Rlogic: Recursive logical rule learning from knowledge graphs. In KDD, pages 179–189, 2022.
  7. Convolutional 2d knowledge graph embeddings. In AAAI, volume 32, 2018.
  8. Amie: association rule mining under incomplete evidence in ontological knowledge bases. In WWW, pages 413–422, 2013.
  9. Geoffrey E Hinton et al. Learning distributed representations of concepts. In Proceedings of the eighth annual conference of the cognitive science society, volume 1, page 12. Amherst, MA, 1986.
  10. Rule-aware reinforcement learning for knowledge graph reasoning. In ACL, pages 4687–4692, 2021.
  11. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023.
  12. Mistral 7b, 2023.
  13. Statistical predicate invention. In ICML, pages 433–440, 2007.
  14. Ni Lao and William W Cohen. Relational retrieval using a combination of path-constrained random walks. Machine learning, 81:53–67, 2010.
  15. Neural multi-hop reasoning with logical rules on biomedical knowledge graphs. In ESWC, pages 375–391. Springer, 2021.
  16. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In AAAI, volume 36, pages 4120–4127, 2022.
  17. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172, 2023.
  18. R5: Rule discovery with reinforced and recurrent relational reasoning. In ICLR, 2021.
  19. Reasoning on graphs: Faithful and interpretable large language model reasoning. arXiv preprint arXiv:2310.01061, 2023.
  20. Scalable rule learning via learning representation. In IJCAI, pages 2149–2155, 2018.
  21. OpenAI. Gpt-4 technical report, 2023.
  22. Unifying large language models and knowledge graphs: A roadmap. arXiv preprint arXiv:2306.08302, 2023.
  23. Probabilistic logic neural networks for reasoning. NeurIPS, 32, 2019.
  24. Rnnlogic: Learning logic rules for reasoning on knowledge graphs. In ICLR, 2020.
  25. Drum: End-to-end differentiable rule mining on knowledge graphs. NeurIPS, 32, 2019.
  26. Sound and complete forward and backward chainings of graph rules. In International Conference on Conceptual Structures, pages 248–262, 1996.
  27. Yago: a core of semantic knowledge. In WWW, pages 697–706, 2007.
  28. Evaluation of chatgpt as a question answering system for answering complex questions. arXiv preprint arXiv:2303.07992, 2023.
  29. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
  30. Explainable reasoning over knowledge graphs for recommendation. In AAAI, volume 33, pages 5329–5336, 2019.
  31. Finetuned language models are zero-shot learners. In ICLR, 2021.
  32. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 35:24824–24837, 2022.
  33. Ruleformer: Context-aware rule mining over knowledge graph. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2551–2560, 2022.
  34. Yuan Yang and Le Song. Learn to explain efficiently via neural logic inductive learning. In International Conference on Learning Representations, 2020.
  35. Differentiable learning of logical rules for knowledge base reasoning. NeurIPS, 30, 2017.
  36. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.
  37. Iteratively questioning and answering for interpretable legal judgment prediction. In AAAI, volume 34, pages 1250–1257, 2020.
  38. Evaluating commonsense in pre-trained language models. In AAAI, volume 34, pages 9733–9740, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Linhao Luo (31 papers)
  2. Jiaxin Ju (6 papers)
  3. Bo Xiong (84 papers)
  4. Yuan-Fang Li (90 papers)
  5. Gholamreza Haffari (141 papers)
  6. Shirui Pan (198 papers)
Citations (19)
View on Semantic Scholar