Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning (2405.01649v3)

Published 2 May 2024 in cs.CL

Abstract: Answering complex queries over incomplete knowledge graphs (KGs) is a challenging job. Most previous works have focused on learning entity/relation embeddings and simulating first-order logic operators with various neural networks. However, they are bottlenecked by the inability to share world knowledge to improve logical reasoning, thus resulting in suboptimal performance. In this paper, we propose a complex reasoning schema over KG upon LLMs, containing a curriculum-based logical-aware instruction tuning framework, named LACT. Specifically, we augment the arbitrary first-order logical queries via binary tree decomposition, to stimulate the reasoning capability of LLMs. To address the difficulty gap among different types of complex queries, we design a simple and flexible logic-aware curriculum learning framework. Experiments across widely used datasets demonstrate that LACT has substantial improvements~(brings an average +5.5% MRR score) over advanced methods, achieving the new state-of-the-art. Our code and model will be released at GitHub and huggingface soon.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. 2020. Complex query answering with neural link predictors. In International Conference on Learning Representations.
  3. 2021. Modeling heterogeneous hierarchies with relation-specific hyperbolic cones. In Advances in Neural Information Processing Systems.
  4. 2023. Answering complex logical queries on knowledge graphs via query computation tree optimization. In International Conference on Machine Learning.
  5. 2009. Curriculum learning. In International Conference on Machine Learning.
  6. 2008. Freebase. In ACM SIGMOD International Conference on Management of Data.
  7. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems.
  8. 2022. Fuzzy logic based logical query answering on knowledge graphs. In the AAAI Conference on Artificial Intelligence.
  9. 2023. Complex logical reasoning over knowledge graphs using large language models. arXiv preprint arXiv:2305.01157.
  10. 2021. Self-supervised hyperboloid representations from logical queries over knowledge graphs. In the Web Conference 2021.
  11. 2007. Efficient query evaluation on probabilistic databases. The VLDB Journal.
  12. 2021. Progressive multi-granularity training for non-autoregressive translation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2797–2803.
  13. 2023. Recurrent graph encoder for syntax-aware neural machine translation. International Journal of Machine Learning and Cybernetics 14(4):1053–1062.
  14. Elman, J. L. 1993. Learning and development in neural networks: the importance of starting small. Cognition.
  15. 2015. Traversing knowledge graphs in vector space. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
  16. 2018. Embedding logical queries on knowledge graphs. In Advances in neural information processing systems.
  17. 2009. Flexible shaping: How learning in small steps helps. Cognition.
  18. 2023. Unified instance and knowledge alignment pretraining for aspect-based sentiment analysis. IEEE/ACM transactions on audio, speech, and language processing 31:2629–2642.
  19. 2024a. Logic query of thoughts: Guiding large language models to answer complex logic queries with knowledge graphs. arXiv preprint arXiv:2404.04264.
  20. 2024b. Let’s learn step by step: Enhancing in-context learning ability with curriculum learning. arXiv preprint arXiv:2402.10738.
  21. 2023. Error analysis prompting enables human-like translation evaluation in large language models: A case study on chatgpt.
  22. 2023. Chatrule: Mining logical rules with large language models for knowledge graph reasoning. arXiv preprint arXiv:2309.01538.
  23. 2024. Reasoning on graphs: Faithful and interpretable large language model reasoning. In International Conference on Learning Representations.
  24. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems.
  25. 2024a. Pomp: Probability-driven meta-graph prompter for llms in low-resource unsupervised neural machine translation. arXiv preprint arXiv:2401.05596.
  26. 2024b. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering.
  27. 2023. Towards making the most of ChatGPT for machine translation. In Bouamor, H.; Pino, J.; and Bali, K., eds., Findings of the Association for Computational Linguistics: EMNLP 2023.
  28. 2020. Beta embeddings for multi-hop logical reasoning in knowledge graphs. In Advances in Neural Information Processing Systems.
  29. 2019. Query2box: Reasoning over knowledge graphs in vector space using box embeddings. In International Conference on Learning Representations.
  30. 2016. Easy questions first? a case study on curriculum learning for question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.
  31. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  32. 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  33. 2023a. Knowledge-tuning large language models with structured medical knowledge bases for reliable response generation in chinese. arXiv preprint arXiv:2309.04175.
  34. 2023b. Query structure modeling for inductive logical reasoning over knowledge graphs. arXiv preprint arXiv:2305.13585.
  35. 2023c. Unifying structure reasoning and language pre-training for complex reasoning tasks. IEEE/ACM Transactions on Audio, Speech, and Language Processing.
  36. 2023d. Gradual syntactic label replacement for language model pre-training. IEEE/ACM Transactions on Audio, Speech, and Language Processing.
  37. 2023e. Aligning large language models with human: A survey. arXiv preprint arXiv:2307.12966.
  38. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
  39. 2020. Curriculum learning for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
  40. 2023. Prediction and calibration: Complex reasoning over knowledge graph with bi-directional directed acyclic graph neural network. In Findings of the Association for Computational Linguistics: ACL 2023.
  41. 2022. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414.
  42. 2022. Knowledge graph reasoning with relational digraph. In Proceedings of the ACM web conference 2022, 912–924.
  43. 2021. Cone: Cone embeddings for multi-hop reasoning over knowledge graphs. In Advances in Neural Information Processing Systems.
  44. 2023a. Autoalign: Fully automatic and effective knowledge graph alignment enabled by large language models. IEEE Transactions on Knowledge and Data Engineering.
  45. 2023b. Making large language models perform better in knowledge graph completion. arXiv preprint arXiv:2310.06671.
  46. 2023c. Siren’s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219.
  47. 2023a. Ke-x: Towards subgraph explanations of knowledge graph embedding based on knowledge information gain. Knowledge-Based Systems 278:110772.
  48. 2023b. A survey of large language models. arXiv preprint arXiv:2303.18223.
  49. 2023a. Knowledge graph augmented network towards multiview representation learning for aspect-based sentiment analysis. IEEE Transactions on knowledge and data engineering.
  50. 2023b. Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint arXiv:2302.10198.
  51. 2021. Self-guided curriculum learning for neural machine translation. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), 206–214.
  52. 2022. Neural-symbolic models for logical queries on knowledge graphs. In International Conference on Machine Learning.
  53. 2023. Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities. arXiv preprint arXiv:2305.13168.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Tianle Xia (1 paper)
  2. Liang Ding (159 papers)
  3. Guojia Wan (3 papers)
  4. Yibing Zhan (73 papers)
  5. Bo Du (264 papers)
  6. Dacheng Tao (829 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets