Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (2307.07697v6)

Abstract: Although LLMs have achieved significant success in various tasks, they often struggle with hallucination problems, especially in scenarios requiring deep and responsible reasoning. These issues could be partially addressed by introducing external knowledge graphs (KG) in LLM reasoning. In this paper, we propose a new LLM-KG integrating paradigm ``$\hbox{LLM}\otimes\hbox{KG}$'' which treats the LLM as an agent to interactively explore related entities and relations on KGs and perform reasoning based on the retrieved knowledge. We further implement this paradigm by introducing a new approach called Think-on-Graph (ToG), in which the LLM agent iteratively executes beam search on KG, discovers the most promising reasoning paths, and returns the most likely reasoning results. We use a number of well-designed experiments to examine and illustrate the following advantages of ToG: 1) compared with LLMs, ToG has better deep reasoning power; 2) ToG has the ability of knowledge traceability and knowledge correctability by leveraging LLMs reasoning and expert feedback; 3) ToG provides a flexible plug-and-play framework for different LLMs, KGs and prompting strategies without any additional training cost; 4) the performance of ToG with small LLM models could exceed large LLM such as GPT-4 in certain scenarios and this reduces the cost of LLM deployment and application. As a training-free method with lower computational cost and better generality, ToG achieves overall SOTA in 6 out of 9 datasets where most previous SOTAs rely on additional training.

Introduction

LLMs are quickly becoming adept at generating coherent and context-appropriate responses across a variety of tasks. Despite their advances, challenges emerge when LLMs are tasked with complex reasoning that demands a deep understanding of factual knowledge. LLMs may generate inaccurate or outdated responses or engage in "hallucination" – providing seemingly confident answers that are unrelated to reality. Moreover, training LLMs can be resource-intensive and does not ensure current knowledge due to the static nature of training datasets.

To enhance the reasoning capabilities of LLMs and keep their knowledge current, integration with external knowledge repositories like knowledge graphs (KGs) presents a promising solution. Prior approaches to integrating LLMs and KGs (denoted LLM⊕KG) have improved LLM performance but are limited by their loose-coupling paradigm, resulting in unexploited KG potential. To address this, the paper introduces a new model, the tight-coupling "LLM ⊗ KG" paradigm, which is realized through an approach called Think-on-Graph (ToG).

Methodology

Think-on-Graph (ToG) capitalizes on LLMs to interactively explore a KB and identify reasoning paths, without additional training. It prompts the LLM to traverse multiple potential reasoning paths on KGs, refining selections iteratively until sufficient information or a set search depth is reached. The selected paths offer a basis for the LLM to perform logical reasoning and yield answers. An extension, ToG-R (Relation-Based Think-on-Graph), focuses on relation chains instead of triples, streamlining reasoning by emphasizing the relevance of relations to the question over the entities themselves.

The process has three key phases:

1. Initialization: Utilizing LLMs to identify initial entities related to the query.
2. Exploration: Directing LLMs to search for and prune relevant relations and entities within the KG.
3. Reasoning: Prompting LLMs to evaluate current reasoning paths for sufficiency in providing an answer. If inadequate, the model loops back through exploration and reasoning until sufficient information is accumulated or a set search boundary is hit.

Advantages and Experimentation

Experiments show several advantages of ToG:

• Enhanced deep reasoning through multi-hop paths.
• Knowledge traceability and correctability due to explicit, editable paths.
• Flexibility in applying various LLMs and KGs.
• Efficiency in updating LLMs with KGs, and improved generality without training costs.

In tests, even when built on smaller LLMs like LLAMA2-70B, ToG's performance matches or surpasses that of larger LLMs like GPT-4 in specific scenarios, suggesting a cost-effective alternative for deploying LLMs.

Conclusion

The Think-on-Graph approach demonstrates state-of-the-art (SOTA) performance across various datasets and tasks, showcasing its potent mix of generality, efficiency, and reasoning ability. Harnessing both structured, editable KGs and the powerful reasoning ability of LLMs, ToG advances the effectiveness of LLMs in knowledge-intensive tasks, offering a compelling solution to the problem of knowledge hallucination. Furthermore, by providing an avenue for knowledge traceability and correctability, ToG aligns with the goal of responsible AI development.