AGENTiGraph: An Interactive Knowledge Graph Platform for LLM-based Chatbots Utilizing Private Data

Abstract: LLMs~(LLMs) have demonstrated capabilities across various applications but face challenges such as hallucination, limited reasoning abilities, and factual inconsistencies, especially when tackling complex, domain-specific tasks like question answering~(QA). While Knowledge Graphs~(KGs) have been shown to help mitigate these issues, research on the integration of LLMs with background KGs remains limited. In particular, user accessibility and the flexibility of the underlying KG have not been thoroughly explored. We introduce AGENTiGraph (Adaptive Generative ENgine for Task-based Interaction and Graphical Representation), a platform for knowledge management through natural language interaction. It integrates knowledge extraction, integration, and real-time visualization. AGENTiGraph employs a multi-agent architecture to dynamically interpret user intents, manage tasks, and integrate new knowledge, ensuring adaptability to evolving user requirements and data contexts. Our approach demonstrates superior performance in knowledge graph interactions, particularly for complex domain-specific tasks. Experimental results on a dataset of 3,500 test cases show AGENTiGraph significantly outperforms state-of-the-art zero-shot baselines, achieving 95.12\% accuracy in task classification and 90.45\% success rate in task execution. User studies corroborate its effectiveness in real-world scenarios. To showcase versatility, we extended AGENTiGraph to legislation and healthcare domains, constructing specialized KGs capable of answering complex queries in legal and medical contexts.

• The paper introduces a multi-agent system that integrates LLMs and KGs to improve reasoning and reduce hallucinations.
• It employs Few-Shot Learning, Chain-of-Thought, and ReAct frameworks for efficient query processing and task decomposition.
• Evaluation shows 95.12% task classification accuracy and 90.45% task execution success, validating its robust application in complex domains.

AGENTiGraph: An Interactive Knowledge Graph Platform for LLM-based Chatbots Utilizing Private Data

AGENTiGraph is proposed to address inherent challenges in the integration of LLMs with Knowledge Graphs (KGs) for domain-specific applications, notably in question answering (QA). This platform aims to mitigate issues such as hallucination and limited reasoning abilities of LLMs by employing KGs for factual consistency and enhancing reasoning capabilities. It introduces an intelligent multi-agent architecture designed for adaptive interaction and management of KGs, optimizing user interaction and ensuring dynamic knowledge integration.

Architecture Design of AGENTiGraph

AGENTiGraph leverages a novel multi-agent system for intelligent interaction with KGs. Each agent specializes in specific tasks such as user intent interpretation, key concept extraction, task planning, KG interaction, reasoning, dynamic knowledge integration, and response generation.

Multi-Agent System Components

1. User Intent Interpretation: Utilizes Few-Shot Learning and CoT reasoning to accurately interpret diverse query types and determine user intents.
2. Key Concept Extraction: Employs Named Entity Recognition and Relation Extraction to identify entities and map them to KGs using semantic similarity with BERT-derived vector representations.
3. Task Planning: Decomposes user intent into executable tasks employing CoT reasoning to model task dependencies.
4. Knowledge Graph Interaction: Converts high-level tasks into executable graph queries using Few-Shot Learning and ReAct framework.
5. Reasoning: Enhances raw query results with logical inference to bridge structured knowledge and natural language understanding.
6. Dynamic Knowledge Integration: Supports continuous knowledge extraction, ensuring KGs remain up-to-date through interactions with the Neo4j database.
7. Response Generation: Synthesizes processed information into structured and contextually relevant responses using CoT, ReAct, and Few-Shot Learning.

   Figure 1: AGENTiGraph Framework: A multi-agent system for intelligent KG interaction and management.

System Demonstration and Interaction Paradigm

AGENTiGraph offers a dual-mode interface combining conversational AI with interactive knowledge exploration. Users interact through natural language in a user-friendly environment that supports seamless KG management and exploration.

Key Features

• Conversational AI Mode: Facilitates dynamic QA through KG traversal, enhancing answer accuracy and coherence.
• Exploration Mode: Provides detailed KG visualization, enabling intuitive navigation through conceptual hierarchies and relationships.

  Figure 2: AGENTiGraph's Dual-Mode Interface: Conversational AI with Interactive Knowledge Exploration.

Evaluation and Performance

AGENTiGraph was evaluated against state-of-the-art zero-shot baselines using various LLMs. The integrated framework demonstrated impressive improvements, achieving 95.12% accuracy in task classification and a 90.45% success rate in task execution. Its capability extends to diverse domains, where it efficiently constructs specialized KGs in areas such as legislation and healthcare, enabling complex queries in these contexts.

Future Developments

AGENTiGraph sets a foundation for further enhancements in multi-hop reasoning, optimizing response conciseness and completeness. Future developments will focus on continuous learning from user interactions and expanding its applicability to additional domains requiring customized KG solutions.

Conclusion

AGENTiGraph significantly advances knowledge graph interaction capabilities, bridging LLMs and structured knowledge for robust domain-specific applications. Its design ensures adaptable, efficient, and user-friendly interactions, demonstrating potential across various complex tasks and real-world scenarios. Future research and development will drive enhancements in reasoning capabilities and expand its applicability to more domains, further refining its role in intelligent knowledge management and exploration systems.