GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaboration (2410.18032v4)

Abstract: Graphs are widely used for modeling relational data in real-world scenarios, such as social networks and urban computing. Existing LLM-based graph analysis approaches either integrate graph neural networks (GNNs) for specific machine learning tasks, limiting their transferability, or rely solely on LLMs' internal reasoning ability, resulting in suboptimal performance. To address these limitations, we take advantage of recent advances in LLM-based agents, which have shown capabilities of utilizing external knowledge or tools for problem solving. By simulating human problem-solving strategies such as analogy and collaboration, we propose a multi-agent system based on LLMs named GraphTeam, for graph analysis. GraphTeam consists of five LLM-based agents from three modules, and the agents with different specialities can collaborate with each other to address complex problems. Specifically, (1) input-output normalization module: the question agent extracts and refines four key arguments from the original question, facilitating the problem understanding, and the answer agent organizes the results to meet the output requirement; (2) external knowledge retrieval module: we first build a knowledge base consisting of relevant documentation and experience information, and then the search agent retrieves the most relevant entries for each question. (3) problem-solving module: given the retrieved information from search agent, the coding agent uses established algorithms via programming to generate solutions, and in case the coding agent does not work, the reasoning agent will directly compute the results without programming. Extensive experiments on six graph analysis benchmarks demonstrate that GraphTeam achieves state-of-the-art performance with an average 25.85% improvement over the best baseline in terms of accuracy. The code and data are available at https://github.com/BUPT-GAMMA/GraphTeam.

• The paper introduces GraphTeam, a multi-agent framework that enhances graph analysis by leveraging large language models for structured collaborative problem solving.
• The framework employs five specialized agents across three modules to normalize inputs, retrieve external knowledge, and execute problem-solving via coding and reasoning.
• Empirical evaluations across six benchmarks demonstrate robust performance improvements and scalability, marking a significant advancement in AI-driven graph analytics.

GraphTeam: Facilitating LLM-based Graph Analysis via Multi-Agent Collaboration

This paper presents GraphTeam, a multi-agent system designed to enhance graph analysis by leveraging the capabilities of LLMs. Through a structured approach that mimics human collaborative problem-solving, GraphTeam achieves notable advancements in the application of LLMs to graph-related tasks, addressing significant limitations found in prior methodologies.

Context and Motivation

Graphs serve as essential tools for modeling relational data across numerous domains such as social networks and urban computing. Current approaches in LLM-based graph analysis often combine LLMs with Graph Neural Networks (GNNs) to handle specific machine learning tasks or rely solely on the inherent reasoning abilities of LLMs. However, these strategies restrict generalizability and performance. The introduction of multi-agent systems, where agents with specialized capabilities collaborate, provides a promising avenue for overcoming these constraints.

System Framework

GraphTeam comprises five agents organized into three modules:

1. Input-Output Normalization:
   • Question Agent: Extracts and refines key arguments from queries, enhancing problem comprehension and formulation.
   • Answer Agent: Ensures that results conform to specified formats, improving output consistency and validity.
2. External Knowledge Retrieval:
   • Search Agent: Taps into a knowledge base constructed from documentation and previous problem-solving experiences, aiding in the retrieval of contextually relevant information.
3. Problem Solving:
   • Coding Agent: Generates Python code to solve problems using retrieved data and experiences. Incorporates retry mechanisms for code refinement.
   • Reasoning Agent: Provides problem solutions through logical inference when coding is insufficient.

Empirical Evaluation

The paper provides thorough empirical evidence of GraphTeam's effectiveness across six benchmarks, including Talk like a Graph, GraphWiz, and a newly introduced GNN-focused benchmark, AutoGL. Results indicate:

• GraphTeam surpasses previous state-of-the-art methods, achieving an average accuracy improvement of 25.85%.
• Robust performance is maintained across various graph analysis tasks, particularly in challenging scenarios where baseline LLMs falter.
• Substantial gains are reported on complex output formats, demonstrating GraphTeam’s adaptability to diverse problem requirements.

Implications and Future Developments

GraphTeam exemplifies the potential of multi-agent systems in pushing the boundaries of LLM applications in graph analysis. Its architecture allows for improved problem understanding and solution accuracy through targeted agent collaboration and external knowledge utilization.

Practically, the system offers a scalable approach suitable for diverse graph-oriented domains. The integration of dynamic knowledge retrieval mechanisms enhances the adaptability and transferability of LLM-based solutions.

Theoretically, the paper sets a precedent for exploring collaborative multi-agent frameworks further, raising pivotal questions regarding the optimization of knowledge bases and agent functionality.

Future advancements may focus on refining agent specialization further, enhancing reasoning-depth capabilities, and expanding the framework to incorporate other data structures or domains, potentially including more sophisticated neural architectures and real-time data processing.

Overall, GraphTeam represents a significant stride in enhancing the interoperability and effectiveness of LLMs for complex graph analysis tasks, marking a notable progression in the field of AI-driven graph analytics.