LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models (2310.03903v2)

Abstract: The emergent reasoning and Theory of Mind (ToM) abilities demonstrated by LLMs make them promising candidates for developing coordination agents. In this study, we introduce a new LLM-Coordination Benchmark aimed at a detailed analysis of LLMs within the context of Pure Coordination Games, where participating agents need to cooperate for the most gain. This benchmark evaluates LLMs through two distinct tasks: (1) \emph{Agentic Coordination}, where LLMs act as proactive participants for cooperation in 4 pure coordination games; (2) \emph{Coordination Question Answering (QA)}, where LLMs are prompted to answer 198 multiple-choice questions from the 4 games for evaluation of three key reasoning abilities: Environment Comprehension, ToM Reasoning, and Joint Planning. Furthermore, to enable LLMs for multi-agent coordination, we introduce a Cognitive Architecture for Coordination (CAC) framework that can easily integrate different LLMs as plug-and-play modules for pure coordination games. Our findings indicate that LLM agents equipped with GPT-4-turbo achieve comparable performance to state-of-the-art reinforcement learning methods in games that require commonsense actions based on the environment. Besides, zero-shot coordination experiments reveal that, unlike RL methods, LLM agents are robust to new unseen partners. However, results on Coordination QA show a large room for improvement in the Theory of Mind reasoning and joint planning abilities of LLMs. The analysis also sheds light on how the ability of LLMs to understand their environment and their partner's beliefs and intentions plays a part in their ability to plan for coordination. Our code is available at \url{https://github.com/eric-ai-lab/LLM_coordination}.

Insights into Multi-Agent Coordination with LLMs

This paper investigates the potential of LLMs in facilitating multi-agent coordination, a critical component of collaborative artificial intelligence applications. The authors present an LLM-Coordinated Framework (LLM-Co) as a method for enabling LLMs to engage effectively in coordination games. They explore five pertinent aspects of coordination: Theory of Mind (ToM), Situated Reasoning, Sustained Coordination, Robustness to Partners, and Explicit Assistance. This paper evaluates LLMs in various game environments, highlighting their strengths and limitations.

LLM-Coordination Framework

The LLM-Co Framework is designed to enable LLMs, like GPT-4, to interact and perform tasks in dynamic multi-agent game environments. It provides a structured approach by translating game details and rules into a textual format that LLMs can process effectively. The framework supports continuous gameplay across environments by helping LLMs infer actionable steps based on the current game state and the feasible actions available.

Game Environments and Evaluations

The evaluations were conducted in three game environments: Collab Escape, Collab Capture, and Overcooked-AI. Each environment presents unique challenges requiring agents to display Theory of Mind, sustained coordination over extended tasks, and the ability to assist explicitly.

1. Theory of Mind and Situated Reasoning: The paper introduced an LLM-ToM-Reasoning Test Set to measure the ToM and situated reasoning capabilities of LLMs. It was observed that GPT-4 outperforms others, approaching near-human reasoning levels, demonstrating its capacity to accurately predict partner intentions.
2. Sustained Coordination: The LLM-Co agents, particularly those using GPT-4, were capable of sustained coordination, outperforming existing RL-based methods in coordination-heavy tasks without pre-training or task-specific fine-tuning.
3. Robustness to Partners: LLM-Co agents were evaluated against varied partner types, including RL baselines trained with human data. Results show that they adaptively align with partner behavior without compromising coordination efficiency.
4. Explicit Assistance: The paper explored scenarios requiring proactive help to enhance joint task completion effectiveness. They introduced specific Overcooked-AI layouts that require explicit assistance, demonstrating the adaptability of LLM-Co agents to these requirements with appropriate directive prompts.

Implications and Future Developments

The positive outcomes from this research indicate a promising direction for using LLMs in collaborative AI agents. They can process complex instructions, adapt to unforeseen partner actions, and execute long-term plans, making them suitable for real-world multi-agent tasks. Future work will likely explore the scalability of such frameworks across diversified agents and environments, potentially integrating real-world variables and constraints.

Conclusion

This paper underscores the utility of LLMs, principally GPT-4, in multi-agent coordination. By developing structured frameworks like LLM-Co and evaluating them against comprehensive scenarios, the research highlights the emergent reasoning capabilities of LLMs in collaborative tasks. These findings lay the groundwork for LLMs to serve as reliable agents in both virtual and real-world applications requiring sophisticated coordination.