Embodied LLM Agents Learn to Cooperate in Organized Teams

Abstract: LLMs have emerged as integral tools for reasoning, planning, and decision-making, drawing upon their extensive world knowledge and proficiency in language-related tasks. LLMs thus hold tremendous potential for natural language interaction within multi-agent systems to foster cooperation. However, LLM agents tend to over-report and comply with any instruction, which may result in information redundancy and confusion in multi-agent cooperation. Inspired by human organizations, this paper introduces a framework that imposes prompt-based organization structures on LLM agents to mitigate these problems. Through a series of experiments with embodied LLM agents and human-agent collaboration, our results highlight the impact of designated leadership on team efficiency, shedding light on the leadership qualities displayed by LLM agents and their spontaneous cooperative behaviors. Further, we harness the potential of LLMs to propose enhanced organizational prompts, via a Criticize-Reflect process, resulting in novel organization structures that reduce communication costs and enhance team efficiency.

• The paper introduces a prompt-based framework that organizes LLM agents into structured teams to overcome communication inefficiencies using hierarchical roles.
• The methodology employs a Criticize-Reflect optimization to iteratively refine communication prompts, leading to novel dual-leader and dynamic team configurations.
• Experimental results show that teams with designated leaders improve task efficiency and proactive coordination, albeit with higher communication overhead.

Embodied LLM Agents Learn to Cooperate in Organized Teams

This paper investigates the potential of organizing LLM agents into structured teams to enhance cooperative behaviors and overall team efficiency. Drawing parallels from human organizations, the authors present a framework designed to optimize organizational prompts, facilitating improved coordination and communication among LLM agents.

Introduction

Traditional multi-agent systems have relied on pre-specified communication strategies such as gradient exchange and state-sharing. The advent of LLMs provides an opportunity for more flexible, nuanced exchanges using natural language. However, LLM agents often exhibit limitations such as redundancy in information sharing and compliance that impede efficient cooperation. By imposing structured organizational prompts, these agents can potentially overcome communication pitfalls, mirroring human-like organizational dynamics.

Despite the robust capabilities of LLMs in individual tasks, their integration within multi-agent systems is fraught with challenges due to these communication inefficiencies. Addressing this, the paper introduces a prompt-based framework that encourages organizational structures akin to human leadership roles to improve efficiency.

Methodology

Multi-Agent Cooperation Architecture

The proposed architecture for organizing LLM agents involves several key modules: configuration, perception, memory, and execution. These modules work in unison to facilitate organized communication and decision-making amongst multiple agents (Figure 1).

Figure 1: Multi-LLM-agent architecture highlighting communication and action phases.

Communication within this framework alternates between a dedicated communication phase and an action phase, allowing agents to broadcast messages, select recipients, or remain silent based on organizational prompts. These structured interactions enable agents to engage in richer and more organized exchanges.

Criticize-Reflect Optimization Framework

An innovative Criticize-Reflect framework is employed to optimize organizational prompts using dual LLM architectures. The Critic evaluates agent performance based on previous dialogues, while the Coordinator generates new prompts using reflection on team dynamics and interaction costs (Figure 2).

Figure 2: Criticize-Reflect architecture for improving organizational structure.

This iterative process allows for the development of novel organizational structures that potentially improve team communication and task efficiency.

Experimental Results

Impact of Leadership

Appointing a designated leader markedly enhances team performance by reducing redundant communication and chaotic decision-making observed in disorganized setups (Figure 3). Leadership election, though beneficial for efficiency, incurs significant communication costs due to frequent message exchanges required for dynamic leadership roles.

Figure 3: Example of disorganized communication without leadership.

Further comparisons demonstrate that Human-led teams outperform AI-led teams in both task completion time and communication efficacy, underscoring the current limitations of AI leadership capabilities (Figure 4).

Figure 4: Organized teams with designated leaders demonstrate higher efficiency.

Cooperative Behavior Emergence

Even without explicit prompts, agents exhibit various cooperative behaviors such as information sharing and leadership-assisted tasks. However, structured hierarchies bolster these behaviors more effectively, fostering proactive guidance requests and enhanced task allocation efficiency (Figure 5).

Figure 5: Examples of cooperative behavior in organized LLM teams.

Novel Organizational Structures

The Criticize-Reflect framework fosters the emergence of complex organizational prompts that yield unique team structures like dual-leader and dynamic configurations (Figure 6).

Figure 6: Communication patterns illustrating novel team structures proposed by LLM reflection.

These innovative structures not only improve communication efficiency but also demonstrate adaptability across varied tasks, affirming their potential for broader applicability in different AI domains.

Conclusion

The exploration of organized LLM teams sheds light on potential improvements in multi-agent cooperation through structured communication frameworks. Demonstrated efficiency gains underscore the feasibility of applying human-like organizational principles to AI agent frameworks. Future research is poised to extend these findings across diverse environments and incorporate human evaluation to further validate these organizational strategies.

In summary, the integration of organizational architecture with LLM agents offers a promising avenue for enhancing multi-agent coordination and realizing more autonomous AI systems in practical applications.