Compositional Coordination for Multi-Robot Teams with Large Language Models (2507.16068v2)

Abstract: Multi-robot coordination has traditionally relied on a mission-specific and expert-driven pipeline, where natural language mission descriptions are manually translated by domain experts into mathematical formulation, algorithm design, and executable code. This conventional process is labor-intensive, inaccessible to non-experts, and inflexible to changes in mission requirements. Here, we propose LAN2CB (Language to Collective Behavior), a novel framework that leverages LLMs to streamline and generalize the multi-robot coordination pipeline. LAN2CB transforms natural language (NL) mission descriptions into executable Python code for multi-robot systems through two core modules: (1) Mission Analysis, which parses mission descriptions into behavior trees, and (2) Code Generation, which leverages the behavior tree and a structured knowledge base to generate robot control code. We further introduce a dataset of natural language mission descriptions to support development and benchmarking. Experiments in both simulation and real-world environments demonstrate that LAN2CB enables robust and flexible multi-robot coordination from natural language, significantly reducing manual engineering effort and supporting broad generalization across diverse mission types. Website: https://sites.google.com/view/lan-cb

• The paper introduces LAN2CB, a framework that converts natural language mission descriptions into executable Python code for multi-robot coordination.
• It uses LLMs to decompose tasks and generate standardized control code, significantly reducing manual intervention and error rates.
• Experiments in simulated and real-world missions demonstrate enhanced flexibility and improved success in dynamic multi-robot environments.

Compositional Coordination for Multi-Robot Teams with LLMs

Introduction

The paper "Compositional Coordination for Multi-Robot Teams with LLMs" explores the use of LLMs to streamline multi-robot coordination through a novel framework called Language to Collective Behavior (LAN2CB). The framework simplifies the traditional pipeline by converting natural language mission descriptions into executable Python code, reducing reliance on domain experts and accommodating dynamic mission requirements effortlessly. The paper underscores the challenges faced in multi-robot coordination due to task-specific approaches and highlights the need for a flexible, unified framework.

Figure 1: LAN2CB is an LLM-assisted multi-robot framework that converts mission descriptions into structured representations (e.g., behavior trees), assigns roles and priorities to each robot, and automatically generates executable code to accomplish complex missions. A mission change does not require human intervention since new code is generated automatically. NL means natural language.

Related Work

Traditional multi-robot systems require substantial expert intervention to translate mission descriptions into mathematical formulations, algorithm designs, and executable code, which is labor-intensive and not accessible to non-experts. LLMs' recent advances offer potential solutions by understanding and generating human-like language, enabling them to bridge the gap between human intent and machine execution in robotics. The paper positions LAN2CB as a response to these limitations, leveraging LLMs for multi-robot systems alongside existing methods that emphasize task-specific coordination, learning-based approaches, or prompt-based frameworks.

Figure 2: A comprehensive workflow demonstration of LAN2CB.

Methodology

LAN2CB comprises several stages, beginning with mission analysis where LLMs decompose missions into atomic tasks and analyze dependencies. Code generation follows, where LLMs produce Python code for robot control, including generating mission completion criteria. Execution uses the generated code while updating mission status and transitioning between tasks based on triggers and completion conditions. The process emphasizes mission flexibility and coding efficiency, with standardized mission templates enhancing LLM performance.

Experiments

The paper proposes a comprehensive evaluation of LAN2CB through simulated and real-world missions across three categories, tailored to test mission analysis and code generation capabilities. Experiments demonstrate that LAN2CB effectively parses mission requirements, generates execution code, and adapts to diverse scenarios. The use of a standardized template significantly improves success rates and reduces error incidence, emphasizing structured prompting's importance for robust multi-robot coordination.

Figure 3: Quantitative results for nine multi-robot missions.

Conclusion

LAN2CB represents a significant step towards generalizable and flexible multi-robot coordination using LLMs. It supports diverse mission executions, reducing manual intervention and enhancing adaptability. Future developments could include retrieval-augmented generation to leverage historical mission data, alongside expanding the behavior library's capabilities. The paper implies potential advancements in dynamic multi-robot environments, advocating further exploration into LLM-driven methodologies.

The paper affirms the efficacy of LLMs in facilitating complex multi-robot coordination, bridging the gap between traditional methods and contemporary AI-driven solutions.