AutoAgents: Dynamic LLM Coordination

• AutoAgents is an adaptive LLM-driven framework that automatically generates, coordinates, and refines specialized agents for complex tasks.
• It employs layered observer agents to critique agent roles and execution plans, ensuring robustness through reflective planning.
• Empirical evaluations reveal that AutoAgents outperforms static multi-agent systems with up to 10% gains on benchmarks like MT-Bench and TriviaQA.

The AutoAgents framework is an adaptive, LLM-driven multi-agent orchestration system that automatically generates, coordinates, and iteratively refines a collaborative “AI team” of specialized agents for complex tasks. Distinct from earlier multi-agent methods reliant on static, manually specified agent roles or templates, AutoAgents dynamically constructs both agent roles and stepwise execution plans, with a layered system of observer agents providing meta-level critique and guidance at both the planning and execution stages. This enables a highly flexible, context-sensitive approach to decomposing, tackling, and reviewing challenging, multi-stage problems.

1. Dynamic Agent Generation and Role Assignment

AutoAgents begins with a drafting stage designed to construct a task-specific team of agents and a corresponding plan of action. The process is iterative and involves three interacting roles:

• Planner ($\mathcal{P}$) is responsible for generating candidate agent roles ($\mathcal{A}_i$) and drafting an execution plan ($P$) decomposed into discrete steps ($\mathcal{S}_j$).
• Agent Observer ($\mathcal{O}_{agent}$) evaluates completeness and necessity of the candidate agent list, ensuring comprehensive coverage and absence of redundancy for the given task. It also assesses the adequacy of agent descriptions, including profile, goals, constraints, and toolset.
• Plan Observer ($\mathcal{O}_{plan}$) reviews and critiques the plan, checking for sufficiency, logical flow, and agent/step alignment.

This collaborative loop continues, with each observer providing feedback and the Planner responding with revisions, until convergence (either no further suggestions or maximum iterations reached). Agents are specified formally as $$\mathcal{A} = \{\mathrm{P}, \mathrm{D}, \mathrm{T}, \mathrm{S}\}$$, encompassing a role’s prompt, detailed description, toolset, and execution suggestions. The resulting output is a dynamically tailored agent roster and an actionable plan where each step has allocated specialists, required inputs, expected outputs, and dependency relationships.

This approach contrasts with static frameworks by building the agent population and workflow de novo based on runtime task analysis, mirroring the adaptability and specialization found in human organizational teams.

2. Plan Structuring, Memory, and Execution Coordination

The execution stage proceeds with instantiated agents operating under the supervision of the Action Observer ($\mathcal{O}_{action}$). This meta-agent acts as coordinator:

• Assigning tasks to agents per the execution plan and current status,
• Managing the integration and interpretation of intermediate outputs,
• Monitoring and, if necessary, dynamically revising the plan mid-execution for resilience.

Two principal collaboration paradigms underpin execution:

• Self-Refinement: Individual agents autonomously review and iteratively improve their own outputs, akin to chain-of-thought or self-debugging methods.
• Collaborative Refinement: Multiple agents share and discuss intermediate results, refining outputs through distributed expertise.

To address context window limitations and the accumulation of multi-step artifacts, memory is tiered:

• Short-term memory: Local and ephemeral, used for immediate stepwise refinements.
• Long-term memory: Persistent log of major steps and key outputs.
• Dynamic memory: Transient, selective context constructed for each action, drawing from both short- and long-term repositories.

A formal workflow for the entire process is provided in the paper and summarizes these steps using structured pseudocode (see algorithm in §2).

3. Observer Roles and Reflective Planning

Observer agents exercise a critical, meta-cognitive role and are unique to AutoAgents among contemporary frameworks:

• Agent Observer: Ensures the sufficiency and relevance of agent specialization for the input task.
• Plan Observer: Audits step decomposition and agent-task allocation for robustness and completeness.
• Action Observer: Supervises execution, monitors memory and context, arbitrates collective agent debates, and adapts plans if execution deviates from expected progress.

This observer-driven system introduces “plan reflection,” systematic feedback, and in-process improvement absent from prior methods—directly analogous to human workflow models featuring managers, peer reviewers, or steering committees.

Ablation studies reveal that each observer subsystem is necessary for top performance: disabling any (or any of self/collaborative refinement, layered memory) introduces material performance drops (e.g., –3% for observers or self-refinement alone, –7% cumulatively).

4. Empirical Evaluation and Comparative Performance

AutoAgents was validated across demanding multi-agent benchmarks:

• Open-ended Question Answering (MT-Bench): Achieved a 96.3% success rate (FairEval auto, vs. ChatGPT/Vicuna-13B) and 75% (HumanEval), surpassing both single-agent LLMs (GPT-4, ChatGPT) and multi-agent systems using static roles.
• Trivia Creative Writing (TriviaQA): Delivered top results, with 82.0% (N=5) and 85.3% (N=10) correct output, besting SPP and other leading frameworks by 7–10% (see quantitative table in §4).

A case paper on automatic Tetris game development showcased the emergence of appropriate roles—game designer, UI designer, programmer, tester—generated and coordinated entirely by AutoAgents, yielding robust, high-quality solution artifacts not matched by baselines.

Ablation analysis demonstrates all core innovations—dynamic agent/plan generation, observer-mediated reflective planning, dual-layer collaboration, and advanced memory—provide distinct and cumulative performance benefits.

5. Architectural Features and Illustrative Figures

AutoAgents is characterized by modular, interoperable prompting:

• Modular, domain-agnostic prompt templates for Planner, Observers, and agents—eschewing static, task-specific prompting.
• Explicit architecture diagrams show (i) user input to agent/team/plan draft, (ii) observer feedback and plan iteration, (iii) dual-action (self vs. collaborative refinement), (iv) dynamic memory flow, and (v) real-world use cases (software engineering, creative writing).

Figures illustrate how plan and agent adaptation emerges, and how memory selection ensures scalable operation even as step history grows.

6. Key Insights and Novel Contributions

AutoAgents advances the state of the art in several respects:

• Dynamic, Task-driven Team Formation: Optimizes not just agent population size, but precise specialization and tool allocation, per task.
• Plan Reflection via Observers: Meta-agents ensure that planning and execution benefit from continuous, structured critique and feedback.
• Layered Collaboration: Combination of self-correction and collective debate; prior frameworks rarely supported both modalities together.
• Memory Abstraction and Management: Explicit splitting of memory domains allows both granular agent operation and large-context task tackling.
• Generalized, Composable Prompting: Prompts are “plug-and-play” across domains, supporting rapid extension and deployment.
• Empirical Superiority: Demonstrated quantitative gains (≥7–10% over best baselines) and qualitative advances (more coherent, rigorous multi-agent outputs) evidence the utility of this design.

7. Conclusion and Future Perspectives

AutoAgents represents a comprehensive strategy for LLM-based multi-agent orchestration, able to automatically and adaptively assemble, coordinate, and refine expert teams for open-ended, multi-domain, and multi-stage tasks. Its hybrid of dynamic agent planning, observer-driven reflective review, dual-collaboration pathways, and multi-layered memory yields advanced robustness, flexibility, and empirical superiority over earlier frameworks. Architectural generality and modular prompting make it a foundation for future, more complex forms of agentic AI and organizational intelligence.

For implementation resources, reference code and explicit prompts are provided by the authors: https://github.com/Link-AGI/AutoAgents

---

References

• AutoAgents: A Framework for Automatic Agent Generation (Chen et al., 2023)