LLM-Based Multi-Agent Framework

• Large Language Model-based Multi-Agent Framework is defined as a system where multiple LLM-driven agents, each with persistent state, collaborate to solve complex, coordinated tasks.
• The framework integrates core modules like the LLM inference engine, agent modules, observation manager, and environment API to enable dynamic, context-rich interactions.
• Empirical evaluations reveal emergent collective intelligence in open-ended planning while also highlighting limitations in procedural tasks and context window constraints.

A LLM-based Multi-Agent Framework defines a system where multiple agents—each powered by an LLM instance and equipped with persistent state, role-specific prompts, and independent observation memory—collaborate within a shared, dynamically evolving environment to solve complex tasks requiring coordination, communication, and world interaction. This paradigm models the synergy observed in human group problem solving, exploiting the interpretive, planning, and communication capacities of LLMs to demonstrate emergent, collective intelligence for structured simulations and real-world-inspired tasks (Sprigler et al., 2024).

1. Architectural Principles and System Components

A prototypical LLM-based multi-agent framework consists of the following core modules ((Sprigler et al., 2024), Table 1):

Module	Function	Notable Features or Mechanisms
LLM Inference Engine	Provides the underlying, shared LLM model for all agents	Single Mistral-7B-instruct; shared weights, per-agent histories
Agent Module	Encapsulates per-agent state, prompts, and memory	System prompt, mutable dialogue history, decayed observation memory
Observation Manager	Scores and retrieves contextually relevant observations	Recency (exponential decay), importance (LLM-rated), semantic relevance (embedding similarity)
Environment API	Maintains and exposes environment state as mutable object graph	Associative arrays, catalog of functions (e.g., thermostat adjustment, message passing)
Moderator (optional)	Mediates agent actions and message validity in complex settings	Action validation, re-prompting, environment-effect control

Within a global event loop, agent selection (round-robin or message-driven), context synthesis (summarized environment and top observations), decision solicitation (via JSON/text schema), validation, state transition, and new observation logging are orchestrated for each turn.

2. Internal Agent Design and Memory

Agents are formalized as stateful LLM-driven entities, each maintaining:

• System Prompt: Defines role and situational identity, e.g., “You are an [accountant/engineer/moderator] living in a studio apartment…”
• History: Message list $[ (role_i, content_i) ]$—provides agent-specific dialogue and decision trace.
• Observation Memory: Set $O = \{ o_j \}$, each with text, timestamp $t_j$, importance $imp_j$, and relevance $rel_j$.
• Observation Scoring: $$R_j = exp(-0.03\cdot(t_{now} - t_j)) + imp_j + rel_j$$, providing recency-weighted, importance-modulated, and semantically filtered retrieval for context injection.

Decision making is formalized as maximizing

$\arg\max_{y}\, P(y \mid \text{history})$

where $y$ is either structured JSON or natural language text.

3. Communication, Messaging, and Synchronization Protocols

Message-passing is JSON-encoded:

• Inter-agent communication: $$\{"to": \text{agent\_name}, "message": \text{string}\}$$
• Environment actions: $$\{"action": \text{function\_name}, "args": \{\ldots\}\}$$

Turn-taking is either round-robin (structured environments) or dynamic (e.g., next speaker equals last receiver in chat scenarios). Moderators may validate action schemas or enforce adherence to environmental constraints.

Let $O = \{ o_j \}$0 denote agents and $O = \{ o_j \}$1 be state at turn $O = \{ o_j \}$2, then for agent $O = \{ o_j \}$3:

1. Receives context $O = \{ o_j \}$4.
2. Outputs $O = \{ o_j \}$5.
3. State transition: $O = \{ o_j \}$6.
4. New observation $O = \{ o_j \}$7 logged to $O = \{ o_j \}$8.

4. Environment Integration and Dynamics

• State representation: World state $O = \{ o_j \}$9 as a set of objects $t_j$0, each $t_j$1 an associative array of attributes.
• API-driven world interaction: Agents call functions $t_j$2, composed in an externally defined, centrally validated catalog.
• State transitions: $t_j$3 with automatic observation extraction post-transition, typically via an auxiliary LLM instance configured to summarize state deltas.

5. Algorithms, Coordination Loop, and Observation Retrieval

The main agent-environment coordination loop:

$t_j$4

Observation retrieval pseudocode: $t_j$5

This ensures prompt context for each agent includes the most relevant, recent, and semantically aligned observations.

6. Empirical Evaluation: Synergy, Performance, and Emergent Behavior

Task-specific simulation studies include:

• Studio apartment scenario: Two roommates plus moderator. High-level coordination tasks (e.g., bills, temperature management, spreadsheet creation) and procedural tasks (cake-baking).
  • Success in complex, multi-faceted collaborative tasks (e.g., finance).
  • Failures due to over-assumed task parallelism or function-selection errors in granular procedure (cake-baking).
• Collaborative coding scenario: Three agents and an “expert” moderator; tasked with implementing linked-list algorithms.
  • No successful direct code solves, but effective use of chatroom modality for explanation and limited decomposition.

Overall observation: Human-like collaborative synergy emerges for broad planning and open-ended coordination tasks, while stepwise, highly procedural sub-problems expose the limits of prompt context, memory retention, and function specification (Sprigler et al., 2024).

7. Limitations and Directions for Advancement

• Context window constraints: Agents may lose track of detailed environmental features during lengthy or highly granular tasks.
• Programmatic API use: Recurrent selection of incorrect function call signatures or over-generation (dumping full solutions in one step) remains prevalent.
• Code environments: Lack of an automatic code execution-feedback loop impedes agent ability to self-correct or debug sequential code outputs.
• Evaluation methodology: Current performance assessment is mostly qualitative; no standardized, quantitative synergy metrics are deployed.

Potential future enhancements include:

• Asynchronous, non-turn-based agent architectures.
• Fine-tuning or ensembling tailored LLMs per agent role or task structure.
• Automated execution and iterative code-testing in coding environments.
• More sophisticated (hierarchical, graph-based) observation memories.
• Explicit modeling of workflow dependencies as directed acyclic graphs to capture subtask relationships.

---

• Synergistic Simulations: Multi-Agent Problem Solving with LLMs (Sprigler et al., 2024)