LLM-Based Multiagent Systems

• LLM-based Multiagent Systems are architectures where autonomous agents utilize pre-trained LLMs like GPT-4 for real-time reasoning and self-adaptation.
• The integration of LLMs into the MAPE-K loop enables agents to monitor, analyze, plan, and execute natural language-based actions leading to emergent strategies.
• Experiments, such as in online marketplaces, highlight enhanced decision-making, transparent communication, and challenges in prompt history and token limitations.

LLM-Based Multiagent Systems (MASs) are a paradigm in which autonomous agents, each powered or orchestrated by modern LLMs, cooperate, compete, or coordinate to solve complex, dynamic tasks. Unlike traditional MASs, which typically rely on hand-coded rule engines or domain-specific solvers for inter-agent communication, LLM-based MASs leverage the generative and adaptive reasoning capacities of models like GPT-4 to achieve highly expressive, context-aware, and adaptive agent interactions. This approach is motivated by the need for enhanced self-adaptation, robustness, and emergent behavior in increasingly intricate computational environments.

1. LLM Integration in Multiagent Systems

The architecture of LLM-based MASs fundamentally extends the established autonomic computing model for self-adaptive systems, notably the MAPE-K loop (Monitor, Analyze, Plan, Execute, Knowledge). Each agent embeds an LLM (specifically, GPT-4 in the reference implementation), which assumes the roles of analyzer, planner, and knowledge synthesizer within the adaptation loop. The canonical agent architecture is decoupled into:

• Managed Element: Interfaces with the environment via sensors and actuators.
• Autonomic Agent: Monitors environmental input and inter-agent messages, generates natural language prompts, invokes the LLM for decision-making, and executes LLM-selected commands.

Within this modified MAPE-K arrangement, the classical Analyze, Plan, and Knowledge components are subsumed by the LLM, reflecting a hybridization of traditional monitoring and execution with advanced language-driven agency. Environmental signals, peer messages, or self-reflections are synthesized as LLM prompts, which are then processed to yield next-step actions, negotiation tactics, or adaptation strategies.

2. Methodological Frameworks and System Architecture

LLM-driven MASs instantiate a feedback-centric architecture. Agent operation is typically cyclical:

1. Monitor: Gather environment and peer state.
2. Synthesize Prompt: Aggregate relevant knowledge and interaction history into a structured LLM-compatible input, taking into account prior dialogues and context (limited by token constraints).
3. Analyze/Plan (LLM Core Phase): The LLM processes the prompt, employing its pretrained language reasoning and planning capabilities to propose an action, negotiation, or counterfactual scenario.
4. Execute: The agent interprets LLM output as commands or communicative acts.

Crucially, system-level implementations replicate this per-agent design across the MAS, allowing for emergent communication and adaptation. For demonstration, an online book marketplace was constructed atop the JADE framework, with GPT-4 agents playing buyer and seller roles, dynamically negotiating based on sequential context and peer messages via OpenAI's API at a temperature of 0.7 to promote strategic diversity and exploration.

3. Emergence, Adaptivity, and Observed Behaviors

Experiments with the online marketplace scenario yielded several salient outcomes:

• Diverse Strategic Behavior: Agents initialized with identical prompts, when deployed with LLM-driven analysis and memory concatenation, rapidly evolved disparate negotiation and bargaining strategies, underscoring the emergence of individualistic behavior in a homogenous MAS population.
• Emergent and Unexpected Behaviors: Instances such as self-messaging (an agent messaging itself due to prompt misconstruction or context ambiguity) highlighted both the flexibility and the unpredictability of LLM-mediated agency.
• Self-Adaptation: Sellers exploited simulation iterations, adjusting pricing tactics to maximize payoff (e.g., conceding a sale in the final iteration to avoid losses), and buyers were observed leveraging dynamic price information from the environment and peer offers.
• Communication Expressiveness: Because the LLM interprets and generates natural language directly, the expressiveness and adaptability of inter-agent exchanges surpass that afforded by rigid protocol-based MASs.

4. Challenges: Prompt History, Memory, and Model Independence

Several challenges are inherent to this class of MAS:

• Long-Term Interaction History: Unlike the consumer-facing ChatGPT interface, typical LLM APIs (e.g., OpenAI's GPT-4 API) do not maintain conversation state, necessitating local context history storage for prompt assembly. Token window limitations mean only a fixed number of exchanges can be preserved, possibly truncating useful prior knowledge.
• Cross-Agent Contamination: Deploying multiple agents through a single shared model instance or API account risks context leakage, whereby agent B's state could inadvertently impact agent A's reasoning (at the cache level). The authors recommend distinct API accounts or model deployments to mitigate this and achieve true independence.
• Scaling and Token Limitations: Context concatenation and token cost scale quickly with increased agent population and communication epochs, which must be managed in practical deployments.

5. Implications for Self-Adaptive Systems and Autonomous Agency

The integration of LLMs into MASs marks a substantial advance in the self-adaptation paradigm:

• Pre-equipped Linguistic Reasoning: Rather than evolving symbolic protocols or neural communication policies from scratch through evolutionary searching, agents inherit advanced linguistic and reasoning capabilities ab initio, expediting adaptation and interoperation.
• Enhanced Problem Solving: LLMs, capable of synthesizing environmental signals, peer dialogues, and knowledge recall in a unified framework, enable complex negotiation, coalition formation, and conjecture far beyond traditional finite-state or rule-based MASs.
• Transparent Decision Chains: Action sequences and adaptation paths are explainable post hoc via the prompts and outputs, bridging the interpretability gap relative to black-box deep RL approaches.

6. Experimental Evaluation and Future Directions

The reference experiment demonstrates key numerical results:

• Decision Accuracy and Reasoning: Agents showcased reasoning and adaptation in a competitive environment, with emergent pricing strategies and negotiation tactics.
• Emergent Diversity: Even with homogeneous initialization, the integration of diverse prompts led to self-organization within the agent population.

Open questions and suggested directions:

• Auxiliary Planning and Memory: Supplementing LLM reasoning with structured, external planning modules and persistent agent memory to improve stability, action rationality, and context recall.
• Heterogeneous MASs: Deploying multiple, non-shared LLM instances per agent to guarantee independence and enable specialization.
• Generalization to Robotics and Human-in-the-Loop Settings: Transferring this paradigm to domains such as evolutionary robotics, where agents receive both environmental and human feedback, poses both opportunities for enhanced adaptability and challenges for transparent audit.

7. Significance and Paradigm Shift

LLM-based MASs, as exemplified by the described MAPE-K+LLM architecture, represent a shift from incremental protocol or neural communication evolution to a model of agentic systems where advanced planning, reasoning, and adaptation arise intrinsically through natural language interaction and context-driven prompt engineering. With appropriate modularization, memory integration, and deployment strategies, these systems may achieve and even surpass the adaptability, transparency, and expressiveness required for complex, dynamic multiagent environments.