--- title: LLM-based Multi-Agent Systems url: https://www.emergentmind.com/topics/llm-based-mas-d23075f5-fa13-4b88-8c96-a4a23a703665 type: topic --- # LLM-based Multi-Agent Systems LLM-based multi-agent systems (LLM-based MAS) are distributed intelligent architectures in which multiple agents, each instantiated as or powered by a large language model (LLM), collaborate to solve tasks requiring collective reasoning, planning, tool-use, critique, or specialized workflows. These systems are engineered with explicit message-passing protocols, shared environments, rigorous benchmarking, and often complex topological structures that reflect problem requirements and optimization considerations. The field has evolved from simple static role-play and chain-of-thought pipelines to dynamic, graph-adaptive ensembles, modular frameworks, and highly specialized inter-agent protocols. This article surveys state-of-the-art methodologies, architectures, benchmarks, domain applications, optimization and scaling trends, and current limitations, drawing directly from foundational open-source infrastructures and controlled experimental studies. ## 1. Unified Software Architectures and Messaging Protocols Modern LLM-MAS frameworks are characterized by a modular, extensible architecture that treats every agent as a subclass or runtime instance inheriting from a common base class. MASLab [2505.16988] exemplifies this structure: all MAS algorithms subclass a unified BaseMAS, which manages LLM invocations (hosted APIs or vLLM), token/time tracking, logging, and debugging. Agents are composable modules ('Planner', 'Critic', 'Executor') that communicate via standardized message buffers. The system enforces explicit message-passing APIs: ``` AgentA.send(message) → AgentB.receive() ``` with shared schemas for role assignment, dynamic prompt templates, and tool invocation routines (e.g., code execution, web search, image analysis). Inter-agent communication can be centralized, decentralized, or layered; topologies (A ⊆ Agents × Agents) are programmed either statically in code, or determined adaptively at runtime (see AMAS [2510.01617] and DynaSwarm [2507.23261]). Benchmark environments and tasks are encapsulated behind uniform Environment interfaces with query generation, ground-truth labeling, and standardized evaluation entrypoints, permitting fair, direct comparison across methods and model backends. Global configurations enforce uniform LLM settings for all agents (model selection, temperature, token limits), ensuring that only algorithmic differences influence outcomes. Extending the system requires subclassing the base MAS, registering new methods and benchmarks, and using provided validation workflows—supporting rapid scaling, fair ablation studies, and reproducibility. ## 2. MAS Methodological Taxonomy: Workflows and Topologies LLM-based MAS architectures are broadly grouped into functional domains and workflow classes: - **Single-Agent Baselines:** Vanilla LLM, Chain-of-Thought (CoT) prompting for stepwise reasoning [2505.16988]. - **Collaborative Planning:** Fixed roles (CAMEL), User–Assistant loops (AutoGen) [2505.16988]. - **Debate and Critique:** Multi-agent debate (MAD, LLM-Debate): iterative rounds where proposer and opponent agents exchange arguments; a judge agent aggregates and decides. Empirical gains scale positively with task depth and width [2510.04311]. - **Workflow Generation:** Systems like AgentVerse dynamically recruit, plan, and critique, with agent selection loops implemented as LLM calls. MAS-GPT reframes MAS design as executable program synthesis by a meta-LLM [2503.03686]. - **Optimization-Based MAS:** GPTSwarm, ADAS, and AFlow treat the agent interaction graph as learnable parameters, using gradient or actor-critic RL to optimize the collaboration topology for maximum downstream utility [2507.23261][2510.01617]. - **Tool-Augmented MAS:** Agents invoke external APIs or toolkits (code, vision, web) within a message-passing protocol, as in OWL-Roleplaying or ReAct-MASLab [2505.16988]. - **Domain-Specific Agentic MAS:** Custom agent libraries for tasks such as medical therapy (MedAgents), mathematical reasoning (MACM), scientific prediction (Urban-MAS), and simulation of strategic marketplaces [2511.13233][2511.00096][2507.10911]. Recent architectures exploit parallelized planning-acting to maximize real-time responsiveness and interruptibility, deploying dual-thread structures synchronized by centralized memory [2503.03505]. Blackboard systems implement shared, evolving memory objects with dynamic agent selection and consensus extraction, achieving improved token efficiency [2507.01701]. ## 3. Benchmarks, Metrics, and Standardized Evaluation Benchmarks are critical to empirical evaluation, capturing both domain generality and method-specific strengths. MASLab [2505.16988] includes 10+ tasks: symbolic math (MATH, AQUA-RAT, AIME), science QA (SciBench, GPQA), common code benchmarks (HumanEval, MBPP, GAIA), and medically oriented QA (MedMCQA). Key metrics: - **Accuracy:** $$\mathrm{Acc} = \frac{1}{N} \sum_{i=1}^N \mathbf{1}[\hat y_i = y_i]$$ - **Average Rank:** $$\mathrm{AvgR} = \frac{1}{M}\sum_{m=1}^M \mathrm{rank}_m$$ - **Token Cost/Latency:** Ratio of total tokens to queries, mean wallclock time. - **Reward:** Pass-rate on tool-augmented tasks, win-rate or secondary criteria in MAS-driven games (WiS Platform [2412.03359]). - **Robustness:** Error rate $\delta$, resilience $R=1-\frac{1}{T}\sum_t \delta_t$, recovery time $\tau$ post-fault [2505.03096]. - **Semantic Agreement:** Human–LLM protocol matching, reported $>98\%$ for LLM-based extraction, $65\%$ for best rule-based [2505.16988]. MAS frameworks such as MAESTRO [2601.00481] supply run-to-run reproducibility signals: structural (Jaccard edge overlap), order-aware (LCS) similarity, and full execution traces (OpenTelemetry). Failure analysis distinguishes explicit system faults from silent semantic errors. ## 4. Optimization, Adaptation, and Scaling Leading LLM-MAS systems incorporate advanced optimization mechanisms, adversarial training, cross-task experiential learning, and dynamic adaptation: - **Token/Brevity Optimization:** Optima [2410.08115] trains agents to maximize task accuracy while minimizing token output using reward–penalized supervised or preference optimization; conversations as trees, with Monte Carlo Tree Search methods to diversify training traces. - **Dynamic Topology Selection:** Actor-critic RL (A2C) is used in DynaSwarm [2507.23261] to refine edge selection in collaboration graphs; AMAS [2510.01617] leverages a lightweight LoRA-adapted LLM to select among small pools of RL-optimized graphs per query, achieving statistically significant gains over static topologies. - **Cross-Task Experiential Learning:** MAEL [2505.23187] equips each agent with an experience pool (state, action, reward triples) harvested in training. At inference, agents retrieve high-reward, context-similar exemplars via text embeddings and cosine similarity, improving sample efficiency and convergence. - **Cooperative MARL:** MAGRPO [2508.04652] models collaborative dialogue as a Dec-POMDP, optimizing joint policies via group-relative advantages and clipped PPO-style objectives. This method produces measurable improvements in both writing and code collaboration settings. Scaling trends show monotonic improvements in accuracy and reliability for most methods with increasing LLM size and sample count; however, some architectures (AgentVerse) exhibit threshold effects, with format errors dominating at small model sizes [2505.16988]. ## 5. Domain Applications: Urban Prediction, Data Marketplaces, Therapy Recommendation LLM-based MAS have demonstrated applicability across heterogeneous real-world domains: - **Urban AI:** Urban-MAS [2511.00096] integrates deep-research agents for factor prioritization, extraction agents for robust feature acquisition (consistency checks and re-extraction by similarity), and fusion agents for multi-dimensional inference (running amount, perception metrics), yielding up to 51% error reduction in perception tasks. - **Data Marketplaces:** Simulated environments with strategic buyer/seller agents [2511.13233]. Agents reason in natural language (“think aloud” via chain-of-thought), execute budgeted transactions, update prices, and model demand. Distributional metrics (e.g., purchase counts, buyer repeat rate) closely mirror real-world market trends. - **Medical Decision Support:** Multi-disciplinary therapy MAS [2507.10911] enables conflict resolution in multimorbidity cases by partitioning tasks among specialist LLMs. Evaluation uses correctness/completeness, DDI ratio, conflict ratio, medication burden, and clinical goals met, with detailed error analyses and ablation studies comparing single-agent to MAS protocols. Domain-specific MAS consistently outperform single-agent baselines on complex, multi-factor problems, particularly when agent decomposition and inter-agent verification protocols are rigorously designed. ## 6. Robustness, Security, and Fault Tolerance Robustness is assessed via chaos engineering (LLM hallucination, crash, communication fault injection) and security-oriented adversarial analysis: - **Chaos Framework:** Quantifies error rates, resilience, recovery times under controlled agent failures and message loss [2505.03096]. Agent redundancy and token-level semantic checks halve error rates and decrease recovery times by over 70%. - **Securing MAS:** AgentShield [2511.22924] introduces three-layer defense: critical node auditing via combined graph centrality/task-contribution; light token auditing through strict-sentry models; and two-round consensus with heavyweight arbiters. This achieves 92.5% recovery rate with 70% lower overhead compared to traditional majority voting. - **Topology-Guided Remediation:** G-Safeguard [2502.11127] constructs multi-agent utterance graphs, applies edge-featured GNN detection, and prunes compromised nodes’ outgoing edges, recovering >40% lost accuracy under prompt injection. - **Intention-Hiding Threats:** AgentXposed [2507.04724] develops HEXACO-based drift scoring and adaptive interrogation protocols to detect malicious agents across centralized, decentralized, and layered topologies, highlighting unique vulnerabilities and cost-inflation strategies. Security-focused MAS development now routinely involves trace-level logging, topological vulnerability analysis, and distributed, decentralized auditing to counter both overt and covert agent compromise. ## 7. Observability, Evaluation Tools, and Future Directions Recent frameworks, such as MAESTRO [2601.00481], have standardized MAS execution and instrumented controlled comparison, integrating third-party MAS via adapters, exporting complete telemetry, and rigorously analyzing call-graph stability, resource consumption, and failure signatures. Empirical findings underscore that MAS architecture exerts a greater influence on cost, latency, accuracy, and reproducibility than base model upgrades or tool choices. Richer workflows increase semantic failure rates and may penalize final accuracy unless carefully designed and debugged. End-to-end signal harvesting enables fine-grained triage and reproducible optimization. Limitations remain: coverage of novel MAS methods is incomplete, and many frameworks lack challenging long-horizon planning benchmarks or comprehensive user studies. Ongoing research targets scaling to larger agent societies, deeper retrieval-augmented models, hierarchical consensus, and benchmark diversification. The methodological and empirical rigor enforced by unified environments, dynamic topologies, principled security, and quantitative observability provides a practical and theoretical foundation for designing robust, efficient, and high-performance LLM-based MAS. As new agent classes, benchmarks, security threats, and optimization paradigms emerge, research-centric codebases (MASLab, MAESTRO) and advanced frameworks (AMAS, DynaSwarm, Optima) will continue to track, validate, and extend the collective intelligence frontier of LLM-based multi-agent systems. Source: https://www.emergentmind.com/topics/llm-based-mas-d23075f5-fa13-4b88-8c96-a4a23a703665