Co-Evolving Multi-Agent Systems (CoMAS)

• CoMAS are multi-agent systems characterized by mutual adaptation of agents, environments, and policies during both training and execution.
• They employ diverse mechanisms such as state-dependent couplings, shared versus private knowledge, and hierarchical adaptation to optimize system performance.
• Research shows that while controlled co-evolution can boost metrics like task accuracy and efficiency, it also introduces challenges like instability and reward hacking.

Co-Evolving Multi-Agent Systems (CoMAS) denote multi-agent systems in which the relevant components of the system change interdependently over training or execution, so that each component becomes part of the others’ learning or adaptation environment. In the literature, this idea appears in several forms: evolving agent populations with state-dependent couplings, decentralized collaborative optimization with personalized and shared parameters, agent–environment co-optimization, governance and topology adaptation, and LLM-based systems in which prompts, workflows, roles, routing, and communication structures are revised from interaction experience (Wilde et al., 2011). A stricter reading reserves the term for systems with symmetric, decentralized co-evolution among autonomous agents; a broader reading includes hierarchical or partially centralized settings in which multiple interacting agentic components mutually adapt over time (Zhang et al., 14 May 2026).

1. Scope and conceptual boundaries

CoMAS is not a single method but a family of formulations unified by mutual adaptation. In one line of work, the evolving object is the population state together with the effective interaction field, as in state-dependent cooperation models where pairwise influence depends on current agent states (Caram et al., 2015). In another, the evolving object is a structured decomposition of shared and private knowledge, as in decentralized collaborative learning over a communication graph (Deng et al., 13 Jun 2025). In recent LLM-based systems, what co-evolves may include prompts, role specializations, routing policies, communication topologies, or organizational scaffolds (Xu et al., 10 May 2026).

A useful distinction runs between partial adaptation and co-evolution proper. Several papers argue that optimizing only one layer of a multi-agent system is insufficient. MetaAgent-X states this as the “frozen-executor ceiling”: if the meta-level designer improves while executors remain fixed, workflow optimization cannot induce new task-specialized execution skills (Zhang et al., 14 May 2026). EvolveRouter makes a related point for question answering: routing over a fixed pool of agents misses the fact that agent quality itself should change, so router diagnostics are used to drive prompt refinement (Huang et al., 6 Apr 2026). TacoMAS extends the same principle to inference-time organization, arguing that capability and topology should both adapt, but on different time scales (Xu et al., 10 May 2026).

At the same time, the literature marks important boundary cases. “When Agents Evolve, Institutions Follow” shows that governance topology is a first-order determinant of performance and that the best institution shifts with model capability and task regime, but it does not implement endogenous institutional learning; it is best read as an empirical precursor to CoMAS rather than a full co-evolutionary algorithm (Fei et al., 30 Apr 2026). MetaAgent-X is strongly CoMAS-like, but its co-evolution is mediated by a central designer and shared reward, so it is only a partial match if one requires fully symmetric decentralized evolution (Zhang et al., 14 May 2026).

Axis of co-evolution	Representative mechanism	Examples
Agent states and couplings	State-dependent interaction laws or population dynamics	(Caram et al., 2015, Wilde et al., 2011)
Shared and private knowledge	Shared adapter exchange plus local personalization	(Deng et al., 13 Jun 2025)
Agent policies and tasks	Curriculum or environment instances evolve with learner competence	(Lin et al., 8 May 2025, Gao et al., 2024)
Capability and organization	Joint adaptation of prompts, routing, roles, topology, or governance	(Huang et al., 6 Apr 2026, Xu et al., 10 May 2026, Fei et al., 30 Apr 2026)

2. Formal foundations

The older analytical literature provides several formal templates for CoMAS. In the generalized Verhulst–Lotka–Volterra model of cooperative peer-to-peer systems, each agent has a scalar state $s_i(t)$, and the pairwise interaction kernel depends on current size similarity,

$$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$

Because the coupling changes with the states and the states change under the coupling, the system exhibits a direct form of co-evolution through state-dependent interaction. The paper shows cluster formation, multistability, and cooperative regimes in which equal-size coexistence can exceed the standalone capacity limit $1$ (Caram et al., 2015).

A complementary population-level formalism appears in “Stability of Evolving Multi-Agent Systems,” which models an evolving MAS as a Markov process over population states. Agent-level transitions depend on the full current population state, the number of agents may vary through fitness-based reproduction and death, and stability is defined through convergence of occupation probabilities to a limit distribution. The paper further introduces an entropy-based degree of instability,

$\delta = H(p^\infty),$

as a macroscopic measure of how concentrated or diffuse the long-run state distribution is (Wilde et al., 2011).

Evolutionary game theory supplies another formal language. The survey on emergent behaviours in MAS treats large populations of strategically diverse agents as frequency-dependent systems in which strategy success depends on current population composition. It explicitly invokes replicator equations and the Fermi imitation rule,

$\left(1 + e^{\beta(f_A - f_B)}\right)^{-1},$

to study cooperation, trust, commitment, guilt, and AI-safety ecologies (Han, 2022). This suggests a broad interpretation of CoMAS in which co-evolution is not limited to explicit gradient updates; it may also be mediated by imitation, selection, and endogenous payoff landscapes.

Competitive search offers an additional foundation. “Evolving Strategies for Competitive Multi-Agent Search” formalizes multi-agent search on an NK landscape in which agents interact through public memory and through dynamic changes in the landscape caused by search itself. The paper’s central contribution is to make environment-mediated competitive co-adaptation explicit: agents search on a landscape whose value structure changes because of prior searches (Bahceci et al., 2023).

3. LLM-based CoMAS

Recent LLM-based work turns CoMAS into an operational design problem for agentic systems. MetaAgent-X formulates automatic MAS learning as a bilevel RL problem with a Designer policy $\pi^{\mathcal D}_{\vartheta_{\mathcal D}}$ that emits executable Python workflow scripts and an Executor policy $\pi^{\mathcal E}_{\vartheta_{\mathcal E}}$ that acts inside the designed system. The paper’s core contribution is end-to-end optimization of both levels, supported by Executor-Designer Hierarchical Rollout and Stagewise Co-evolution. Across six benchmarks and two backbones, it reports an up to 21.7% gain, and ablations show that designer-only training improves little, executor-only training saturates, coupled simultaneous training collapses, and stagewise alternation yields the strongest results (Zhang et al., 14 May 2026).

The paper “CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards” takes a different route. It trains separate LLM agents through structured interaction consisting of solution, evaluation, and scoring roles. Rewards are derived from judged discussion quality rather than external verifiers, with complementary solver and evaluator rewards based on scores in $\{1,2,3\}$. The framework is explicitly decentralized, uses separate replay buffers and independent policy updates, and reports consistent gains over untrained agents across GSM8K, MATH-500, HumanEval, MBPP, SciBench, GPQA, and MMLU. Its ablations are particularly important: removing the evaluation step or the scoring step degrades performance and can induce reward hacking, whereas the original adversarial interaction design stabilizes reward near $0.5$ (Xue et al., 9 Oct 2025).

TacoMAS addresses test-time rather than training-time evolution. It represents the system at round $t$ as a directed agent graph $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$0, where $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$1 is the communication topology and $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$2 the set of role-specific capabilities. Its central claim is that effective test-time evolution requires jointly adapting both axes, but on different time scales: a fast capability loop updates expertise every round, while a slow meta-LLM topology loop performs bounded edge edits and birth-death operations every $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$3 rounds. On four benchmarks, TacoMAS reports an average improvement of 13.3% over the strongest baseline (Xu et al., 10 May 2026).

Meta-Team shifts the focus from trajectory search to collaborative self-evolution. It preserves each agent’s local execution context, adds post-task communication after the final result is known, and performs evolution at three scales: L1 agent-level patches and skill updates, L2 teammate profiles and pairwise collaboration notes, and L3 revisions to the shared constitution and candidate roster. Across six long-horizon benchmarks, it reports an average of 62.7, compared with 56.1 for fixed MAS and 56.8 for the single-agent baseline; ablations also show that collaborative experience organization beats both centralized and partitioned experience schemes (Hao et al., 28 May 2026).

EvolveRouter illustrates a weaker but still recognizable CoMAS form. It jointly improves a graph-based query router and the prompts of a fixed pool of 24 QA agents. Router diagnostics determine which prompts to rewrite, and updated prompts change the soft routing targets used for later router training. The paper also adds adaptive inference through router-weighted answer agreement, reducing average agent calls from 24 to between 2.9 and 7.4 depending on the dataset while maintaining or improving F1 (Huang et al., 6 Apr 2026).

4. Decentralized optimization, curricula, and collective learning

Not all CoMAS work is organized around LLM prompting. PE-MA is a decentralized, parameter-efficient framework in which each agent holds a frozen backbone $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$4, a communicated shared adapter $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$5, and a private personalized adapter $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$6. The shared adapters are mixed over a communication graph $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$7 via a doubly stochastic matrix $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$8, while personalized adapters remain local. This decomposition is the paper’s definition of co-evolution: each agent evolves along a collaborative trajectory through $$\gamma(s_i,s_j)=K\exp\!\left[-\left(\frac{s_i-s_j}{\sigma}\right)^2\right].$$9 and a private trajectory through $1$0. The theory gives an asymptotically optimal convergence rate of $1$1, and the communication results are substantial: on Office-Home, PE-MA communicates $1$2M parameters versus $1$3M for DSGD-ALT (Input) and about $1$4M for DSGD-SIM (Adapter) (Deng et al., 13 Jun 2025).

HRCL addresses decentralized combinatorial optimization in evolving environments by splitting adaptation across two levels. A high-level MARL layer chooses plan-constraint groups and behavior ranges, reducing the action space, while a low-level collective learning layer based on EPOS performs decentralized coordinated plan selection with minimal communication. The paper argues that this hierarchy yields a “win-win synthesis solution,” improving performance, scalability, and adaptability relative to standalone MARL and standalone collective learning in synthetic scenarios, energy self-management, and drone swarm sensing (Qin et al., 22 Sep 2025).

CCL moves the co-evolving object from agents and coordination to the task distribution itself. In sparse-reward cooperative MARL, it evolves intermediate tasks using population-based crossover, mutation, and prototype-based fitness estimation, while agents are trained on the resulting curriculum with MAPPO. The fitness signal is centered on moderate success rates, so tasks that are too easy or too hard are downweighted. On five cooperative tasks from MPE and Hide-and-Seek, CCL is reported to outperform MAPPO, POET, GC, GoalGAN, and VACL, reaching 98.4% on Ramp-Use, 95.4% on Lock-back/Lock and Return, and about 99% on Simple-Spread and Hard-Spread (Lin et al., 8 May 2025).

These optimization-centric frameworks suggest that CoMAS need not be defined by end-to-end neural co-training alone. Shared/private decomposition, hierarchical collective learning, and co-evolving curricula all instantiate mutual adaptation in ways that are algorithmically distinct from LLM dialogue systems.

5. Environment, governance, and external information

Several papers broaden CoMAS beyond agents alone. “Co-Optimization of Environment and Policies for Decentralized Multi-Agent Navigation” treats the environment as a decision variable. Agents with a decentralized GNN policy $1$5 and an environment generator $1$6 are optimized in an alternating coordinated scheme, with the joint problem written as

$1$7

The environment is reconfigurable through obstacle-layout variables, and the paper reports that optimized environments can provide structural guidance for de-conflicting agent motion (Gao et al., 2024).

At a higher organizational level, “When Agents Evolve, Institutions Follow” translates seven historical political institutions into executable MAS architectures formalized as

$1$8

Its principal result is that governance topology strongly shapes collective performance: on PinchBench, within a single model, the gap between the best and worst institution exceeds 57 percentage points, and the optimal institution shifts across model backends and task regimes. The paper explicitly states that it does not implement endogenous institutional evolution, but it establishes the empirical precondition for it by showing that no single governance form is universally optimal (Fei et al., 30 Apr 2026).

Infrastructure co-design provides another adjacent perspective. “Computation and Communication Co-Design for Real-Time Monitoring and Control in Multi-Agent Systems” jointly optimizes per-agent local processing $1$9 and communication scheduling under an AoI objective. Its contribution is not framed as CoMAS, but it is clearly a coupled multi-agent adaptation problem in which local computation, shared communication, and the base station’s global information state reshape one another. The paper reports 18–82% performance improvement across occupancy-grid mapping and ride sharing (Tripathi et al., 2021).

A more formal symbolic extension appears in DACmMCMAS, which augments commitment-based, data-aware MAS with access to heterogeneous external information sources modeled as contexts. The key claim is that commitment-based first-order agent systems can interact with external contexts while retaining the formal properties of the original approaches (Costantini, 2014). This suggests a line of CoMAS research in which co-evolution involves not only agents and environments but also externally grounded institutional knowledge.

6. Evaluation, controversies, and open directions

The literature converges on several recurring difficulties. First, mutual adaptation is often unstable. MetaAgent-X reports that naive simultaneous designer–executor training can collapse into degenerate repetitive outputs, and its sensitivity analysis shows that switching stages too frequently can also cause collapse (Zhang et al., 14 May 2026). CoMAS via interaction rewards shows that if the adversarial structure of evaluation and scoring is relaxed, the system can drift toward “support all solutions” reward hacking (Xue et al., 9 Oct 2025). These results indicate that co-evolutionary learning requires carefully engineered credit assignment and update schedules.

Second, the definition of CoMAS remains contested. MetaAgent-X, EvolveRouter, and Meta-Team all qualify under a broad notion of mutually adapting agentic components, but each retains some centralized element: a meta-designer, a routing controller, or a scaffold-level evolution manager (Zhang et al., 14 May 2026). PE-MA, by contrast, is explicitly decentralized and graph-constrained, while TacoMAS combines local fast loops with a centralized slow meta-controller (Deng et al., 13 Jun 2025). A plausible implication is that “CoMAS” now spans a spectrum from centralized orchestration-plus-adaptation to symmetric decentralized co-evolution, rather than a single architectural template.

Third, evaluation is shifting from static benchmark success to open-ended system properties. “Static Sandboxes Are Inadequate” argues that predefined tasks, limited dynamics, and rigid evaluation criteria suppress exactly the phenomena relevant to societal complexity. It proposes evaluation dimensions such as exploration capacity, adaptation and learning, emergent innovation, societal stability and alignment, and memory and consistency, including Shannon entropy

$\delta = H(p^\infty),$0

as a diversity measure (Chen et al., 15 Oct 2025). This position paper does not provide a full formal CoMAS theory, but it sharply states the field’s next challenge: balancing stability, diversity, evaluation of unexpected behaviors, and scaling to greater complexity (Chen et al., 15 Oct 2025).

Open problems recur across the surveyed work. PE-MA highlights asynchronous communication, larger agent populations, and integration with foundation models (Deng et al., 13 Jun 2025). “When Agents Evolve, Institutions Follow” calls for a meta-governance layer that can reselect and reconfigure $\delta = H(p^\infty),$1 online (Fei et al., 30 Apr 2026). HRCL identifies fully decentralized training as future work (Qin et al., 22 Sep 2025). Meta-Team explicitly leaves infrastructure evolution and base-model parameter updates to future systems (Hao et al., 28 May 2026). Taken together, these directions suggest that the next generation of CoMAS will likely combine dynamic topology, adaptive institutional scaffolds, richer environment mutation, and multi-timescale learning.

CoMAS is therefore best understood as a general paradigm in which the relevant units of adaptation are not fixed in advance. Depending on the formulation, these units may be agent states, shared adapters, curricula, prompts, routes, communication graphs, constitutions, or environments. The unifying principle is that effective multi-agent intelligence arises when these coupled components are allowed to improve together rather than being optimized in isolation.