Coevolutionary Multimodal Multi-Agent Systems

• Coevolutionary multimodal multi-agent systems are computational frameworks where heterogeneous agents evolve their morphologies and policies through competitive and cooperative interactions while integrating multiple information modalities.
• These systems employ techniques like parallel self-play, co-evolution of attackers and defenders, and modular architectures to optimize both individual strategies and team-level performance.
• Recent research demonstrates emergent role specialization, adaptive robustness, and quantifiable performance gains in metrics such as win rates and safety scores through rigorous coevolutionary training.

A coevolutionary multimodal multi-agent system (CMMAS) is a computational framework wherein multiple agents, potentially with heterogeneous architectures or capabilities, simultaneously evolve their parameterizations and behavioral policies through competitive or cooperative interaction, and where agent decision-making leverages multiple information modalities. Evolutionary dynamics are shaped by reward signals that may depend on agent interactions and can act both on individual parameters (e.g., morphology, control, language generation) and on the mechanisms underlying inter-agent communication and multimodal perception. Recent work demonstrates that the coevolutionary paradigm can generate emergent adaptive behaviors, facilitate role specialization, internalize safety, and solve complex tasks that require integrating multimodal cues in competitive and cooperative environments (Huang et al., 2024, Pan et al., 5 Aug 2025, Rollins et al., 2017, Yu et al., 29 Sep 2025).

1. System Architectures: Structural and Functional Diversity

Coevolutionary multimodal multi-agent systems span a range of architectural patterns, typically characterized by modular agent compositions and structured communication protocols.

• In the physical intelligence setting, such as CompetEvo, agents possess parameterized morphologies (e.g., leg segment lengths and girths for simulated creatures with ant, bug, or spider skeletons), and multi-layer perceptron controllers whose policy inputs concatenate proprioceptive sensor data, observations of opponents, and morphology encodings (Huang et al., 2024). The system evolves both the “body” and “brain”.
• In multimodal reasoning and safety settings (Evo-MARL, PhysicsMinions), each agent implements a variant of a large multimodal LLM (MLLM), equipped with text and vision encoders and accepting role embeddings to condition policy (Pan et al., 5 Aug 2025, Yu et al., 29 Sep 2025). Systems often follow a chain-of-expertise or pipeline architecture—e.g., problem analyst, solver, verifier in Evo-MARL, or Visual, Logic, and Review Studios in PhysicsMinions.
• Modular neural controllers with preference-based arbitration (multiple policy modules plus preference neuron) enable individual agents to dynamically switch behavioral modes, further broadening the behavioral capacity of the team in roles involving cooperation and role specialization (Rollins et al., 2017).

2. Coevolutionary Algorithms and Training Paradigms

CMMASs employ coevolutionary optimization, with key variants including:

• Parallel self-play with policy pools: As in CompetEvo, two (or more) agent policy pools evolve jointly via self-play. At each generation, agents select partners or opponents (often via δ-uniform sampling favoring recent entries) and update both morph-generators and tactical controllers using policy-gradient methods (e.g., PPO) on experience buffers aggregated from competitive rollouts (Huang et al., 2024).
• Co-evolution of attackers and defenders: Evo-MARL maintains parallel populations: a pool of defender policies (parameter-shared across all agents of a type) and a pool of adversarial prompts evolved by selection, mutation, and crossover. Evolutionary pressure is applied to attacker prompts via attack success rate, whereas defender agents are updated by RL on cumulative rewards composed of safety and task utility terms. The evolutionary cycle alternates between adversary pool refinement and defender gradient ascent (Pan et al., 5 Aug 2025).
• Cooperative coevolution with modularity: Multi-objective NEAT-based algorithms (e.g., MM-NEAT) evolve separate agent sub-populations, each representing a role. Fitness is assessed by evaluation of randomly composed teams over multiple runs and aggregated via Pareto ranking and crowding distance, combining individual- and team-level objectives to avoid premature convergence and support specialization (Rollins et al., 2017).
• Iterative refinement and dual-stage verification: PhysicsMinions introduces a coevolutionary iterative loop among its studios. Candidate solutions are repeatedly critiqued and revised through two verification stages (domain-specific and general), and only accepted if they pass CV consecutive rounds, while failures reset the candidate, ensuring rigorous self-correction (Yu et al., 29 Sep 2025).

3. Multimodality and Information Integration

CMMASs extend beyond single-modality (e.g., vision or text only) processing by explicitly integrating heterogeneous information streams:

• In Evo-MARL, each agent encodes both text (history, prompt, agent utterances) and image features (via vision encoder) as part of its observation. Role embeddings condition the policy network for agent-specific behavior (Pan et al., 5 Aug 2025).
• PhysicsMinions constructs a structured JSON representation of the visual component (e.g., plots, circuits, free-body diagrams), which is then explicitly concatenated with the problem statement and provided to reasoning modules, in contrast to relying solely on raw image-based embeddings. The review process further fuses domain-specific and general logic checks (Yu et al., 29 Sep 2025).
• In CompetEvo and MM-NEAT, although modalities are primarily physical (morphology and control), controllers can be considered "multimodal" by virtue of fusing proprioceptive, exteroceptive, and configuration encoding inputs (Huang et al., 2024, Rollins et al., 2017).

4. Fitness Functions, Evolutionary Pressure, and Objective Assignment

Reward structures in CMMASs are often explicitly multimodal and multiobjective.

• Mixed dense and sparse rewards: CompetEvo employs a curriculum-based schedule in which a weighted sum of dense locomotion rewards ($R_d$) and sparse game-outcome rewards ($R_s$) is used. The weighting factor $\kappa$ anneals from 1 (focusing initially on locomotion) to 0 (emphasizing competitive success) across generations (Huang et al., 2024).
• Task–safety objective composition: Evo-MARL's defender agents receive a composite episodic reward: $R_d = R^\text{safety} + R^\text{task}$, where $R^\text{safety}=+1$ for safe terminal output, $-1$ for unsafe, and $R^\text{task}=+0.5$/$-0.5$ for correct/incorrect problem solving. Adversarial prompt fitness is attack success rate (ASR), i.e., percentage of unsafe completions (Pan et al., 5 Aug 2025).
• Dual-level and modular objectives: MM-NEAT's multiobjective setup allows both individual-level (e.g., personal prey captures, distance to prey) and team-level (collective capture and distance objectives) selection pressure, avoiding reward dilution that penalizes specialized behavior such as blocking or herding. Pareto-based non-dominated sorting maintains diverse solutions (Rollins et al., 2017).
• Structured evaluation and verification: In PhysicsMinions, solution acceptance requires passing both domain-specific (physical units, constants) and general (completeness, logic, calculation) checks across $CV$ consecutive rounds. Performance is evaluated via exam-score and mean normalized score, quantified by thresholds for olympiad medals (Yu et al., 29 Sep 2025).

5. Emergence of Specialization, Robustness, and Complex Behavior

Coevolutionary multimodal systems consistently induce role emergence, behavioral diversification, and resilience:

• Role and tactic specialization: In predator–prey domains, distinct agent roles (Blocker, Herder, Aggressor) arise, with neural modularity allowing behavioral modules to encode complementary subpolicies (e.g., support/blocking vs. aggressive pursuit), dynamically selected at inference (Rollins et al., 2017).
• Morphological and strategic adaptation: In Competitive Morphology, evolved morphs outperform fixed baselines for both symmetric and asymmetric matchups, attaining 80–85% win rates after 1000 generations. Emergent tactics, such as “throwing,” “wrestling,” “standing,” and “defending,” are directly linked to morphology encoding and evolve via PPO-driven gradients (Huang et al., 2024).
• Internalized safety and adversarial robustness: Evo-MARL demonstrates that, by embedding safety objectives into the MARL loop and coevolving the attack pool, attack success rates fall by up to 22% and task accuracy rises by 5pp, with smaller (1.5B param) multi-agent systems achieving greater safety than untrained 7B parameter baselines (Pan et al., 5 Aug 2025).
• Multi-stage correctness verification: PhysicsMinions' dual-verifier loop ensures that only consistently correct and physically plausible solutions are accepted, resulting in rapid improvement on high-stakes benchmarks, including achieving first-ever open-source gold medals and surpassing 99% of human contestants on IPhO (Yu et al., 29 Sep 2025).

6. Experimental Results, Evaluation, and Limitations

CMMASs have demonstrated substantial advancements in both synthetic simulation and real-world reasoning contexts.

• CompetEvo: Evolved species display persistent morphological and tactical advantages, with win rate boosts of 15–30% over originals in asymmetric settings. However, task-specific failures (e.g., evo-bug vs ant in run-to-goal) indicate persistence of exploitable morph–strategy interactions (Huang et al., 2024).
• Evo-MARL: Attack success rates (lower is better) drop from 69% to 48% on HarmBench and from 22.6% to 17.4% on JailBreakV. MATH accuracy improves from 43% to 48% following coevolution, and parameter sharing allows linear scaling in agent number (Pan et al., 5 Aug 2025).
• MM-NEAT: Modular networks consistently outperform single-module variants, reaching near-perfect prey capture rates of 1.95–2.0 out of 2, with the Both objective scheme adding robustness across runs and selection pressures (Rollins et al., 2017).
• PhysicsMinions: Achieves gold-medal-level performance across all 7 Olympiads (evaluated by average exam score), with Pass@32 reaching 26.8/30 points on IPhO (ranked 4th out of 406 humans). Ablation studies indicate the necessity of structured visual input and dual-verification for top-tier results (Yu et al., 29 Sep 2025).
• Scalability: Parameter sharing, distributed rollouts, and constrained evolutionary pools facilitate tractable scaling. Limiting factors include compute requirements for large agent pools, non-stationarity in rapidly coevolving adversaries, and architectural rigidity (e.g., chain topologies, lack of topology evolution).

7. Extensions, Open Challenges, and Implications

CMMASs provide a foundation for further exploration in both technical and application-centric directions.

• Open challenges include extending agent coevolution to richer topological spaces (e.g., evolving agent network graphs or physical joint structure), balancing non-stationary evolutionary arms races, integrating semantic mutation and memory-augmented state, and enabling dynamic team composition (Huang et al., 2024, Pan et al., 5 Aug 2025).
• Cross-domain generalization as demonstrated by PhysicsMinions suggests that coevolutionary feedback can be effective in other STEM domains demanding high-level, multimodal reasoning. The structured modular approach is directly portable across evaluation-centric workflows (Yu et al., 29 Sep 2025).
• Multiobjective optimization and modularity: Balancing fine-grained (individual) and coarse-grained (team) rewards, along with neural modularity, is crucial for escaping the limitations of isolated reward signals; preserving specialization through evolutionary complexification prevents premature convergence and supports behavioral innovation (Rollins et al., 2017).
• Internalizing complex objectives (e.g., safety, robustness) into shared agent policies via coevolution reduces reliance on brittle post-hoc guard modules and supports scalable, distributed learning (Pan et al., 5 Aug 2025).

A plausible implication is that CMMASs will continue to underpin advances in domains requiring robust, adaptive, and interpretable teamwork, particularly where multiple information streams and emergent specializations are essential for optimal task performance.