Beyond Static Evaluation: Co-Evolutionary Mechanisms for LLM-Driven Strategy Evolution in Adversarial Games

Abstract: Recent advances in LLM-driven code evolution have enabled automated discovery by iteratively generating and improving programs. However, applying these methods to adversarial multi-agent games introduces a fundamental challenge: the evaluation landscape shifts as strategies improve, causing fixed evaluators to become unreliable and evolution to stagnate. We propose three mechanisms to address this challenge: evaluator co-evolution, which incorporates discovered champions into the opponent pool; hierarchical deep evaluation, which replaces noisy few-game scores with statistically reliable assessments; and weakness pressure, which dynamically up-weights the most difficult opponents to break through plateaus. We implement these mechanisms within FAMOU, a framework built upon the same foundation-model code-evolution paradigm as OpenEvolve and ShinkaEvolve. On the MCTF 2026 3v3 maritime capture-the-flag task, FAMOU consistently outperforms both baselines under two backbone LLMs, achieving the highest combined score (0.526) and the best generalization to unseen opponents (61.7% win rate), while ablations confirm that each mechanism contributes to performance. Notably, the LLM mutation process generates tactical structures entirely absent from the seed strategies -- including lookahead search and adaptive interception -- demonstrating that code-level evolution can produce nontrivial algorithmic innovations in adversarial settings. The FAMOU-evolved strategy further achieved 1st place in the hardware round-robin and 3rd in simulation at the AAMAS 2026 MCTF Competition, validating its real-world transferability. The optimized implementation and corresponding evaluation codes developed through our evolutionary process are available at: https://github.com/1xiangliu1/FAMOU-CoEvo

• The paper demonstrates a novel co-evolutionary LLM-driven framework, integrating evaluator co-evolution, hierarchical deep evaluation, and weakness pressure to evolve robust strategies.
• The methodology leverages LLM-based code mutations in complex adversarial game settings, significantly improving win rates and generalization over established baselines.
• Empirical results highlight FAMOU's breakthrough performance with superior win rates, real-world sim-to-real transfer, and the emergence of innovative tactical mechanisms.

Co-Evolutionary LLM-Driven Strategy Evolution in Adversarial Multi-Agent Games

Introduction and Problem Context

The paper "Beyond Static Evaluation: Co-Evolutionary Mechanisms for LLM-Driven Strategy Evolution in Adversarial Games" (2606.10389) presents a systematic study of co-evolutionary mechanisms for LLM-driven code-level strategy evolution in non-stationary, adversarial multi-agent game environments, specifically within the Maritime Capture-The-Flag (MCTF) 3v3 domain. Adversarial games involving multiple agents present unique challenges: the opponent population is non-stationary, the combinatorial action space grows exponentially with team size, and non-transitive dominance makes static evaluation unreliable. The work builds upon recent advances in LLM-supported program synthesis and code-level policy mutations, extending the classical foundation of program evolution to the much harder regime of adversarial, multi-agent, and non-transitive games.

Framework and Co-Evolutionary Mechanisms

To address the brittleness and staleness of evaluation in evolving populations, the authors introduce three central mechanisms: evaluator co-evolution, hierarchical deep evaluation, and weakness pressure. These are implemented within the FAMOU framework, itself a successor to previous work on LLM-driven code evolution. The architecture is illustrated as follows:

Figure 1: FAMOU self-evolving coding-agent framework for adversarial strategy evolution, incorporating LLM-driven mutation, dynamic co-evolution of the opponent pool, deep evaluation for selection, and targeted weakness pressure for accelerated learning.

First, evaluator co-evolution continually augments the evaluation pool by including newly discovered "champion" strategies, so evolved candidates are always tested against prior high-performing solutions. This mitigates the risk of overfitting to a fixed or narrow opponent set. Second, hierarchical deep evaluation addresses the noise in stochastic game environments by distinguishing fast, low-fidelity evaluation (for throughput) and slow, high-fidelity evaluation (for actual selection and archiving), providing statistically meaningful performance signals that reduce premature convergence. Third, the weakness pressure mechanism adaptively increases the evaluation weight of the most problematic (difficult-to-defeat) opponents, driving targeted improvement rather than untargeted or purely fitness-proportional selection. The underlying mutation process operates on full program source files (500-1700 lines), utilizing LLMs to propose semantically meaningful structural changes, rather than isolated token-level edits.

Experimental Methodology

A comprehensive empirical suite evaluates the framework and ablates its core components. The main experimental environment is the MCTF 2026 (3v3) competition, featuring a 160m × 80m field and non-trivial strategies for maximization of capture scores rather than mere win rate. The performance is measured not only by overall win rate, but also by scoring margin and robustness versus unseen held-out opponents. Baseline comparisons include two state-of-the-art program evolution frameworks: OpenEvolve and ShinkaEvolve.

All methods are subject to identical strategy seeds, iteration budgets, backbone LLMs (DeepSeek-V4-Flash and Gemini-2.5-Flash), and benchmark protocols that include both seen and held-out opponents for evaluation of generalization. Ablation studies systematically disable co-evolution, deep evaluation, and weakness pressure either individually or in combination, providing insight into the contribution of each mechanism. Rigorous statistical significance testing (e.g., Wilcoxon signed-rank, paired $t$-test, and bootstrapped CIs) is carried out to ensure result reliability.

Empirical Results and Analysis

FAMOU achieves statistically significant improvements over both baselines across all LLM backbones and evaluation metrics. Under DeepSeek-V4-Flash, FAMOU obtains a Combined Score (CS) of 0.526, corresponding to a 68.0% win rate and +0.83 average score margin, contrasted with ShinkaEvolve (CS=0.437, WR=56.1%) and OpenEvolve (CS=0.382, WR=50.0%). Notably, the generalization gap to unseen test opponents is substantially reduced: FAMOU achieves a 61.7% win rate on held-out adversaries, where baselines fall to 34.2% (ShinkaEvolve) and 31.2% (OpenEvolve), illustrating the tangible advantage of co-evolution in maintaining robustness.

Learning curves reveal that all baselines stagnate well before the end of the run, while FAMOU variants demonstrate continued improvement throughout. Analysis of program discovery trajectories shows that FAMOU produces frequent "breakthroughs"—large, sudden score jumps—throughout the training horizon, reflecting an effective escape from fitness plateaus.

Ablation analyses show deep evaluation to be the single most critical component (ΔCS = –0.136), followed by weakness pressure and co-evolution (ΔCS = –0.097 and –0.071, respectively). Mechanistic interactions are non-additive: the full FAMOU framework outperforms any ablated configuration, and the sum of drop-offs is greater than the observed gap between full and baseline, suggesting interplay between mechanisms.

Cross-evaluation experiments further highlight that FAMOU-evolved strategies not only dominate in aggregate benchmarking but also exhibit strong pairwise robustness against all baseline champions, an essential property given the non-transitivity inherent in the game dynamics.

Emergence of Novel Tactics via LLM-Driven Evolution

A critical qualitative result is the emergence of novel tactical structures not present in any seed code. LLM mutation produced mechanisms such as single-step lookahead (H-DWA), adaptive role locking (A-Lock), and EWMA-based interception (K-Filter), indicating that LLMs can serve as powerful mutation operators even in codebases exceeding 1,000 lines. Specific examples include full program regime changes (e.g., static to dynamic role assignment, waypoint-based avoidance, and multi-lane patrol for defenders), underscoring the capacity of LLMs for real architectural innovation rather than mere parameter tuning.

Implications, Limitations, and Future Perspectives

This work provides strong evidence that LLM-based code evolution—when coupled to adaptive, co-evolving evaluation processes and statistically robust selection—can exceed the performance of both static evaluation and prior program-evolution frameworks in adversarial settings. An important implication is that evaluator adaptivity is at least as significant as strategy diversity in dynamic environments; rigid benchmarks inevitably induce overfitting and bias evolutionary search toward locally rather than globally robust solutions.

Practically, the results extend beyond MCTF: the FAMOU-evolved strategy achieved first place in the hardware round-robin and third in simulation at the AAMAS 2026 MCTF Competition, indicating real-world sim-to-real transfer. However, the experiments are limited to a single domain and largely single-seed ablations; extension to other complex games (e.g., RoboCup, StarCraft II micromanagement) and systematic multi-objective search remains an open direction.

Theoretically, these findings suggest that co-evolutionary dynamics—long studied in classical evolutionary computation but largely neglected in LLM-based program synthesis—deserve renewed attention. There is a clear path forward for multi-objective and Pareto-frontier optimization, adversarial environment co-design, and online adaptation in embodied robotics and AGI-aligned software systems.

Conclusion

The integration of evaluator co-evolution, deep evaluation, and targeted weakness pressure within an LLM-driven code evolution framework demonstrably advances the learning of robust, high-performance policies in adversarial multi-agent games. The emergence of qualitatively novel tactics, coupled with superior quantitative results and generalization, supports the claim that appropriately designed evaluation pressure is a necessary complement to LLM-based program synthesis. These findings highlight new opportunities for automated strategy discovery and, more generally, for self-improving software systems under complex, shifting evaluation landscapes.