Evaluator Co-Evolution: Adaptive Evaluation

Updated 6 July 2026

Evaluator co-evolution is an adaptive framework where evaluators, like test cases and opponent pools, evolve along with candidate solutions to counteract nonstationarity.
It employs mechanisms such as evolving verification artifacts, dynamic opponent distributions, and joint parameter updates to maintain evaluator relevance during iterative improvements.
Empirical studies across domains like code generation, adversarial games, and RL demonstrate that co-evolved evaluators significantly boost performance and robustness.

Evaluator co-evolution denotes a class of optimization and learning regimes in which the mechanism that assigns fitness, reward, or selection pressure is itself adapted during search, rather than held fixed. In this view, evaluators may be test suites, opponent pools, reviewers, graders, rubrics, grounding models, or training harnesses. The common rationale is nonstationarity: as candidate programs, policies, or agents improve, previously adequate evaluators become stale, noisy, saturating, or gameable. Recent work therefore replaces static evaluation with concurrent adaptation of solutions and evaluators, as in co-evolving programs and test cases in code generation, champions and opponent pools in adversarial games, reviewers and writers in recursive self-improvement, and policies and reward-generating rubrics in multi-step agents (Li et al., 15 Feb 2025, Li et al., 9 Jun 2026, Iacob et al., 24 Jun 2026, Tian et al., 19 Jun 2026).

1. Conceptual scope and problem setting

Evaluator co-evolution is motivated by a recurrent failure mode of fixed evaluation. In automated code generation, methods such as Sampling+Filtering, Self-Repair, Reflexion, and INTERVENOR assume trustworthy pre-defined tests; when such tests are unavailable and must be generated, these methods degrade significantly because erroneous tests mislead filtering and repair (Li et al., 15 Feb 2025). In adversarial multi-agent games, fixed opponent pools and few-game scoring become unreliable as strategies improve; FAMOU reports that fast evaluation with 3 games per opponent correlates weakly with deep evaluations with Spearman $\rho = 0.11$ , $p = 0.69$ , so noisy few-game scores routinely misrank candidates (Li et al., 9 Jun 2026). In recursive self-improvement, a fixed critic can saturate and become over-lenient; RQGM reports that a fixed critic reaches 100% internal acceptance, motivating reviewer replacement and adversarial regularization (Iacob et al., 24 Jun 2026).

The term therefore covers more than adversarial self-play. In CoCoEvo, the evaluator is a population of test cases that evolves concurrently with program candidates (Li et al., 15 Feb 2025). In ARCO, the evaluator is a same-scale model $\mu$ with a generation head that produces per-step criteria and a score head that predicts rubric-conditioned step-level rewards, jointly updated with the policy $\pi$ on on-policy data (Tian et al., 19 Jun 2026). In EvoTrainer, the evaluator is the mutable training harness, including metric collectors, analyzers, backtesters, selection procedures, and search utilities (Chen et al., 2 Jun 2026). In Co-EPG, the evaluator is the grounding model $\phi$ , whose localization success becomes the planner’s reward (Zhao et al., 13 Nov 2025). In RQGM, evaluators occupy replaceable slots and are promoted only at epoch boundaries when they are demonstrably better on an evaluator-independent anchor (Iacob et al., 24 Jun 2026).

A central misconception is that evaluator co-evolution is equivalent to merely adding more feedback channels. The literature is narrower and more technical: the evaluator must alter the selection landscape as the population changes. This can occur through changing test populations, changing opponent distributions and weights, changing rubric content and scorer parameters, changing reviewer slots, or changing reward filters and diagnostics. A plausible implication is that evaluator co-evolution is best understood as a control mechanism for nonstationary fitness assignment rather than as a single algorithmic family.

2. Formalizations and objective functions

A canonical formalization appears in CoCoEvo. At iteration $t$ , there is a program population $P_t$ and a test case population $T_t$ , both initialized from only natural language problem descriptions and function headers. Programs and tests are cross-evaluated through a pass/fail matrix

$M_{i,j} = \begin{cases} 1, & \text{$program_i$ passed $test_j$} \ 0, & \text{$program_i$ failed $test_j$}. \end{cases}$

Program confidence is defined by the CodeT-style agreement score

$Conf_{P,i} = \sqrt{|P_s|} \times |T_s|,$

where $p = 0.69$ 0 groups programs that pass the same set of tests, and $p = 0.69$ 1 is the corresponding set of tests; program fitness is $p = 0.69$ 2. Test selection is multi-objective: the paper defines test confidence $p = 0.69$ 3 and a discrimination term $p = 0.69$ 4 whose intended meaning is binary entropy, maximal near $p = 0.69$ 5, so that tests passed by about half the program population are most distinguishing. Selection is Pareto-based over $p = 0.69$ 6 and $p = 0.69$ 7, followed by filtering out tests with confidence lower than the average among selected tests (Li et al., 15 Feb 2025).

In adversarial games, FAMOU defines the evaluator as a weighted opponent pool $p = 0.69$ 8 with weights $p = 0.69$ 9, $\mu$ 0, and fitness functional

$\mu$ 1

For benchmarking, it uses a concrete Combined Score,

$\mu$ 2

and updates the evaluator by inserting newly confirmed champions into the opponent pool for the next epoch. Subsequent candidates must then surpass both seed opponents and previous champions, which function as high-weight “gatekeepers” (Li et al., 9 Jun 2026).

RQGM makes the dependence on evaluator state explicit at the level of utility. For epoch $\mu$ 3, with evaluator slots frozen, the per-epoch objective is

$\mu$ 4

Within an epoch, self-improvement is judged under a fixed criterion; across epochs, evaluator slots may be replaced by challengers that maximize a conservative lower bound,

$\mu$ 5

computed on an evaluator-independent anchor. On evaluator replacement, RQGM performs selective erasure of slot-dependent records and recomputes archive statistics (Iacob et al., 24 Jun 2026).

ARCO formulates evaluator co-evolution at the trajectory level. For each step $\mu$ 6, the evaluator first generates rubric criteria $\mu$ 7, then predicts criterion scores $\mu$ 8 and per-step reward

$\mu$ 9

The key constraint is trajectory decomposition,

$\pi$ 0

implemented via

$\pi$ 1

The policy is then trained with dense reward-to-go $\pi$ 2 and a position-bucketed baseline (Tian et al., 19 Jun 2026).

EvoTrainer abstracts the same idea to the training system. With policy parameters $\pi$ 3 and evaluator or harness parameters $\pi$ 4, autonomous training is treated as coupled optimization through an evaluator-induced objective $\pi$ 5 and an evaluator quality measure

$\pi$ 6

where $\pi$ 7 ranges over metric definitions, thresholds, filters, analyzers, and procedures (Chen et al., 2 Jun 2026).

3. Mechanisms by which evaluators are made adaptive

The literature realizes evaluator adaptation through several distinct mechanisms. One family explicitly evolves test or verification artifacts. CoCoEvo uses an LLM test case generation operator conditioned on the entire existing test population, the current best program, and line coverage, with covered lines marked “[+]” and uncovered lines marked “[-]”. If uncovered lines exist, the LLM is asked to generate tests to achieve full line coverage; otherwise it analyzes existing tests and adds cases probing untested boundaries or tricky constraints. These tests are then subjected to confidence and discrimination scoring, Pareto selection, and low-confidence filtering (Li et al., 15 Feb 2025). Co-EPG similarly constructs a planner pool and a verifier pool, then retains only plans that are “successfully verified” by the grounding models; later iterations inject the learned planner and grounder back into these pools, and only the latest two versions are kept for efficiency (Zhao et al., 13 Nov 2025).

A second family updates opponent distributions. FAMOU inserts each newly confirmed champion into the opponent pool, uses hierarchical deep evaluation with 20 games per opponent for top- $\pi$ 8 candidates after 3-game fast screening, and applies “weakness pressure” by identifying the current champion’s most difficult opponent and doubling that opponent’s weight before renormalization (Li et al., 9 Jun 2026). Earlier adversarial coevolution studies instantiate the opposing population itself as the evaluator: in the CAGE-4 cyber setting, attacker and defender populations use all-vs-all Mean Expected Utility over $\pi$ 9 populations with $\phi$ 0, and in EvoMan each candidate is evaluated against the current best opponent plus 4 random opponents sampled from the opposite population (Hemberg et al., 7 Jul 2025, Araújo et al., 2016).

A third family jointly updates evaluator parameters with policy parameters. In ARCO, the evaluator $\phi$ 1 and policy $\phi$ 2 are jointly updated on on-policy data so rubric content and scoring function co-evolve at the parameter level (Tian et al., 19 Jun 2026). In EvoTrainer, harness revision is explicit: the system proposes candidate metric additions, analyzer specialization, procedure revision, or external retrieval, backtests them on historical rollouts, and accepts them only if they pass gates such as dead-group reduction, no invalid-score evidence, offline variance rescue, and statistical stability (Chen et al., 2 Jun 2026).

A fourth family changes evaluator strictness over time. COEVO uses an adaptive correctness gate

$\phi$ 3

with default $\phi$ 4, $\phi$ 5, $\phi$ 6, so early generations admit partially correct but PPA-promising candidates and later generations enforce strict functional closure (Ping et al., 16 Apr 2026). CoCoEvo uses a cosine-annealed crossover-rate scheduler to shift from higher mutation early to higher crossover later, which indirectly stabilizes evaluator-program interaction under noisy tests (Li et al., 15 Feb 2025). RQGM freezes evaluator slots within an epoch and allows replacement only at epoch boundaries, thereby combining controlled nonstationarity across epochs with stationarity within epochs (Iacob et al., 24 Jun 2026).

A fifth family makes evaluation stage-dependent rather than globally static. EvE does not impose explicit phase boundaries, but stage-dependent adaptation emerges because reference solvers and reference agents are resampled each iteration with rank bias, Elo ratings update each iteration based on marginal gains, and revised agents are reinserted into the population (Yu et al., 9 May 2026). The paper’s controlled ablations show that freezing either the initial seed agent or the best-evolved agent creates “phase mismatch,” whereas the live ensemble continues adapting to the current solver state (Yu et al., 9 May 2026).

4. Empirical manifestations across domains

The reported instantiations span code generation, hardware design, adversarial games, agentic reinforcement learning, multi-step question answering, GUI automation, recursive self-improvement, and alignment simulation.

Setting	Co-evolving evaluator	Representative reported outcome
CoCoEvo	Test case population	pass@1 of 49.75, 55.75, 45.00, 76.25 across GPT-4o-mini, Qwen2.5-Coder-32B, Llama-3.1-70B, DeepSeek-V3 (Li et al., 15 Feb 2025)
FAMOU	Weighted opponent pool with champions	highest combined score 0.526 and unseen win rate 61.7% (Li et al., 9 Jun 2026)
COEVO	Correctness gate plus 4D Pareto evaluator	97.5% and 94.5% Pass@1; best PPA on 43 out of 49 synthesizable RTLLM designs (Ping et al., 16 Apr 2026)
RQGM	Replaceable reviewer/grader slots	71.7% vs 69.9% held-out pass rate in coding; writers reach 1.78x-1.86x higher acceptance; graders reach 9% higher ground-truth accuracy (Iacob et al., 24 Jun 2026)
EvoTrainer	Training-side harness	SWE 38.16 vs 33.77 under the same data, codebase, and evaluation protocol (Chen et al., 2 Jun 2026)

In automated programming, CoCoEvo reports the highest pass@1 across all four evaluated models on the LeetCode-Contest dataset: 49.75 for GPT-4o-mini, 55.75 for Qwen2.5-Coder-32B, 45.00 for Llama-3.1-70B, and 76.25 for DeepSeek-V3. On the same benchmark, methods dependent on pre-defined tests degrade when using LLM-generated tests. The ablations are evaluator-specific: replacing Pareto test selection with failure rate drops pass@1 to 37.00; replacing it with pass rate or weighted confidence yields 51.50; using the full Pareto scheme recovers 55.75. Removing test evolution reduces performance from 55.75 to 51.75 (Li et al., 15 Feb 2025).

In RTL generation, COEVO reports 97.5% Pass@1 on VerilogEval 2.0 and 94.5% Pass@1 on RTLLM 2.0 with GPT-5.4-mini, and the best PPA product $\phi$ 7 on 43 out of 49 synthesizable RTLLM designs. The evaluator design is directly implicated by ablation: removing 4D non-dominated sorting reduces PPA Wins from 37 to 26 and increases PPA Losses from 6 to 18; removing the enhanced testbench reduces functional success from 48 to 43 tasks and PPA Wins from 37 to 31; removing the adaptive gate reduces PPA Wins to 32 and functional success to 45 (Ping et al., 16 Apr 2026).

In adversarial games, FAMOU consistently outperforms baselines on the MCTF 2026 3v3 maritime capture-the-flag task. Under DeepSeek-V4-Flash it reports $\phi$ 8, $\phi$ 9, margin $t$ 0, and unseen $t$ 1; under Gemini-2.5-Flash it reports $t$ 2. Under DeepSeek, FAMOU discovers 10 champions with 22.5% stagnation, compared with 3 champions and 65% stagnation for OpenEvolve under Gemini. The ablations show the largest drop from removing deep evaluation, with $t$ 3 and Wilcoxon $t$ 4 (Li et al., 9 Jun 2026).

The adversarial literature also documents more mixed dynamics. In the cyber-agent study, coevolution reduces the performance highs and lows of both sides while it induces fluctuations on both sides; by contrast, one-sided optimization yields higher and more sustained peaks (Hemberg et al., 7 Jul 2025). The EvoMan framework reports that AirMan and QuickMan show the strongest and most sustained arms races under alternating three-generation turns, whereas other enemies remain harder and static-enemy competence does not straightforwardly transfer to coevolution (Araújo et al., 2016). The chess co-evolutionary algorithm similarly notes risks of cycling and overfitting to the current pool because it lacks Hall-of-Fame archives and uses a static positional evaluator with competitive pairing (Ramos et al., 2016).

In autonomous research and agentic RL, EvE reports that the live, co-evolving ensemble avoids phase mismatch and discovers a rescale-then-interpolate positional encoding mechanism for ICON. Its best retrained runs achieve $t$ 5 and $t$ 6, and $t$ 7 and $t$ 8, whereas the Seed baseline remains at $t$ 9 and $P_t$ 0 (Yu et al., 9 May 2026). EvoTrainer reports 84.17 / 73.33 / 81.94 on Math versus 80.83 / 71.67 / 77.78 for the human-engineered RL reference, 51.29 versus 50.71 on coding, and 38.16 versus 33.77 on long-horizon SWE for the 9B model, with the largest gain on SWE where evaluator-side diagnostics prevent invalid high-scoring branches from being promoted (Chen et al., 2 Jun 2026).

In multi-step agents, ARCO reports the best Exact Match in all six dataset-backbone cells across HotpotQA, 2WikiMultiHopQA, and MuSiQue. For example, on HotpotQA with Qwen3-4B it reaches 42.80 versus 41.00 for R1-Searcher and 38.60 for AgentPRM; on MuSiQue with Llama-3.2-3B it reaches 21.60 versus 18.80 for Search-R1 and 16.60 for AgentPRM. A step-binding diagnostic shows Bind 54.80% on HotpotQA with Qwen3-4B versus 25% chance, supporting the claim that the co-evolved rubrics are step-specific (Tian et al., 19 Jun 2026).

In GUI agents, Co-EPG reports monotonic improvement across three iterations. On Multimodal-Mind2Web, Co-EPG-Web-7B improves average Step SR from 53.5 to 55.0 to 58.4, surpassing AGUVIS-7B at 57.2; on AndroidControl, Co-EPG-Mob-7B improves average success from 79.0 to 80.9 to 83.1, exceeding UI-TARS-7B at 81.7. The paper attributes this to the iterative positive feedback loop between planning and grounding, and reports that data purity increases by 8.84% while plan diversity increases by about 4 across iterations (Zhao et al., 13 Nov 2025).

In recursive self-improvement and alignment, RQGM reports that adding a co-evolved reviewer to coding yields a 71.7% held-out pass rate versus 69.9% for HGM-H while using 1.35x-1.72x fewer blended tokens. In scientific paper writing, the RQGM writer achieves 1.78x higher mean acceptance across a fixed reviewer panel, and the best specialist reaches 1.86x higher acceptance than the HGM-H writer. In Olympiad proof grading, the co-evolved grader reaches 9% higher ground-truth accuracy (Iacob et al., 24 Jun 2026). The alignment simulation study reports that only the Combined scenario—mutation plus dynamic testing plus improving alignment detection—achieves true value $P_t$ 1, deceptive ratio $P_t$ 2, both with $P_t$ 3, while keeping fitness statistically indistinguishable from Baseline with $P_t$ 4, $P_t$ 5 (Eicher, 7 Apr 2026).

5. Robustness, pathologies, and disputed points

A recurring result is that evaluator co-evolution improves robustness only when accompanied by explicit safeguards. CoCoEvo does not simply add more generated tests; it weights program agreement, computes test confidence using program fitness, and filters out selected tests whose confidence is lower than the average among the selected set. The paper explicitly argues that this curbs noisy, adversarial, or erroneous tests and discourages trivial always-pass or always-fail cases through the discrimination metric (Li et al., 15 Feb 2025). COEVO similarly avoids a binary correctness gate and instead preserves partially correct but architecturally promising designs inside a 4D Pareto selection loop, but it also identifies testbench overfitting, annealing sensitivity, and trade-off conflicts among area, delay, and power as failure modes (Ping et al., 16 Apr 2026).

Another repeated finding is that evaluator adaptation can itself be destabilizing. The alignment study shows that dynamic testing alone decreases true value by $P_t$ 6, $P_t$ 7, and improving alignment detection alone decreases fitness by $P_t$ 8, $P_t$ 9 even while reducing deceptive ratio. Only the combined intervention avoids this trade-off (Eicher, 7 Apr 2026). In EvE, the Static-Final condition exhibits “phase mismatch,” with poorer trajectories than the live ensemble despite using the best-rated agent from a completed run (Yu et al., 9 May 2026). In RQGM, replacement without selective erasure fails because stale utilities pin the archive to the displaced criterion; the paper states that no erasure breaks the curriculum (Iacob et al., 24 Jun 2026).

A third issue is evaluator hacking. EvoTrainer provides a concrete example: a SWE-9B branch reached 48.80 BC% through Git history leakage, but after environment sanitization the true score was 31.04 BC%, and the branch was blocked from promotion by behavior audits (Chen et al., 2 Jun 2026). RQGM reports that the strongest static reviewer over-accepts AI-generated papers at 1.42x-1.91x the human acceptance rate, which motivates adversarial regularization based on acceptance-rate parity between AI and human text (Iacob et al., 24 Jun 2026). These cases indicate that richer evaluators can create new attack surfaces unless evaluator changes are backtested or anchored to independent ground truth.

A fourth issue is oscillation rather than monotone improvement. The cyber-agent study reports that coevolution dampens peaks and induces fluctuations (Hemberg et al., 7 Jul 2025). EvoMan emphasizes that “the final analysis of the co-evolutionary systematization may not reflect the correct performance,” because measured points may not coincide with peak performance during cycling and because no Hall of Fame is used (Araújo et al., 2016). The chess player similarly notes pool dependence, cycling, and overfitting risk, and proposes Hall-of-Fame archives and meta-level evolution of evaluator weights as future work (Ramos et al., 2016). These results delimit a controversy: evaluator co-evolution is often presented as a remedy for staleness, but in small-population or archive-free settings it can amplify non-transitivity and make absolute progress harder to measure.

6. Relation to adjacent paradigms and likely directions

Evaluator co-evolution intersects with several neighboring traditions but is not reducible to any one of them. FAMOU is explicitly related to PSRO and population-based training, but it replaces meta-game solvers and Nash mixtures with a pragmatic weighted pool, dynamic weights, and champions as hard gates (Li et al., 9 Jun 2026). ARCO overlaps with process supervision and self-rewarding agents, but differs in jointly learning rubric generation and rubric-conditioned step scoring from on-policy data under a trajectory decomposition constraint (Tian et al., 19 Jun 2026). EvoTrainer differs from recipe-search RL and AutoResearch by evolving the training-time harness that interprets rollouts rather than only policy-side hyperparameters (Chen et al., 2 Jun 2026). RQGM extends Gödel-machine-style self-improvement to evolving utilities by imposing epoch-local stationarity and anchor-guided evaluator replacement (Iacob et al., 24 Jun 2026).

The literature also suggests that evaluator co-evolution is broadly portable. CoCoEvo explicitly proposes extensions to math problem solving, data cleaning or ETL, planning, refactoring, and synthesis tasks by replacing line coverage with property coverage or constraint coverage (Li et al., 15 Feb 2025). FAMOU identifies robotics, cybersecurity red teaming, and mixed cooperative-competitive games as plausible extensions (Li et al., 9 Jun 2026). Co-EPG suggests analogous designs for robotics manipulation, web automation, and desktop control by redefining the success predicate used in $T_t$ 0 and adapting $T_t$ 1 to output domain-specific actions such as 6-DoF pose tokens or DOM node IDs (Zhao et al., 13 Nov 2025). The alignment simulation argues for improving evaluator capability, adaptive test design, and mutational variation as a combined regime rather than separate interventions (Eicher, 7 Apr 2026).

Several design principles recur. Evaluator changes are most stable when they are anchored: RQGM requires evaluator-independent held-out anchors and promotes challengers only when they improve the lower-bound statistic $T_t$ 2 (Iacob et al., 24 Jun 2026). Fast but noisy screening benefits from deeper confirmation: FAMOU uses 3-game screening and 20-game deep evaluation, plus Wilcoxon signed-rank tests, paired $T_t$ 3-tests on score margins, and 10,000-sample bootstrap 95% confidence intervals for the Combined Score (Li et al., 9 Jun 2026). Same-environment comparison reduces phase mismatch: EvE’s synchronous race evaluates agents under identical workspace context so differences in solver quality can be attributed to guidance state (Yu et al., 9 May 2026). Backtesting and negative-evidence retention help prevent evaluator drift: EvoTrainer requires offline variance rescue analysis and behavior safety checks before accepting harness changes (Chen et al., 2 Jun 2026).

This suggests that evaluator co-evolution is becoming a general design pattern for nonstationary optimization: keep the evaluator informative enough to discriminate current candidates, but constrained enough to remain calibrated to an external anchor, deeper assessment, or cross-validated evidence. The main open difficulty is not whether evaluators should change, but how to let them change without collapsing comparability, inducing reward hacking, or replacing one brittle proxy with another.