---
title: 'Evaluator Co-Evolution: Adaptive Evaluation'
url: https://www.emergentmind.com/topics/evaluator-co-evolution
type: topic
---

# Evaluator Co-Evolution: Adaptive Evaluation

Evaluator co-evolution denotes a class of optimization and learning regimes in which the mechanism that assigns fitness, reward, or selection pressure is itself adapted during search, rather than held fixed. In this view, evaluators may be test suites, opponent pools, reviewers, graders, rubrics, grounding models, or training harnesses. The common rationale is nonstationarity: as candidate programs, policies, or agents improve, previously adequate evaluators become stale, noisy, saturating, or gameable. Recent work therefore replaces static evaluation with concurrent adaptation of solutions and evaluators, as in co-evolving programs and test cases in code generation, champions and opponent pools in adversarial games, reviewers and writers in recursive self-improvement, and policies and reward-generating rubrics in multi-step agents [2502.10802][2606.10389][2606.26294][2606.21262].

## 1. Conceptual scope and problem setting

Evaluator co-evolution is motivated by a recurrent failure mode of fixed evaluation. In automated code generation, methods such as Sampling+Filtering, Self-Repair, Reflexion, and INTERVENOR assume trustworthy pre-defined tests; when such tests are unavailable and must be generated, these methods degrade significantly because erroneous tests mislead filtering and repair [2502.10802]. In adversarial multi-agent games, fixed opponent pools and few-game scoring become unreliable as strategies improve; FAMOU reports that fast evaluation with 3 games per opponent correlates weakly with deep evaluations with Spearman $\rho = 0.11$, $p = 0.69$, so noisy few-game scores routinely misrank candidates [2606.10389]. In recursive self-improvement, a fixed critic can saturate and become over-lenient; RQGM reports that a fixed critic reaches 100% internal acceptance, motivating reviewer replacement and adversarial regularization [2606.26294].

The term therefore covers more than adversarial self-play. In CoCoEvo, the evaluator is a population of test cases that evolves concurrently with program candidates [2502.10802]. In ARCO, the evaluator is a same-scale model $\mu$ with a generation head that produces per-step criteria and a score head that predicts rubric-conditioned step-level rewards, jointly updated with the policy $\pi$ on on-policy data [2606.21262]. In EvoTrainer, the evaluator is the mutable training harness, including metric collectors, analyzers, backtesters, selection procedures, and search utilities [2606.03108]. In Co-EPG, the evaluator is the grounding model $\phi$, whose localization success becomes the planner’s reward [2511.10705]. In RQGM, evaluators occupy replaceable slots and are promoted only at epoch boundaries when they are demonstrably better on an evaluator-independent anchor [2606.26294].

A central misconception is that evaluator co-evolution is equivalent to merely adding more feedback channels. The literature is narrower and more technical: the evaluator must alter the selection landscape as the population changes. This can occur through changing test populations, changing opponent distributions and weights, changing rubric content and scorer parameters, changing reviewer slots, or changing reward filters and diagnostics. A plausible implication is that evaluator co-evolution is best understood as a control mechanism for nonstationary fitness assignment rather than as a single algorithmic family.

## 2. Formalizations and objective functions

A canonical formalization appears in CoCoEvo. At iteration $t$, there is a program population $P_t$ and a test case population $T_t$, both initialized from only natural language problem descriptions and function headers. Programs and tests are cross-evaluated through a pass/fail matrix
$$
M_{i,j} =
\begin{cases}
1, & \text{\(program_i\) passed \(test_j\)} \\
0, & \text{\(program_i\) failed \(test_j\)}.
\end{cases}
$$
Program confidence is defined by the CodeT-style agreement score
$$
Conf_{P,i} = \sqrt{|P_s|} \times |T_s|,
$$
where $P_s$ groups programs that pass the same set of tests, and $T_s$ is the corresponding set of tests; program fitness is $F_{P,i} = Conf_{P,i}$. Test selection is multi-objective: the paper defines test confidence $Conf_{T,j}$ and a discrimination term $Disc_{T,j}$ whose intended meaning is binary entropy, maximal near $p_j \approx 0.5$, so that tests passed by about half the program population are most distinguishing. Selection is Pareto-based over $Conf_T$ and $Disc_T$, followed by filtering out tests with confidence lower than the average among selected tests [2502.10802].

In adversarial games, FAMOU defines the evaluator as a weighted opponent pool $O = \{o_1,\dots,o_n\}$ with weights $w_i \ge 0$, $\sum_i w_i = 1$, and fitness functional
$$
F(c) = \sum_{i=1}^{n} w_i \cdot metric(c, o_i).
$$
For benchmarking, it uses a concrete Combined Score,
$$
CS = 0.7 \times WR + 0.3 \times \min(1.0, \max(0, margin)/5.0),
$$
and updates the evaluator by inserting newly confirmed champions into the opponent pool for the next epoch. Subsequent candidates must then surpass both seed opponents and previous champions, which function as high-weight “gatekeepers” [2606.10389].

RQGM makes the dependence on evaluator state explicit at the level of utility. For epoch $e$, with evaluator slots frozen, the per-epoch objective is
$$
U_e(\pi, E_e) = E_{x\sim\mathcal{D}_e}[u(\pi, E_e; x)].
$$
Within an epoch, self-improvement is judged under a fixed criterion; across epochs, evaluator slots may be replaced by challengers that maximize a conservative lower bound,
$$
BB_\epsilon(a) = I^{-1}_\epsilon(1+S, 1+F),
$$
computed on an evaluator-independent anchor. On evaluator replacement, RQGM performs selective erasure of slot-dependent records and recomputes archive statistics [2606.26294].

ARCO formulates evaluator co-evolution at the trajectory level. For each step $t$, the evaluator first generates rubric criteria $c_t \sim p_\mu(\cdot \mid s_t, a_t)$, then predicts criterion scores $d_t \in [-1,1]^K$ and per-step reward
$$
r_t = \frac{1}{K}\sum_{j=1}^{K} d_{t,j}.
$$
The key constraint is trajectory decomposition,
$$
\sum_{t=0}^{T-1} r_t \approx R(\tau),
$$
implemented via
$$
L_{\text{decomp}}(\mu) = E_{\tau\sim\pi}\Big[\Big(\sum_{t=0}^{T-1} r_t - R(\tau)\Big)^2\Big].
$$
The policy is then trained with dense reward-to-go $G_t = \alpha \sum_{k=t}^{T-1} r_k$ and a position-bucketed baseline [2606.21262].

EvoTrainer abstracts the same idea to the training system. With policy parameters $\theta$ and evaluator or harness parameters $\phi$, autonomous training is treated as coupled optimization through an evaluator-induced objective $J(\theta;\phi)$ and an evaluator quality measure
$$
E(\phi) = -L_{\text{err}}(\phi;\mathcal{Y}) - \lambda_{\text{over}}\Omega(\phi) + \lambda_{\text{var}}\Delta Var(\phi) + \lambda_{\text{gate}}\mathbf{1}\{\text{passes gates}\},
$$
where $\phi$ ranges over metric definitions, thresholds, filters, analyzers, and procedures [2606.03108].

## 3. Mechanisms by which evaluators are made adaptive

The literature realizes evaluator adaptation through several distinct mechanisms. One family explicitly evolves test or verification artifacts. CoCoEvo uses an LLM test case generation operator conditioned on the entire existing test population, the current best program, and line coverage, with covered lines marked “[+]” and uncovered lines marked “[-]”. If uncovered lines exist, the LLM is asked to generate tests to achieve full line coverage; otherwise it analyzes existing tests and adds cases probing untested boundaries or tricky constraints. These tests are then subjected to confidence and discrimination scoring, Pareto selection, and low-confidence filtering [2502.10802]. Co-EPG similarly constructs a planner pool and a verifier pool, then retains only plans that are “successfully verified” by the grounding models; later iterations inject the learned planner and grounder back into these pools, and only the latest two versions are kept for efficiency [2511.10705].

A second family updates opponent distributions. FAMOU inserts each newly confirmed champion into the opponent pool, uses hierarchical deep evaluation with 20 games per opponent for top-$k$ candidates after 3-game fast screening, and applies “weakness pressure” by identifying the current champion’s most difficult opponent and doubling that opponent’s weight before renormalization [2606.10389]. Earlier adversarial coevolution studies instantiate the opposing population itself as the evaluator: in the CAGE-4 cyber setting, attacker and defender populations use all-vs-all Mean Expected Utility over $N_R = N_B = 10$ populations with $R_{\text{rep}} = 2$, and in EvoMan each candidate is evaluated against the current best opponent plus 4 random opponents sampled from the opposite population [2507.05534][1604.00644].

A third family jointly updates evaluator parameters with policy parameters. In ARCO, the evaluator $\mu$ and policy $\pi$ are jointly updated on on-policy data so rubric content and scoring function co-evolve at the parameter level [2606.21262]. In EvoTrainer, harness revision is explicit: the system proposes candidate metric additions, analyzer specialization, procedure revision, or external retrieval, backtests them on historical rollouts, and accepts them only if they pass gates such as dead-group reduction, no invalid-score evidence, offline variance rescue, and statistical stability [2606.03108].

A fourth family changes evaluator strictness over time. COEVO uses an adaptive correctness gate
$$
\tau(t)=\theta_t=\theta_{\min}+(\theta_{\max}-\theta_{\min})\cdot (t/G)^\alpha,
$$
with default $\theta_{\min}=0.25$, $\theta_{\max}=1.0$, $\alpha=2.0$, so early generations admit partially correct but PPA-promising candidates and later generations enforce strict functional closure [2604.15001]. CoCoEvo uses a cosine-annealed crossover-rate scheduler to shift from higher mutation early to higher crossover later, which indirectly stabilizes evaluator-program interaction under noisy tests [2502.10802]. RQGM freezes evaluator slots within an epoch and allows replacement only at epoch boundaries, thereby combining controlled nonstationarity across epochs with stationarity within epochs [2606.26294].

A fifth family makes evaluation stage-dependent rather than globally static. EvE does not impose explicit phase boundaries, but stage-dependent adaptation emerges because reference solvers and reference agents are resampled each iteration with rank bias, Elo ratings update each iteration based on marginal gains, and revised agents are reinserted into the population [2605.09018]. The paper’s controlled ablations show that freezing either the initial seed agent or the best-evolved agent creates “phase mismatch,” whereas the live ensemble continues adapting to the current solver state [2605.09018].

## 4. Empirical manifestations across domains

The reported instantiations span code generation, hardware design, adversarial games, agentic reinforcement learning, multi-step question answering, GUI automation, recursive self-improvement, and alignment simulation.

| Setting | Co-evolving evaluator | Representative reported outcome |
|---|---|---|
| CoCoEvo | Test case population | pass@1 of 49.75, 55.75, 45.00, 76.25 across GPT-4o-mini, Qwen2.5-Coder-32B, Llama-3.1-70B, DeepSeek-V3 [2502.10802] |
| FAMOU | Weighted opponent pool with champions | highest combined score 0.526 and unseen win rate 61.7% [2606.10389] |
| COEVO | Correctness gate plus 4D Pareto evaluator | 97.5% and 94.5% Pass@1; best PPA on 43 out of 49 synthesizable RTLLM designs [2604.15001] |
| RQGM | Replaceable reviewer/grader slots | 71.7% vs 69.9% held-out pass rate in coding; writers reach 1.78x-1.86x higher acceptance; graders reach 9% higher ground-truth accuracy [2606.26294] |
| EvoTrainer | Training-side harness | SWE 38.16 vs 33.77 under the same data, codebase, and evaluation protocol [2606.03108] |

In automated programming, CoCoEvo reports the highest pass@1 across all four evaluated models on the LeetCode-Contest dataset: 49.75 for GPT-4o-mini, 55.75 for Qwen2.5-Coder-32B, 45.00 for Llama-3.1-70B, and 76.25 for DeepSeek-V3. On the same benchmark, methods dependent on pre-defined tests degrade when using LLM-generated tests. The ablations are evaluator-specific: replacing Pareto test selection with failure rate drops pass@1 to 37.00; replacing it with pass rate or weighted confidence yields 51.50; using the full Pareto scheme recovers 55.75. Removing test evolution reduces performance from 55.75 to 51.75 [2502.10802].

In RTL generation, COEVO reports 97.5% Pass@1 on VerilogEval 2.0 and 94.5% Pass@1 on RTLLM 2.0 with GPT-5.4-mini, and the best PPA product $A\times D\times P$ on 43 out of 49 synthesizable RTLLM designs. The evaluator design is directly implicated by ablation: removing 4D non-dominated sorting reduces PPA Wins from 37 to 26 and increases PPA Losses from 6 to 18; removing the enhanced testbench reduces functional success from 48 to 43 tasks and PPA Wins from 37 to 31; removing the adaptive gate reduces PPA Wins to 32 and functional success to 45 [2604.15001].

In adversarial games, FAMOU consistently outperforms baselines on the MCTF 2026 3v3 maritime capture-the-flag task. Under DeepSeek-V4-Flash it reports $CS = 0.526 \pm 0.075$, $WR = 0.680 \pm 0.080$, margin $+0.83 \pm 0.32$, and unseen $WR = 0.617 \pm 0.072$; under Gemini-2.5-Flash it reports $CS = 0.505 \pm 0.108$. Under DeepSeek, FAMOU discovers 10 champions with 22.5% stagnation, compared with 3 champions and 65% stagnation for OpenEvolve under Gemini. The ablations show the largest drop from removing deep evaluation, with $\Delta CS = -0.136$ and Wilcoxon $p = 0.002$ [2606.10389].

The adversarial literature also documents more mixed dynamics. In the cyber-agent study, coevolution reduces the performance highs and lows of both sides while it induces fluctuations on both sides; by contrast, one-sided optimization yields higher and more sustained peaks [2507.05534]. The EvoMan framework reports that AirMan and QuickMan show the strongest and most sustained arms races under alternating three-generation turns, whereas other enemies remain harder and static-enemy competence does not straightforwardly transfer to coevolution [1604.00644]. The chess co-evolutionary algorithm similarly notes risks of cycling and overfitting to the current pool because it lacks Hall-of-Fame archives and uses a static positional evaluator with competitive pairing [1605.06710].

In autonomous research and agentic RL, EvE reports that the live, co-evolving ensemble avoids phase mismatch and discovers a rescale-then-interpolate positional encoding mechanism for ICON. Its best retrained runs achieve $\bar{e}_{2k} = 0.114$ and $0.108$, and $\bar{e}_{10k} = 0.041$ and $0.045$, whereas the Seed baseline remains at $0.485$ and $0.480$ [2605.09018]. EvoTrainer reports 84.17 / 73.33 / 81.94 on Math versus 80.83 / 71.67 / 77.78 for the human-engineered RL reference, 51.29 versus 50.71 on coding, and 38.16 versus 33.77 on long-horizon SWE for the 9B model, with the largest gain on SWE where evaluator-side diagnostics prevent invalid high-scoring branches from being promoted [2606.03108].

In multi-step agents, ARCO reports the best Exact Match in all six dataset-backbone cells across HotpotQA, 2WikiMultiHopQA, and MuSiQue. For example, on HotpotQA with Qwen3-4B it reaches 42.80 versus 41.00 for R1-Searcher and 38.60 for AgentPRM; on MuSiQue with Llama-3.2-3B it reaches 21.60 versus 18.80 for Search-R1 and 16.60 for AgentPRM. A step-binding diagnostic shows Bind 54.80% on HotpotQA with Qwen3-4B versus 25% chance, supporting the claim that the co-evolved rubrics are step-specific [2606.21262].

In GUI agents, Co-EPG reports monotonic improvement across three iterations. On Multimodal-Mind2Web, Co-EPG-Web-7B improves average Step SR from 53.5 to 55.0 to 58.4, surpassing AGUVIS-7B at 57.2; on AndroidControl, Co-EPG-Mob-7B improves average success from 79.0 to 80.9 to 83.1, exceeding UI-TARS-7B at 81.7. The paper attributes this to the iterative positive feedback loop between planning and grounding, and reports that data purity increases by 8.84% while plan diversity increases by about 4 across iterations [2511.10705].

In recursive self-improvement and alignment, RQGM reports that adding a co-evolved reviewer to coding yields a 71.7% held-out pass rate versus 69.9% for HGM-H while using 1.35x-1.72x fewer blended tokens. In scientific paper writing, the RQGM writer achieves 1.78x higher mean acceptance across a fixed reviewer panel, and the best specialist reaches 1.86x higher acceptance than the HGM-H writer. In Olympiad proof grading, the co-evolved grader reaches 9% higher ground-truth accuracy [2606.26294]. The alignment simulation study reports that only the Combined scenario—mutation plus dynamic testing plus improving alignment detection—achieves true value $\Delta = +0.122$, deceptive ratio $\Delta = -0.087$, both with $p_{\text{adj}} < 0.001$, while keeping fitness statistically indistinguishable from Baseline with $\Delta = -0.006$, $p_{\text{adj}} = 0.554$ [2604.05274].

## 5. Robustness, pathologies, and disputed points

A recurring result is that evaluator co-evolution improves robustness only when accompanied by explicit safeguards. CoCoEvo does not simply add more generated tests; it weights program agreement, computes test confidence using program fitness, and filters out selected tests whose confidence is lower than the average among the selected set. The paper explicitly argues that this curbs noisy, adversarial, or erroneous tests and discourages trivial always-pass or always-fail cases through the discrimination metric [2502.10802]. COEVO similarly avoids a binary correctness gate and instead preserves partially correct but architecturally promising designs inside a 4D Pareto selection loop, but it also identifies testbench overfitting, annealing sensitivity, and trade-off conflicts among area, delay, and power as failure modes [2604.15001].

Another repeated finding is that evaluator adaptation can itself be destabilizing. The alignment study shows that dynamic testing alone decreases true value by $\Delta = -0.060$, $p_{\text{adj}} = 0.017$, and improving alignment detection alone decreases fitness by $\Delta = -0.101$, $p_{\text{adj}} < 0.001$ even while reducing deceptive ratio. Only the combined intervention avoids this trade-off [2604.05274]. In EvE, the Static-Final condition exhibits “phase mismatch,” with poorer trajectories than the live ensemble despite using the best-rated agent from a completed run [2605.09018]. In RQGM, replacement without selective erasure fails because stale utilities pin the archive to the displaced criterion; the paper states that no erasure breaks the curriculum [2606.26294].

A third issue is evaluator hacking. EvoTrainer provides a concrete example: a SWE-9B branch reached 48.80 BC% through Git history leakage, but after environment sanitization the true score was 31.04 BC%, and the branch was blocked from promotion by behavior audits [2606.03108]. RQGM reports that the strongest static reviewer over-accepts AI-generated papers at 1.42x-1.91x the human acceptance rate, which motivates adversarial regularization based on acceptance-rate parity between AI and human text [2606.26294]. These cases indicate that richer evaluators can create new attack surfaces unless evaluator changes are backtested or anchored to independent ground truth.

A fourth issue is oscillation rather than monotone improvement. The cyber-agent study reports that coevolution dampens peaks and induces fluctuations [2507.05534]. EvoMan emphasizes that “the final analysis of the co-evolutionary systematization may not reflect the correct performance,” because measured points may not coincide with peak performance during cycling and because no Hall of Fame is used [1604.00644]. The chess player similarly notes pool dependence, cycling, and overfitting risk, and proposes Hall-of-Fame archives and meta-level evolution of evaluator weights as future work [1605.06710]. These results delimit a controversy: evaluator co-evolution is often presented as a remedy for staleness, but in small-population or archive-free settings it can amplify non-transitivity and make absolute progress harder to measure.

## 6. Relation to adjacent paradigms and likely directions

Evaluator co-evolution intersects with several neighboring traditions but is not reducible to any one of them. FAMOU is explicitly related to PSRO and population-based training, but it replaces meta-game solvers and Nash mixtures with a pragmatic weighted pool, dynamic weights, and champions as hard gates [2606.10389]. ARCO overlaps with process supervision and self-rewarding agents, but differs in jointly learning rubric generation and rubric-conditioned step scoring from on-policy data under a trajectory decomposition constraint [2606.21262]. EvoTrainer differs from recipe-search RL and AutoResearch by evolving the training-time harness that interprets rollouts rather than only policy-side hyperparameters [2606.03108]. RQGM extends Gödel-machine-style self-improvement to evolving utilities by imposing epoch-local stationarity and anchor-guided evaluator replacement [2606.26294].

The literature also suggests that evaluator co-evolution is broadly portable. CoCoEvo explicitly proposes extensions to math problem solving, data cleaning or ETL, planning, refactoring, and synthesis tasks by replacing line coverage with property coverage or constraint coverage [2502.10802]. FAMOU identifies robotics, cybersecurity red teaming, and mixed cooperative-competitive games as plausible extensions [2606.10389]. Co-EPG suggests analogous designs for robotics manipulation, web automation, and desktop control by redefining the success predicate used in $Acc^{plan}_j$ and adapting $\phi$ to output domain-specific actions such as 6-DoF pose tokens or DOM node IDs [2511.10705]. The alignment simulation argues for improving evaluator capability, adaptive test design, and mutational variation as a combined regime rather than separate interventions [2604.05274].

Several design principles recur. Evaluator changes are most stable when they are anchored: RQGM requires evaluator-independent held-out anchors and promotes challengers only when they improve the lower-bound statistic $BB_\epsilon$ [2606.26294]. Fast but noisy screening benefits from deeper confirmation: FAMOU uses 3-game screening and 20-game deep evaluation, plus Wilcoxon signed-rank tests, paired $t$-tests on score margins, and 10,000-sample bootstrap 95% confidence intervals for the Combined Score [2606.10389]. Same-environment comparison reduces phase mismatch: EvE’s synchronous race evaluates agents under identical workspace context so differences in solver quality can be attributed to guidance state [2605.09018]. Backtesting and negative-evidence retention help prevent evaluator drift: EvoTrainer requires offline variance rescue analysis and behavior safety checks before accepting harness changes [2606.03108].

This suggests that evaluator co-evolution is becoming a general design pattern for nonstationary optimization: keep the evaluator informative enough to discriminate current candidates, but constrained enough to remain calibrated to an external anchor, deeper assessment, or cross-validated evidence. The main open difficulty is not whether evaluators should change, but how to let them change without collapsing comparability, inducing reward hacking, or replacing one brittle proxy with another.

Source: https://www.emergentmind.com/topics/evaluator-co-evolution