Co-evolving Evaluation Methods

• Co-evolving evaluation is a design pattern where both the evaluation criteria and targets adapt in tandem to counter evaluator staleness and improve differentiation.
• It is applied in diverse settings like adversarial games, LLM training, and time-series analysis, refining evaluators through feedback loops and diagnostic measures (e.g., OFC, drift scores).
• These methods balance computational cost with enhanced accuracy, addressing challenges such as static critics, evaluator saturation, and the need for adaptive, interaction-dependent quality metrics.

Co-evolving evaluation denotes a family of methods in which the objects being assessed and the mechanisms that assess them are refined together rather than kept fixed. In the literature, this includes classical coevolutionary fitness assignment between solution and test populations, adaptive critics and reward mechanisms for LLM agents, jointly refined conversational rubrics and test templates, evolving opponent pools in adversarial games, executable software benchmarks grounded in code–test co-change, and sequence or physical-system formalisms in which evaluation tracks changing interaction structure over time (Yo et al., 2019, Li et al., 18 Jan 2026, Zhu et al., 3 Jun 2026, Iacob et al., 24 Jun 2026). A common motivation is that static evaluators become stale, weakly informative, noisy, or semantically untied as the evaluated system improves, whereas co-evolving evaluation attempts to preserve discriminative pressure, calibration, and coverage.

1. Definitions and major formulations

Across the cited works, co-evolving evaluation is not a single algorithm but a design pattern. In one formulation, two populations explicitly evaluate one another, as in solution–test coevolution for the Majority Function problem (Yo et al., 2019). In another, a policy and its critic, reward mechanism, rubric generator, or verifier are updated from the same on-policy interaction data, so that evaluation remains synchronized with changing failure modes and capability profiles (Li et al., 11 Jan 2026, Liu et al., 26 Sep 2025, Li et al., 5 May 2026). A third formulation treats benchmarks, templates, or opponent pools as live objects that must be updated as models, tasks, or strategies change (Wang et al., 2 Jul 2026, Li et al., 9 Jun 2026).

Static evaluation is repeatedly identified as inadequate in settings with long horizons, emergent behavior, or non-stationary opponents. Multi-turn dialogue evaluation is described as under-covering real failure modes when it relies on fixed contexts and fixed rubrics (Li et al., 18 Jan 2026). Deep-research training is described as suffering evaluator saturation when report quality is open-ended and no definitive ground truth exists (Zhu et al., 3 Jun 2026). Critique-guided RL is described as suffering from stale critics when policy error patterns drift during on-policy learning (Li et al., 11 Jan 2026). Adversarial code-evolution is described as failing under fixed opponent pools because apparent progress can be an artifact of evaluator staleness and few-game noise (Li et al., 9 Jun 2026).

Setting	Co-evolving objects	Representative mechanism
Classical coevolution	solutions and tests	AS, WS, AI, WI; OFC (Yo et al., 2019)
LLM/agent training	policy with critic, evaluator, reward, or rubric	ECHO, SCORE, SPARK, EvoLM (Li et al., 11 Jan 2026, Zhu et al., 3 Jun 2026, Liu et al., 26 Sep 2025, Li et al., 5 May 2026)
Benchmarks and adversarial environments	templates, rubrics, tests, opponents, evaluators	CoReflect, TestEvo-Bench, FAMOU, RQGM (Li et al., 18 Jan 2026, Wang et al., 2 Jul 2026, Li et al., 9 Jun 2026, Iacob et al., 24 Jun 2026)
Time-series and physical systems	latent concepts, drift representations, subsystem constraints	Wormhole, CORAL, fluctuation and speed-limit formalisms (Xu et al., 2024, Xu et al., 2 Jan 2025, Wolpert, 2020, Tasnim et al., 2021)

A plausible implication is that the unifying criterion is not the domain but the coupling: evaluation becomes part of the adaptive system rather than an exogenous observer.

2. Classical coevolutionary fitness assignment

A canonical formulation appears in the comparison of evaluation methods on the Majority Function problem for one-dimensional cellular automata (Yo et al., 2019). The setup uses a lattice size $N=9$, radius $r=2$, $2^N=512$ possible binary initial conditions, and dual populations of test cases and solutions. Because all 512 test cases are enumerable, the study distinguishes subjective fitness, computed from the current coevolving test population, from objective fitness, computed against the full test set. This makes it possible to measure not only performance but also how accurately the coevolutionary evaluator tracks ground-truth quality.

The paper compares four methods: average score (AS), weighted score (WS), average informativeness (AI), and weighted informativeness (WI). AS is uniform averaging over successes or failures. WS reweights interactions by opponent strength through inverse total successes. AI combines average score with normalized distinctions, where distinctions count how often a test separates solution pairs or a solution separates test pairs. WI further weights distinctions by inverse frequency so that common distinctions matter less than rare ones. For solutions, the defining WI form is

$$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$

Two quality measures are used. Objective fitness evaluates each solution against all 512 test cases. Objective Fitness Correlation (OFC) is the Pearson correlation between subjective fitness and objective fitness across the solution population in a generation. OFC is therefore a diagnostic of evaluator fidelity rather than merely of search progress.

On the $N=9$ Majority Function problem, weighted informativeness consistently outperforms AI, WS, and AS on objective fitness, with the reported ordering $WI > AI \approx WS > AS$; AI versus WS is not significantly different on objective fitness, whereas OFC yields the clearer ordering $WI > AI > WS > AS$ (Yo et al., 2019). The paper interprets this by arguing that distinctions expose meaningful differences among solutions and that inverse-frequency weighting emphasizes rare, high-value discriminations instead of easy common ones. In the same study, coevolution uses $64 \times 100 = 6{,}400$ interactions per generation, whereas full enumeration in the GA baseline uses $512 \times 100 \approx 51{,}200$ interactions per generation, though WI adds distinction-computation overhead.

A related line of work addresses evaluation cost directly rather than changing the scoring formula. Phylogeny-informed interaction estimation replaces many exact pairwise interactions with estimates inferred from nearby relatives in online phylogenies, using a $k$-nearest evaluated-interaction scheme with $r=2$0 in the reported experiments (Garbus et al., 2024). In the tested domains, this substantially reduces the number of evaluations and yields an early acceleration effect, while the best matchmaking policy is domain-dependent.

3. Adaptive critics, rewards, and judges in LLM agent training

A large cluster of recent work treats evaluation as a trainable component of the agent loop. In ECHO, a policy $r=2$1 and critic $r=2$2 are updated synchronously on the same on-policy cascaded rollouts: the critic samples multiple diagnoses for an initial trajectory, the policy produces critique-conditioned refinements, and the critic is rewarded by the realized gain it induces (Li et al., 11 Jan 2026). ECHO’s saturation-aware shaping defines

$r=2$3

which increases the reward for last-mile improvements near the ceiling. The stated motivation is that frozen critics become stale as the policy’s error profile changes from coarse mistakes to subtle, localized failures.

SCORE addresses an adjacent problem for deep-research report generation, where report quality is multi-dimensional and lacks definitive ground truth (Zhu et al., 3 Jun 2026). It uses a shared-parameter actor with evaluator and solver roles. The evaluator samples rubric dimensions and weights, the solver generates candidate reports, and the evaluator can perform constrained supplementary retrieval before assigning structured judgments. The solver reward is a rubric-weighted scalar built from validity and dimension scores, while the evaluator reward is a rollout-consistency term $r=2$4. To restrict evaluator drift, SCORE introduces a non-trainable meta-harness that controls admissible evaluation dimensions, requires minimum verification depth $r=2$5, and refreshes constraints periodically. The reported results show consistent improvement on DeepResearchBench and DeepResearchEval, with particularly large gains in Valid Citation ratio for Plan-and-Execute agents.

SPARK and EvoLM use a single model in dual roles, but with different anchors. SPARK recycles RLVR rollouts and verifiable correctness signals so that the same model is both policy $r=2$6 and evaluator $r=2$7, trained with pointwise, pairwise, and reflection-conditioned reward objectives rather than a separate reward model (Liu et al., 26 Sep 2025). EvoLM similarly co-trains a policy and rubric generator within a single Qwen3-8B, but keeps a small judge frozen; the rubric generator is optimized for discriminative utility, maximizing the frozen judge’s ability to separate temporally contrasted responses from earlier and later checkpoints (Li et al., 5 May 2026). In both cases, the evaluator is meant to track the current policy distribution rather than lag behind it.

Other frameworks use evaluation signals produced by interactions rather than by explicit rubric scoring. CoMAS assigns complementary rewards to solver and evaluator turns from an LLM-as-a-judge score in $r=2$8, with decentralized replay buffers for each agent and no external verifier (Xue et al., 9 Oct 2025). Evolving-RL derives two supervisory signals from the same downstream evaluation rollouts: a transfer-based skill reward for the extractor and within-task group-normalized advantages for the solver, allowing coordinated co-evolution of experience extraction and utilization (Fan et al., 11 May 2026). Sci-CoE alternates a Solver and Verifier role within one policy, first using sparse correctness anchors and then a geometric reward that combines consensus, reliability, and diversity over strategy embeddings; in the unsupervised stage the solver reward is the verifier pass rate across strategies, while verifier reward combines consistency, centroid proximity, and angular diversity (He et al., 12 Feb 2026).

These systems differ in architecture, but they share a technical premise: evaluative capacity is itself a trainable object, and training that object on stale or external distributions is treated as a bottleneck.

4. Co-evolving tests, rubrics, opponents, and evaluators

Co-evolving evaluation also appears in systems where the evaluation environment itself is revised over time. CoReflect defines a closed loop for multi-turn conversational evaluation in which a conversation planner generates structured templates, a user simulator executes them, an LLM judge scores them under a six-rubric scheme, and a reflective analyzer clusters rationales to update both rubrics and future templates (Li et al., 18 Jan 2026). The rubrics are $r=2$9: Domain Conceptual Alignment, Output Delivery Integrity, Functional Task Progression, Anticipatory Flow Management, Output Structure Fit, and Sustained Style Adherence. CoReflect measures rubric discriminability

$2^N=512$0

and stability

$2^N=512$1

Across $2^N=512$2 iterations, the reported values are $2^N=512$3, $2^N=512$4, and Spearman rank consistency $2^N=512$5.

In software engineering, TestEvo-Bench operationalizes co-evolving evaluation through executable, change-anchored code–test tasks rather than adaptive training-time judges (Wang et al., 2 Jul 2026). It mines adjacent Java commits in which code and tests co-change, packages exact revisions and build environments, and supports two tracks: test generation and test update. The current snapshot contains 746 test generation tasks and 509 test update tasks, curated from 59,950 candidate co-evolution records across 152 open-source Java projects. Evaluation is execution-grounded rather than heuristic, using success criteria defined by cross-revision behavior, plus pass rate, focal-line coverage, and mutation score. Because tasks are timestamped and periodically refreshed, evaluation can be restricted to tasks postdating a model’s training cutoff to reduce data leakage risk.

In adversarial games, FAMOU treats the evaluator as a weighted opponent pool $2^N=512$6 that must evolve together with discovered strategies (Li et al., 9 Jun 2026). Three mechanisms are central: evaluator co-evolution, in which confirmed champions enter the opponent pool as new gatekeepers; hierarchical deep evaluation, in which three-game screening is followed by twenty-game confirmation; and weakness pressure, in which the hardest opponent’s weight is doubled and the rest renormalized. The reported motivation is empirical: three-game scores correlate poorly with 20–40-game deep scores, with Spearman $2^N=512$7 and $2^N=512$8. In the MCTF 2026 3v3 maritime capture-the-flag task, the full system outperforms baselines, and ablations show the largest drop when deep evaluation is removed.

RQGM generalizes evaluator evolution beyond games by introducing controlled utility evolution across epochs (Iacob et al., 24 Jun 2026). Within an epoch, the evaluator is frozen and standard binary-outcome self-improvement proceeds. At epoch boundaries, an evaluator slot can be replaced only if a challenger exceeds the incumbent on a fixed, evaluator-independent ground-truth anchor under an $2^N=512$9-best-belief rule

$$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$0

Selective erasure then removes archive utility records that depended on the displaced evaluator, restoring within-epoch stationarity. This mechanism is used for code review, paper reviewing, and Olympiad proof grading. In paper reviewing, an adversarial objective is added to reduce AI-over-acceptance, while evaluator replacement remains anchored to APReS.

5. Time-series, drift, and physical-system perspectives

In co-evolving time-series, evaluation often means identifying changing concepts or interaction regimes rather than scoring policies. Wormhole models a multivariate time series $$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$1 as windowed segments, encodes them into latent vectors $$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$2, and introduces a self-representation layer $$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$3 with temporal smoothness penalty

$$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$4

Abrupt changes in the columns of $$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$5 define “wormholes,” and the change-point score is computed from $$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$6, with the paper using column-wise means $$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$7 for peak picking (Xu et al., 2024). On Motion Capture, Stock Market, and Online Activity Logs, Wormhole is reported to outperform StreamScope, TICC, and AutoPlait in F1 and ARI.

CORAL, mapped in the paper to the Drift2Matrix methodology, evaluates co-evolving time series through kernel-induced self-representation matrices $$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$8 whose block-diagonal structure encodes concepts and whose temporal variation encodes drift (Xu et al., 2 Jan 2025). It defines drift scores such as

$$F_{WI}(S_j)=0.3 \cdot D'_S(S_j)+0.7 \cdot F_{AS}(S_j).$$9

and tracks concept transitions through spectral clustering and empirical transition probabilities across windows. The reported results show low Type I errors and competitive or best RMSE on many forecasting datasets, with particular gains on financial data where recurring volatility concepts matter.

The notion also appears outside machine learning. In fluctuation-theorem analyses of multiple co-evolving systems, evaluation concerns entropy production, information flow, and current fluctuations under random asynchronous timing rather than fixed update schedules (Wolpert, 2020). The paper derives vector-valued detailed fluctuation theorems over local entropy productions, conditional integral fluctuation theorems, and generalized thermodynamic uncertainty relations that can bound global entropy production even when conventional steady-state criteria fail. A related paper derives thermodynamic speed limits for multipartite co-evolving systems, strengthening global bounds by resolving entropy production and dynamical activity at subsystem and unit levels (Tasnim et al., 2021).

A different observational usage appears in prestellar chemistry, where co-evolution is evaluated through temporal abundance ratios and radial co-spatiality of ion–neutral pairs in molecular cloud cores (Tassis et al., 2012). There the criterion is whether abundance ratios remain approximately constant across density and radius, and the study concludes that HCO$N=9$0/HCN is often not reliably co-evolving at the relevant scales, whereas HCO$N=9$1/NO, HCO$N=9$2/CO, and NO$N=9$3/NO are better candidates.

6. Diagnostics, computational trade-offs, and open problems

A notable feature of this literature is that evaluator quality is itself measured by explicit diagnostics. In classical coevolution, OFC quantifies the Pearson correlation between subjective and objective fitness and functions as a direct measure of evaluator fidelity (Yo et al., 2019). In CoReflect, $N=9$4, $N=9$5, and rank consistency $N=9$6 quantify whether rubric refinement is increasing inter-model separation without destabilizing scores (Li et al., 18 Jan 2026). In EvoTrainer, the harness tracks signal-health quantities such as dead-group ratio and uses offline backtesting before adopting reward or diagnostic changes; invalid high-scoring branches are blocked by behavior-first audits and environment checks rather than by scalar scores alone (Chen et al., 2 Jun 2026). In RQGM, evaluator promotion is constrained by anchor-based $N=9$7-best-belief, and stale evaluator-dependent records are removed by selective erasure (Iacob et al., 24 Jun 2026).

Computational cost is a second recurrent theme. WI in classical coevolution improves fidelity but adds $N=9$8 distinction overhead (Yo et al., 2019). Wormhole and CORAL both incur quadratic or cubic costs from self-representation matrices, motivating sparse updates, block processing, or online variants (Xu et al., 2024, Xu et al., 2 Jan 2025). FAMOU uses hierarchical evaluation because deep confirmation is expensive, while uniform deep evaluation would be prohibitive in large adversarial simulations (Li et al., 9 Jun 2026). TestEvo-Bench reports macro execution footprints on the order of tens of machine-hours across hundreds of tasks and therefore standardizes packaging, caching, and runner behavior (Wang et al., 2 Jul 2026).

The failure modes reported across papers are strikingly consistent. Static critics become stale as policies improve (Li et al., 11 Jan 2026). Fixed evaluators saturate and can induce reward hacking or superficial rubric selection in unverifiable RL (Zhu et al., 3 Jun 2026). Learned judges may become biased or overly permissive; paper reviewing in RQGM is a concrete example, where the baseline reviewer over-accepts AI-generated papers at up to $N=9$9 the human rate (Iacob et al., 24 Jun 2026). In adversarial coevolution, short-horizon evaluation can mis-rank candidates because the game is non-transitive and noisy (Li et al., 9 Jun 2026). In benchmark construction, non-executable or semantically weak tasks can make agent performance look better than it is (Wang et al., 2 Jul 2026).

Open problems are correspondingly broad. Several papers call for more efficient approximations: sampled or approximate distinction computation in coevolution, memory-efficient or online updates for self-representation matrices, and cheaper deep evaluation schedules (Yo et al., 2019, Xu et al., 2024, Li et al., 9 Jun 2026). Others emphasize the lack of formal guarantees connecting evaluator diagnostics to convergence or true quality in open-ended settings (Zhu et al., 3 Jun 2026, Li et al., 18 Jan 2026). Broader ecosystem coverage also remains open, as TestEvo-Bench is currently centered on Java/Maven/JUnit and several LLM frameworks depend strongly on a particular judge architecture or anchor dataset (Wang et al., 2 Jul 2026, Li et al., 5 May 2026). This suggests that co-evolving evaluation has matured into a recognizable methodological family, but not yet into a settled theory.

The literature therefore supports a precise generalization: co-evolving evaluation is most useful when quality is interaction-dependent, non-stationary, multi-dimensional, or expensive to verify directly. In those regimes, the evaluator must be treated as an adaptive component with its own objectives, diagnostics, costs, and failure modes rather than as a static scoring oracle.