Red Queen Gödel Machine

• The Red Queen Gödel Machine (RQGM) is an evolutionary framework for recursive self-improvement that co-evolves agents and evaluators under non-stationary utility functions.
• It organizes search into fixed-criterion epochs with anchor-guided evaluator promotion and selective record erasure to ensure convergence and efficiency.
• Empirical results show that RQGM outperforms fixed-evaluator baselines in coding, scientific writing, and proof tasks, enhancing sample efficiency and accuracy.

The Red Queen Gödel Machine (RQGM) is an evolutionary framework for recursive self-improvement that explicitly incorporates non-stationary utility functions by co-evolving agents and their evaluators. Unlike prior self-improving systems that operate under a stationary evaluation criterion (fixed verifiers, benchmarks, or static datasets), the RQGM framework organizes search into fixed-criterion epochs within which classic convergence guarantees apply, while permitting evaluator evolution and utility update at epoch boundaries. This architecture enables controlled utility evolution via anchor-guided evaluator promotion, selective erasure, and adversarial interventions, thereby extending the self-improvement paradigm to domains where direct ground-truth criteria are unavailable or insufficient, such as scientific writing, peer review, and mathematical proof grading (Iacob et al., 24 Jun 2026).

1. Formal Framework

RQGM generalizes the Huxley–Gödel-Machine (HGM) archive-search architecture by supporting the co-evolution of task agents and their learned evaluators under controlled utility dynamics. The system maintains an archive $\mathcal{T}_t$ structured as a rooted tree of workspace nodes $a$, each containing $K$ learnable roles $r\in\mathcal{R}$ (including both task- and evaluator-slots) and a meta-agent with edit privileges. Each role $r$ interacts with its finite task pool $\mathcal{D}_r$; outcomes $o\in\{0,1\}$ are determined by fixed ground-truth for evaluator-independent roles and by evolving, learned evaluators for evaluator-dependent slots.

Epochs are defined by an epoch-vector $j=(j_1,\dots,j_M)\in\mathbb{N}^M$, with evaluator $e_m^{(j_m)}$ frozen within each epoch for slot $m$. Validity of archival records depends on alignment between evaluation criteria $a$0 stored in each record and the current epoch-vector $a$1. Selective record erasure ensures only valid evidence persists as evaluators change at epoch transitions. The epoch-local expected success probability is given by $a$2, and the utility function aggregates success over all roles and tasks: $a$3 (Iacob et al., 24 Jun 2026).

2. Controlled Utility Evolution and Evaluator Replacement

Between epochs, only the evaluator-slots may be updated. At each evaluation checkpoint, the system aggregates a candidate pool $a$4 for each slot $a$5, consisting of the incumbent evaluator $a$6 and archived challenger snapshots. These candidates are scored using a fixed, slot-independent anchor dataset $a$7, recording successes $a$8 and failures $a$9. The $K$0-best-belief lower bound for each evaluator is computed as: $K$1 where $K$2 denotes the inverse regularized incomplete Beta quantile. The promoted evaluator $K$3 maximizes $K$4, with ties resolved in favor of the incumbent. Upon promotion, the epoch-vector is updated ($K$5), the new evaluator is frozen, and all archival records invalidated by the transition are erased (Iacob et al., 24 Jun 2026).

3. Algorithmic Structure

RQGM’s search interleaves within-epoch HGM-style search with anchor-based evaluator replacement at checkpoints. The process initializes with a root agent and frozen evaluators, trains and evaluates initial lineages, and iterates through the following phases until a budget $K$6 of evaluations is reached:

• Within an epoch: sample and expand the archive, evaluating agents using frozen evaluators and recording utilities.
• At epoch checkpoints: for each evaluator slot, score all candidates with the anchor metric, promote the top candidate if superior, erase outdated records, and update statistics.

A summary of the primary algorithmic flow is captured in the following table:

Phase	Activity	State Modifications
Within-epoch search	Expand/search archive, evaluate on frozen slots	Archive, utility updated
Checkpoint (between-ep)	Anchor scorer selection, evaluator freezing	Epoch vector, erasures

(Iacob et al., 24 Jun 2026).

4. Theoretical Guarantees

The formal convergence properties of RQGM are inherited from HGM, contingent on key per-epoch assumptions:

• C1: Evaluator-dependent criteria $K$7 are frozen within the epoch.
• C2: Each record generation process $K$8 and associated scoring kernel $K$9 is time-homogeneous.
• C3: Erasure only removes records invalid under the new criterion, preserving stationarity.

Under these conditions, each epoch forms a fixed-criterion, binary-outcome search problem with time-homogeneous statistics. HGM convergence theorems are thus valid per epoch (“epoch-local validity”). Evaluator promotions are anchor-guided and satisfy probability bounds on true anchor accuracy, specifically with probability $r\in\mathcal{R}$0: $r\in\mathcal{R}$1 across $r\in\mathcal{R}$2 replacements (jointly with probability $r\in\mathcal{R}$3). Exponential spacing of checkpoints ensures $r\in\mathcal{R}$4 total re-evaluation cost, avoiding quadratic blowup (Iacob et al., 24 Jun 2026).

5. Experimental Domains and Protocols

RQGM has been evaluated on three co-evolutionary domains, each with fixed-evaluator (HGM-H) baselines:

• Polyglot Coding: Coder (evaluator-independent, anchor=Polyglot test suite) and code-reviewer (evaluator-dependent, anchor=CRAVE). Train/val/test: 10/49/166 tasks.
• Scientific Paper Writing: Writer (evaluator-dependent, no ground truth) and reviewer (evaluator-dependent, anchor=APReS). Writer artifacts scored by a panel of fixed reviewers; reviewer accuracy tested on 100 held-out APReS papers.
• Olympiad Proof Writing: Prover (evaluator-dependent, no ground truth) and grader (evaluator-dependent, anchor=IMO-GradingBench). 20 IMO problems per run; outputs scored by three graders.

A uniform expansion/exploitation schedule (UCB-Air $r\in\mathcal{R}$5) was used, with all runs structured to budget $r\in\mathcal{R}$6 validation evaluations. Best-belief agents were extracted according to posterior lower bounds from anchor evidence (Iacob et al., 24 Jun 2026).

6. Empirical Results

Quantitative results demonstrate that RQGM consistently outperforms fixed-evaluator baselines with improved efficiency and accuracy:

• Polyglot Coding: Pass rate of 71.7% vs. 69.9% for HGM-H, at 1.35–1.72× fewer search tokens.
• Paper Writing: Generalist writer achieves 38.8% mean panel acceptance (vs. 21.8%, 1.78×); specialist 40.5% (vs. 21.8%, 1.86×).
• Proof Writing: Prover specialist achieves 4.33 mean panel score (vs. 3.73); Pass@6=61.7% (vs. 51.7%). Co-evolved grader attains ≈9% higher accuracy on IMO-GradingBench at 3× lower search cost.

This suggests that co-evolving evaluators in concert with agents yields stronger utility signals and improved sample efficiency, even on domains with verifiable external benchmarks (Iacob et al., 24 Jun 2026).

7. Agent-as-Judge Paradigm and Adversarial Interventions

RQGM enables richer “agent-as-a-judge” signals through learned evaluator co-evolution. In Polyglot coding, an evolved code reviewer prompt focused on “concrete-blocker” heuristics, providing a lower-cost quality signal complementary to test execution. In scientific writing, adversarial objectives mitigated reviewer preferences affecting AI-generated content. By penalizing reviewers for over-accepting AI-produced papers and orchestrating selective erasure/replay, RQGM produced evaluators that scored human and AI work with comparable stringency while maintaining ≈80% APReS accuracy. This harder-to-hack criterion guided task agents toward higher-quality outputs and enabled search objectives to adapt dynamically in pursuit of more robust real-world performance (Iacob et al., 24 Jun 2026).