Red Queen Gödel Machine (RQGM)

• RQGM is a framework for recursive self-improvement that integrates dynamic evaluator evolution with controlled utility shifts, ensuring reliable progress.
• It employs a tree-search architecture with epoch-based freezing and selective record erasure to maintain convergence within fixed evaluation windows.
• Empirical results in coding, scientific writing, and proof reasoning demonstrate improved performance, token efficiency, and calibrated evaluator accuracy.

The Red Queen Gödel Machine (RQGM) is a framework for recursive self-improvement that enables the co-evolution of agents and their evaluators under non-stationary utility functions. Unlike prior self-improving agent architectures that assume fixed evaluation criteria, the RQGM integrates evaluation into the improvement loop via controlled utility evolution, allowing for dynamic objectives, adversarial challenges, and evaluator learning without sacrificing self-improvement guarantees within each epoch. This structure supports open-ended progress across domains—such as code generation, scientific writing, and proof reasoning—by incrementally updating both agent and evaluator populations and tracking improvement with respect to reliable ground-truth anchors (Iacob et al., 24 Jun 2026).

1. Formal Model and Controlled Utility Evolution

The RQGM generalizes tree-search-based self-improvement frameworks (such as Darwin-, Huxley-, and HyperAgents) by allowing the utility function to change, but only at discrete epoch boundaries. Within each epoch, the evaluation criterion—a combination of benchmark tasks and the current evaluator policies—remains fixed. This preserves the convergence and self-improvement guarantees of classical approaches while supporting evaluator evolution at the boundaries.

• Workspace Tree: The framework maintains an archive $\mathcal{T}_0, \ldots, \mathcal{T}_B$ via two operations: expansion by meta-agent edits (creating new candidate solutions) and evaluation by running a chosen node on a specified role-task pair with a binary outcome.
• Multi-Agent Workspaces: Each node contains $K$ roles, parameterized by a set of evaluator slots $\{E_m\}$, each carrying an epoch counter $j_m$.
• Per-Epoch Utility: For epoch vector $j$, each node $a$ evaluated on role $r$ and task $d$ yields a Bernoulli outcome with fixed probability $p_{r,d,j}(a)$. The node's utility is

$$U_j(a) := \frac{1}{|R|} \sum_{r \in R} \frac{1}{|D_r|} \sum_{d \in D_r} p_{r,d,j}(a) \in [0,1].$$

• Controlled Utility Evolution: The evaluation budget $K$0 is partitioned into epochs via checkpoints. Each checkpoint triggers possible promotion of new evaluators and selective erasure of now-invalid records, ensuring all archives remain interpretable under the current evaluator regime.
• Epoch Transition Rule: At epoch boundary $K$1, candidate evaluators are scored on ground-truth anchors via the $K$2-best-belief score:

$K$3

where $K$4 are ground-truth anchor results. Upon promotion, $K$5, and only dependent records are erased.

This epochal freezing and selective erasure mechanism establishes a hybrid model, supporting piecewise-stationary optimization while avoiding catastrophic forgetting and evaluation drift within epochs (Iacob et al., 24 Jun 2026).

2. Core Algorithmic Structure

The RQGM algorithm orchestrates expansion, evaluation, and evaluator promotion via the following steps:

1. Initialization: Set up the workspace tree and freeze initial evaluators.
2. Candidate Expansion and Evaluation:
   • Expand a node by applying meta-agent edits.
   • Otherwise, evaluate nodes on the least-measured role-task pairs under the frozen evaluators, recording outcomes.
3. Checkpoint and Evaluator Promotion:
   • At each checkpoint, for every evaluator slot:
     • Collect incumbent and challenger evaluators.
     • Compute anchor-based $K$6-best-belief scores.
     • Promote the candidate with maximal score; increment epoch counter and erase dependent records.
4. Termination: Return the node with maximal $K$7-best-belief on the anchor set.

The following pseudocode condenses the essential structure:

$\{E_m\}$5 (CMP indicates clade-level Thompson sampling) (Iacob et al., 24 Jun 2026).

3. Theoretical Foundations

RQGM builds on the Huxley-Gödel Machine (HGM) abstraction, adapting its guarantee structure to non-stationary objectives:

• Epoch-Local Validity: If evaluators are held fixed within each epoch and only invalid records are erased, then classical convergence properties of HGM apply per epoch (Proposition D.4).
• Anchor-Guided Improvement: Promotion of new evaluators is contingent on their $K$8-best-belief scores over ground-truth anchors. This ensures with probability at least $K$9 (for $\{E_m\}$0 promotions) that every new evaluator promoted has ground-truth accuracy at least as high as its reported belief bound (Propositions D.7–D.8).
• Amortized Transition Cost: Exponential checkpoint scheduling (with ratio $\{E_m\}$1) bounds the total number of erased or re-evaluated records over budget $\{E_m\}$2 at $\{E_m\}$3, circumventing the prohibitive $\{E_m\}$4 cost of per-evaluation checkpointing (Proposition D.10).

This theoretical apparatus guarantees that the self-improving agent's archive search is always valid relative to the current utility, even as utilities evolve across epochs, thereby facilitating principled agent-evaluator co-evolution (Iacob et al., 24 Jun 2026).

4. Experimental Evaluation and Results

RQGM was empirically evaluated in three distinct domains, each pairing a generator with a learned evaluator role anchored to a ground-truth reference:

Domain	Anchor	RQGM Result(s)
Coding (Polyglot+CRAVE)	166 Polyglot tests; CRAVE verdicts	71.7% pass rate vs. 69.9% SOTA; 1.35×–1.72× fewer tokens
Scientific paper writing	100 APReS decisions	Writer: 38.8% (1.78×) generalist; 40.5% (1.86×) specialist; Reviewers calibrated to 80% APReS accuracy
Olympiad-level proof writing & grading	IMO-GradingBench human grades	Grader: best anchor accuracy at 3× lower token cost; Prover: mean 4.33 (↑over 3.73), Pass@6 61.7% (vs. 51.7%)

• Code-Review Augmentation: Co-evolving a code-reviewer role provided a complementary evaluation to test execution, yielding higher pass rates and efficiency.
• Adversarial Reviewer Design: In scientific reviewing, adversarially trained evaluators recalibrated acceptance rates of human and AI-generated content by penalizing acceptance of “adversarial pool” papers previously misclassified, mitigating LLM self-preference bias.
• Token efficiency: Across domains, the system achieved substantial blended-token savings under fixed evaluation budgets.
• Curriculum and Rubric Discovery: Successive evaluator promotions often led to stricter, more explicit evaluation rubrics, while key agent lineages remained resilient, supporting persistent improvement across successive epochs (Iacob et al., 24 Jun 2026).

5. Innovations and Contributions

Key advances introduced by the RQGM architecture include:

• Joint Agent-Evaluator Evolution: Learning evaluation criteria as part of the search, co-evolving evaluators alongside generators, rather than relying on fixed external benchmarks.
• Controlled Utility Evolution: Using epoch-level freezing and selective erasure to allow dynamic objectives while retaining provable improvement properties within epochs.
• Agent-as-Judge Mechanism: Learned evaluator roles (such as code reviewers) augment direct metric evaluation, improving outcomes and resource usage.
• Adversarial Objective Construction: After evaluator replacement, an adversarial pool mechanism penalizes acceptance of known problematic outputs, debiasing evolution and preventing over-acceptance of AI-generated content.
• Efficiency Metrics: RQGM quantifies token-efficiency gains, reporting blended token use across all domains for comparative analysis.

A plausible implication is that this framework bridges the gap between fixed-criterion self-improvement and open-ended, evolution-inspired agentic learning, supporting scalable, autonomous, and dynamically evaluated AI systems (Iacob et al., 24 Jun 2026).

6. Relation to Prior Work and Broader Impact

RQGM generalizes existing self-improvement frameworks by removing the stationarity assumption from the evaluation process and building robust mechanisms for evaluator co-evolution. It sets itself apart from the Darwin-, Huxley-, and HyperAgents lines by allowing for learned, dynamic evaluators and integrating anchors for objective ground-truth calibration. The use of per-epoch utility freezing with selective erasure provides both practical tractability and theoretical guarantees, while the empirical results demonstrate applicability to real-world generator-evaluator tasks in code, scientific review, and formal mathematical domains.

This suggests that RQGM constitutes a foundational step toward open-ended, self-calibrating agent populations that can adapt to evolving performance standards and new adversarial challenges without losing provable improvement guarantees (Iacob et al., 24 Jun 2026).