RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
This presentation examines a method for evolving AI agent systems that improves performance on unseen tasks rather than merely fitting to training benchmarks. RRSI addresses the core problem of adaptive overfitting in harness evolution by regularizing how agents search for improvements, achieving transfer gains of up to 4.7 points on out-of-distribution benchmarks while using 58% less computation than unregularized methods. The approach demonstrates that controlling the search trajectory matters more than restricting which components can be modified.Script
Recursive self-improvement in AI usually means retraining model weights, but what if you kept the model frozen and evolved everything around it instead? The authors study agent harnesses: the prompts, tools, control flow, memory systems, and orchestration logic that turn a language model into a capable agent.
When researchers evolve harnesses by maximizing score on a fixed benchmark, they face adaptive overfitting. The same evaluation tasks guide every update, so improvements can come from benchmark-specific tricks, noise chasing, or piling on unnecessary computation rather than learning reusable mechanisms. Prior methods gain up to 3.6 points on their training sets but lose most of that advantage when tested elsewhere.
RRSI regularizes the search trajectory without restricting what can be edited. It constrains proposal capacity through annealed edit budgets that start permissive and decay to single sparse updates, while selection criteria screen for task-specific leakage, enforce stability floors based on measured noise, and require that additional inference cost remain proportional to measured gains.
The method improves every held-out benchmark tested. On Harvey LAB workspace tasks, RRSI reaches 90.5 on the evolution set, lower than Meta-Harness at 93.0, yet it achieves 43.6 on out-of-distribution tasks compared to 39.7 for all baselines. In engineering design, it gains 4.3 Medal points on Frontier-Eng, a 24% relative improvement, despite never training on those tasks.
Ablations reveal that proposal and selection regularizers address different failure modes. Removing proposal constraints reduces out-of-distribution performance by 1.7 points and raises cost by 11%. Removing acceptance rules costs 2.6 points and increases token usage by 48%. Removing both yields the highest evolution score but returns out-of-distribution performance nearly to baseline, confirming that maximizing training performance is not a proxy for learning transferable mechanisms.
The evolved harness transfers not only across benchmarks but also across policies. A harness evolved with Gemini 3.5 Flash improves an unseen, weaker Gemini model by 30% relative performance, and cross-policy evolution with Claude and Gemini both yield gains on SWE-bench. The central insight is that regulating how feedback shapes the search matters more than restricting which components can change. Visit EmergentMind.com to explore the full paper and generate your own research videos.
