RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Abstract: An LLM agent's capability is largely magnified by its harness, namely the prompts, control flow, tooling, memory, and context management surrounding the frozen backbone model. Recent methods increasingly automate this process by iteratively proposing and selecting component-wise edits of an agent harness, practically establishing a form of recursive self-improvement (RSI) at the agent-system level. However, such recursive evolution may overfit by memorizing the training tasks, showing large in-distribution gains that shrink or even vanish on out-of-distribution benchmarks. We introduce Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), which incorporates the principles of regularizations into harness self-improvement by constraining the evolution candidate proposal and selection. The proposer operates with a temporally annealed budget, limiting how many edits a candidate can bundle, and it encourages unexplored trajectories based on evolution history. The selector is equipped with a critic and a pruner: the critic screens benchmark-specific proposals, while the pruner, removes changes that are too small, too expensive, or no longer useful. Together these constraints favor reusable agent mechanisms over benchmark-specific ones or even noises. Across eight benchmarks spanning coding, agentic workspace and engineering design tasks, RRSI gains up to 14.1 points on the split it evolves against and up to 4.7 points on the five out-of-distribution benchmarks, while producing a harness that runs on 30% fewer policy tokens than the unregularized evolution. Code is available at https://github.com/google-research/rrsi and project page is https://regularized-rsi.com/.

• The paper successfully demonstrates Regularized Recursive Self-Improvement as a method to improve agent harness evolution and transferability to unseen benchmarks.
• RRSI enhances harness evolution by limiting adaptive overfitting and using mechanisms such as annealed edit budgets and structured exploration to perform targeted improvements.
• When compared to various harness-evolution baselines, RRSI shows superior performance in retention of improvements across held-out benchmarks even when trained in domains such as coding, agentic workspace, and engineering design.

Problem formulation and motivation

“RRSI: Regularized Recursive Self-Improvement of Agent Harnesses” studies recursive self-improvement at the agent-system level rather than at the level of model weights (2609.24972). The central object is an agent $A=(\pi,H)$ consisting of a frozen backbone policy $\pi$ and a harness $H$. The harness includes prompts, control flow, tool interfaces, memory, skills, context management, output plumbing, and subagent orchestration. Harness evolution treats $H$ as the optimization variable: the system executes the current harness on an evolution set, analyzes trajectories and scores, proposes source-level modifications, evaluates candidate harnesses, and retains a candidate according to an empirical selection rule.

The paper’s central claim is that ordinary harness evolution is vulnerable to adaptive overfitting. The same finite evolution set is repeatedly reused to generate feedback, propose modifications, and select the next incumbent. Consequently, the evolution process performs adaptive empirical optimization over a highly expressive search space. Improvements may therefore arise from benchmark-specific behavior, stochastic evaluation effects, or unnecessary increases in inference-time computation rather than from reusable agent mechanisms.

This distinction is empirically consequential. The paper reports that several existing evolution methods obtain strong gains on their evolution split but retain little of those gains on held-out benchmarks; some perform worse than the initial harness after transfer. The authors consequently define generalization as the ability of an evolved harness to transfer to unseen benchmarks with different task descriptions, tool interfaces, and verifiers.

RRSI addresses this problem without restricting the set of harness components that may be modified. Prompts, control flow, tools, memory, skills, context management, and subagents remain editable. Instead, RRSI regularizes the trajectory through the harness search space by controlling both candidate proposal and candidate acceptance.

Figure 1: Evolution-set gains from prior methods often fail to transfer, whereas RRSI preserves improvements across held-out coding, workspace, and engineering benchmarks.

Method: regularizing the search trajectory

The paper distinguishes the potential edit space from the process used to traverse it. Let $\Omega(H)$ denote all harnesses reachable through arbitrary source edits. RRSI leaves $\Omega(H)$ open and imposes constraints on how proposals are generated and which proposals are allowed to become persistent harness state.

This design is important because a conventional restriction on the edit space could exclude precisely the structural changes that make an agent more capable. RRSI instead applies regularization to the adaptive search process. Its proposal-side mechanisms regulate search capacity and exploration; its selection-side mechanisms regulate evidence, leakage, stability, complexity, and structural sparsity.

Figure 2: RRSI constrains proposal capacity and selection criteria while preserving an open harness edit space.

Proposal-side regularization

The first proposal-side mechanism is an annealed edit budget. Early evolution rounds permit several independently attributable edits in a candidate, allowing the system to discover coordinated mechanisms. Later rounds progressively reduce the budget, eventually permitting only sparse updates. The schedule is cosine-annealed from an initial maximum to a minimum of one edit. The authors describe this as an $L_0$-style constraint, but explicitly clarify that it is not optimization of an $L_0$-penalized objective over a fixed parameter vector. It is instead a cardinality constraint on discrete harness edits in each update.

The second mechanism is evidence-aware credit assignment. Each evaluated candidate is recorded with its edited component, underlying hypothesis, source diff, measured score change, token-cost change, and acceptance outcome. Later proposals condition on this history. Rejected mechanisms remain available as negative evidence, while accepted mechanisms retain explicit credit. This is intended to reduce repeated testing of hypotheses that have already failed and to improve attribution as the edit budget becomes sparser.

The third mechanism is structured exploration. When performance stagnates within an empirically estimated noise band, RRSI reserves part of the proposal budget for components that have not yet been exercised. The editable vocabulary includes prompts, control flow, configuration, output plumbing, context management, client tools, skills, memory, and subagents. This exploration policy functions as a targeted diversity mechanism: it discourages collapse onto a narrow family of edits, such as repeatedly rewriting prompts while leaving structural mechanisms untouched.

Finally, proposal-side pruning identifies components that have produced no strictly positive measured gain over a recent window and presents them as deletion targets. This is described as analogous in role to $L_1$ sparsification: unproductive discrete structure is removed from the retained harness rather than merely tolerated.

Selection-side regularization

RRSI applies a leakage critic before full candidate evaluation. The critic rejects diffs that encode task names, entities, task-specific values, answers, or benchmark-specific logic, as well as inert machinery. The method does not reject generic improvements to prompts, tools, or control flow; its target is task-specific content rather than particular component types. Screening before evaluation prevents a leaking candidate from receiving an inflated evolution-set score that could influence subsequent search.

The selector also estimates an empirical noise tolerance $\delta$ by repeatedly evaluating the unchanged base harness. A candidate must satisfy a noise-adjusted floor relative to the best observed score. This prevents the evolution path from drifting through a sequence of small regressions that individually appear compatible with stochastic variation.

Candidates that exceed the noise band must satisfy a gain-dependent cost rule. If $\pi$0 is the candidate’s measured score improvement and $\pi$1 its relative policy-token increase, acceptance requires that additional inference cost remain below a baseline allowance plus a term proportional to $\pi$2. The resulting rule permits greater computation only when the measured gain justifies it. The paper calls this Ridge-like complexity control, while carefully noting that it is not a squared-norm penalty.

Candidates whose measured gain lies within the noise band are evaluated using a shaped admissibility rule incorporating score change, cost change, and structural novelty. In coding experiments, within-band score increases receive no direct credit; such candidates must instead reduce cost or introduce previously unused structural mechanisms. Engineering-design evolution additionally uses non-compensatory guards on valid-output rate and no-submission rate, preventing a primary-score gain from offsetting substantial degradation in execution reliability.

The final selector chooses the highest-scoring candidate among those that pass leakage screening, the noise-adjusted floor, the relevant cost or novelty rule, and any domain-specific guards. If no candidate is admissible, the incumbent remains unchanged.

Experimental design

RRSI is evaluated across eight benchmarks and three domains:

Domain	Evolution benchmark	Held-out or OOD benchmarks
Coding	Terminal-Bench 2.1	SWE-bench Verified
Agentic workspace	Harvey LAB	Harvey LAB held-out, JobBench, GDPval, APEX-Agents
Engineering design	EngDesign	Frontier-Eng

The experiments use a frozen Claude Opus 4.8 policy for the primary evaluation. The proposer, trajectory analyst, and leakage critic also use Claude Opus 4.8. The coding environment contains 89 Terminal-Bench tasks, while Harvey LAB uses a fixed 120-task evolution split and 40-task in-distribution held-out split. EngDesign contains 61 tasks and is evaluated with deterministic task-specific simulators or testbenches. The OOD benchmarks introduce different task distributions, tool ecosystems, and evaluation procedures.

The comparison includes the unevolved harness $\pi$3 and four harness-evolution baselines: Meta-Harness, AHE, TTHE, and HarnessX. All methods begin from the same base harness, use the same frozen policy, evolve on the same split, and receive the same candidate budget. This design isolates the effect of RRSI’s proposal and selection rules, although the evaluation remains dependent on the particular backbone, evolution budget, and hyperparameter choices.

Main empirical results

RRSI improves every reported held-out split. On the evolution benchmarks, it gains 6.0 points on Terminal-Bench 2.1, 4.9 points on EngDesign, and 1.1 points on Harvey LAB. The more important result is transfer: RRSI improves SWE-bench Verified by 1.8 points despite never being evolved against repository-level bug-fixing tasks; it improves the Harvey LAB held-out split by 2.3 points; it gains between 3.5 and 4.7 points on JobBench, GDPval, and APEX-Agents; and it improves Frontier-Eng by 4.3 Medal points, corresponding to a 24.3% relative improvement.

Figure 3: RRSI’s primary results across coding, agentic workspace, and engineering-design domains.

The agentic-workspace comparison makes the transfer-versus-fitting distinction particularly clear. RRSI reaches 90.5 on the Harvey LAB evolution split, lower than Meta-Harness at 93.0 and lower than the other principal evolved systems in several cases. Yet it obtains the strongest held-out performance:

Method	Harvey LAB evolution	Harvey LAB held-out	OOD average
$\pi$4	89.4	86.9	39.7
Meta-Harness	93.0	89.2	39.7
AHE	90.7	88.7	39.2
TTHE	91.1	88.5	37.9
HarnessX	91.8	89.1	39.7
RRSI	90.5	89.2	43.6

RRSI therefore has the smallest evolution-set gain among the reported evolved harnesses in this comparison while achieving the highest OOD average. Relative to the average baseline, the paper reports an OOD advantage of up to 22.9%. The result supports the paper’s main thesis: maximizing the evolution-set score is not an adequate proxy for learning a transferable harness mechanism.

The transfer result is not confined to judge-mediated evaluation. Harvey LAB, JobBench, and GDPval use LLM-based grading, creating a possible concern that RRSI might optimize for judge preferences. EngDesign and Frontier-Eng address this concern through deterministic simulators or testbenches. RRSI improves EngDesign during evolution and Frontier-Eng out of distribution, indicating that the observed transfer is not solely attributable to stylistic adaptation to an LLM judge.

Ablation and efficiency analysis

The ablation study separates proposal-side and acceptance-side regularization. Removing either group increases the evolution-set score but reduces OOD performance.

Variant	Harvey LAB evolution	ID held-out	OOD average	Tokens per trial
$\pi$5	89.4	86.9	39.7	1.56M
Unregularized evolution	92.8	88.9	40.3	3.80M
Without proposal regularizers	90.7	88.8	41.9	2.69M
Without acceptance regularizers	91.5	88.7	41.0	3.59M
RRSI	90.5	89.2	43.6	2.42M

Removing proposal regularization reduces OOD performance from 43.6 to 41.9, despite changing the evolution score by only 0.2 points. This indicates that controlling where the search looks matters independently of the final acceptance criterion. Removing acceptance regularization increases the evolution score by one point but reduces OOD performance by 2.6 points and raises token cost by approximately 48% relative to RRSI. Removing both groups produces the highest evolution score, 92.8, but leaves the OOD average at 40.3, close to the unevolved harness.

The implication is direct: the proposal and selection mechanisms address different failure modes. Proposal regularization limits adaptive search concentration and improves exploration; selection regularization prevents noisy, leaking, or computationally excessive candidates from becoming permanent state. Neither mechanism is reducible to the other.

RRSI also reduces the cost of the final harness relative to all evolved baselines. It uses 2.42 million policy tokens per trial in the agentic-workspace analysis, compared with 3.80 million for unregularized evolution. AHE uses 3.82 million tokens per trial, 58% more than RRSI, while obtaining an OOD average 4.4 points lower.

Figure 4: RRSI dominates the other evolved harnesses in the reported cost–OOD-performance comparison, although the unevolved harness remains cheaper.

RRSI does not achieve the absolute minimum cost: $\pi$6 uses 1.56 million tokens per trial. Thus the method’s efficiency result is relative to harnesses that evolve and improve. Its regularizers constrain the computational price of improvement rather than eliminating the cost of adaptation. The final harness also averages 26.3 steps per trial, compared with 27.3–34.6 for prior evolved harnesses, while $\pi$7 averages 21.2 steps.

Policy and backbone transfer

The paper evaluates whether RRSI depends on the backbone used during evolution. In coding, independent evolution runs with Claude Opus 4.8 and Gemini 3.5 Flash both yield transfer to SWE-bench Verified. With Claude Opus 4.8, Terminal-Bench performance increases from 74.2 to 80.2 and SWE-bench performance from 82.0 to 83.8. With Gemini 3.5 Flash, Terminal-Bench improves from 64.6 to 78.7, a 14.1-point gain, while SWE-bench improves from 76.8 to 79.0.

The paper also evaluates the harness evolved with Gemini 3.5 Flash using the unseen, weaker Gemini 3.1 Flash Lite. Terminal-Bench performance rises from 11.2 to 14.6, a 3.4-point absolute gain and a 30.4% relative improvement. This result supports the claim that at least some learned mechanisms are not specific to the policy used during search. The absolute gain is nevertheless substantially smaller than that obtained with the search policy, and the low base performance of Gemini 3.1 Flash Lite makes the relative percentage sensitive to the denominator.

The cross-policy evidence is encouraging but bounded. It covers two policy families in the main coding comparison and one additional unseen policy in cross-model evaluation. It does not establish policy independence across arbitrary model architectures, context windows, tool APIs, or capability regimes.

Interpretation of the regularization perspective

The paper’s conceptual contribution is to characterize harness evolution as adaptive data analysis over executable agent systems. Every round reuses the evolution set, but the proposal distribution is itself conditioned on earlier measurements. Standard greedy selection therefore compounds multiple sources of selection bias:

• Benchmark-specific fitting: candidate diffs can encode properties of the evolution tasks or their expected outputs.
• Noise chasing: repeated stochastic evaluations can promote candidates that are transient winners.
• Complexity accumulation: additional prompts, tools, memory, or control-flow branches can raise the evolution score while increasing inference cost without improving the underlying mechanism.
• Search collapse: the proposer can repeatedly explore a familiar component family and neglect structural alternatives.

RRSI’s response is not to define a narrower class of harnesses. Instead, it regulates adaptive capacity per round, records evidence over the entire run, forces exploration under stagnation, screens leakage before scoring, imposes stability floors, links cost increases to measured gains, and prunes components lacking evidence of utility.

The analogies to $\pi$8, $\pi$9, and $H$0 regularization are useful organizationally but should not be interpreted as a formal reduction to norm-regularized optimization. The edits are heterogeneous, discrete, and dynamically generated; the harness does not have a fixed continuous parameterization. The paper explicitly acknowledges this distinction. RRSI is best understood as a collection of procedural regularizers for a sequential program-search process.

Limitations and open questions

The evaluation holds backbone weights fixed, so it does not establish whether the same regularization principles remain effective when harness evolution is coupled to model-weight updates or post-training. The approach also depends on a finite evolution set, an empirical noise estimate, and several hyperparameters governing edit budgets, pruning windows, exploration, and cost acceptance. Although these are selected without consulting held-out or OOD benchmarks, their robustness across evolution budgets and task scales is not fully established.

The leakage critic is itself an LLM-based component, and its ability to identify subtle benchmark-specific behavior is not quantified independently. A candidate can avoid explicit task names or values while still exploiting distributional regularities in the evolution set. Similarly, the attribution of bundled edits remains imperfect in early rounds, because all edits in a candidate inherit the same measured score and cost change. The annealed budget improves attribution later but does not provide causal identification of individual edits.

The experiments cover multiple domains, verifiers, and policies, but the evidence remains limited to the reported harness architectures and evolution procedures. EngDesign lacks an in-distribution held-out split because of its size, and the engineering evaluation contains a domain relationship between EngDesign and Frontier-Eng that is not completely independent. The paper therefore leaves open how RRSI behaves under substantially longer evolution horizons, larger candidate populations, continual task streams, different proposer policies, and architectures whose harnesses are not naturally represented as editable source components.

Conclusion

RRSI frames recursive harness evolution as an adaptive optimization problem whose principal risk is not lack of evolution-set improvement but failure to transfer that improvement. Its proposal-side and selection-side regularizers reduce benchmark-specific fitting, noise chasing, structural accumulation, and unnecessary inference cost while preserving an open harness edit space. Across coding, agentic workspace, and engineering-design tasks, RRSI sacrifices some evolution-set score in exchange for stronger held-out performance, achieving up to 14.1 points of in-distribution improvement, up to 4.7 points out of distribution, and substantially lower policy-token cost than unregularized evolution. The central empirical conclusion is that persistent harness improvement depends on regulating how feedback changes the search trajectory, not merely on expanding the set of editable agent components.