SoL-Pi: Coding Agent Optimization

• SoL-Pi is an agent-control mechanism that enhances efficiency in long-horizon coding tasks by optimizing action fusion, context management, and observation handling.
• Across GPT-5.6 Sol and Opus 5, SoL-Pi reduced recorded tokens by 44.7-49.0% and API costs by approximately one third compared to Pi, with minor performance trade-offs
• Using the new approach, researchers demonstrated towering of hash task solutions in benchmarks such as EdgeBench and Terminal-Bench.

SoL-Pi is a coding-agent harness discovered through an RSI-inspired automated research process and described in “SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness” (Liu et al., 17 Sep 2026). It modifies the software layer surrounding an underlying LLM—controlling context presentation, tool invocation, action sequencing, observation storage, context compaction, and delegated reading—rather than retraining the model itself. Its fixed efficiency configuration combines four mechanisms: Action Fusion, Online Context Compact, ObservationPack, and Evidence-Preserving Reducer. On the 51-task EdgeBench evaluation, SoL-Pi achieved performance comparable to Pi across GPT-5.6 Sol and Opus 5 while reducing recorded token traffic by 44.7–49.0% and API cost by approximately one third relative to Pi.

1. Conceptual scope and design objective

SoL-Pi addresses long-horizon coding-agent execution rather than isolated code completion. A coding agent may inspect a repository, formulate a plan, modify multiple files, execute tests and builds, diagnose failures, compact its context, delegate information extraction, and repeat these operations for extended periods. The resulting cost depends not only on generated output tokens, but also on repeated prompt input, cache reads and writes, tool observations, context rewrites, redundant actions, and auxiliary model calls.

The harness treats efficiency as a trajectory-level systems problem. Its objective is expressed as:

$$\text{Token efficiency} = \frac{\text{API cost}}{\text{aggregate task score}}.$$

The four retained mechanisms operate at different stages of this trajectory:

• Action execution: Action Fusion combines a mutation with an immediately following command.
• Context management: Online Context Compact performs economically gated context compaction.
• Observation handling: ObservationPack replaces repeatedly transmitted large outputs with handles and excerpts.
• Delegated reading: Evidence-Preserving Reducer extracts verified evidence from build and test logs using a cheaper auxiliary model.

SoL-Pi is therefore neither a LLM nor a new programming language. It is a control and orchestration layer for coding-agent interactions. The paper distinguishes the fixed SoL-Pi [Efficiency] configuration from SoL-Pi [Performance], which uses the single best-scoring mechanism for a particular backend.

The term “RSI-inspired” refers to the use of an AI research process to propose, implement, evaluate, and retain improvements to the harness controlling another AI system. The reported work does not establish unrestricted recursive self-modification or indefinite compounding improvement.

2. Recursive auto-research methodology

SoL-Pi was developed through a broad-to-deep research funnel. An outer search began with 152 proposed directions organized into six families: context, progress, tools, delegation, prompt and policy, and improvement and evaluation. Each direction specified a source of overhead, a concrete harness change, and an intended efficiency effect.

Promising directions were assigned independent search lineages. Each lineage repeatedly proposed a change, implemented it, ran a fixed experiment, inspected traces and metrics, revised or rejected the candidate, and subjected the implementation to review. The process was partly based on a Ralph Loop-style procedure: an implementer continued refining a candidate until an explicit completion criterion was satisfied, after which an independent reviewer assessed it. Failed reviews triggered revision.

Search lineages were isolated and disposable. Each received a shared minimal skill template, copied it into a separate experiment instance, configured its parameters, and ran independently. Modified orchestration code was discarded after the experiment, while candidates and supporting evidence were retained. This separation provided breadth, depth, fault containment, and reduced coupling between candidate investigations.

The reported search scope included approximately 150 proposed directions, approximately 500 executable environments, more than 3,000 runs, and more than 60,000 agent–environment interactions. The detailed method section specifies 535 environments: 495 repository-derived environments and 40 synthetic verifier-driven environments. These counts describe search scope and do not establish a scaling law.

Repository-derived environments contained a GitHub issue, a repository at its pre-fix state, offline dependencies, a hidden accepted patch, and hidden regression tests. An environment was retained only if:

$$\text{tests fail before the accepted patch} \quad\land\quad \text{tests pass after the accepted patch}.$$

The 40 verifier-driven environments were synthetic tasks with executable success conditions, permitting multiple valid solution paths rather than requiring reproduction of a historical patch.

Candidate selection used two sequential gates. First, every capability metric had to remain within its predeclared tolerance relative to the baseline. Second, the candidate had to improve at least one declared efficiency metric. Among candidates passing both gates, nondominated candidates were retained. The held-out evaluation set was excluded from optimization.

The final transfer procedure was one-way: mechanisms were discovered across development environments, selected using development-only metrics, frozen, evaluated on unseen EdgeBench tasks, and then tested on a second model backend without further adaptation. The final EdgeBench set contained 51 public tasks, with 11 used for one-way acceptance of frozen candidates and 40 reserved for final generalization evaluation.

3. Operational mechanisms

Action Fusion

Action Fusion targets the common sequence in which an agent performs a file mutation, waits for a separate model turn, and then issues an obvious follow-up command such as a test, build, or run. It adds an optional follow-up command directly to file-mutation tools, returning the mutation and command outcomes in one observation.

The resulting interaction is reduced from a separate mutation–decision–command sequence to a combined mutation-and-follow-up operation. Commands requiring inspection of the mutation result before a decision remain separate; fusion is therefore selective rather than unconditional.

The development process found that prompt-only triggering was unreliable. The final implementation exposed the fused action through the tool schema, producing a stable interface without invalid calls.

Online Context Compact

Online Context Compact treats compaction as an economic decision rather than as a fixed periodic operation. Compaction can reduce future input tokens but may incur an immediate context-rewrite cost and reduce cache reuse.

At completed plan steps, the harness estimates:

1. the remaining number of model requests;
2. the observed context-growth rate;
3. the number of unfinished plan steps;
4. the number of requests that would fill the current context window;
5. the cost of rewriting the prompt cache;
6. projected future input-token savings.

The projected number of future requests is capped by the number needed to fill the current context window at the observed growth rate. A simplified decision condition is:

$$\text{projected input savings} > \text{estimated context-rewrite cost}.$$

Later compactions account for unrecovered prior rewrite costs and therefore require a larger savings margin. Compaction occurs when the cost gate passes or context usage approaches the window limit, provided that compaction is expected to shorten the context.

The policy estimates cache-rewrite overhead from context size and the cache write/read price ratio. It does not separately price the summarization call. Because context shortening can change an existing cached prefix, it may reduce cache reuse even while reducing visible context size.

ObservationPack

ObservationPack addresses large tool outputs that are repeatedly transmitted even when only a small part remains relevant. It uses a threshold of 10 KiB. Outputs exceeding this threshold are archived locally and transmitted in full for the first two provider requests. From the third provider request onward, the harness replaces them with a stable handle, the original size, and approximately a 1 KiB excerpt containing complete head and tail lines.

The original content remains available on demand through the handle:

$$\text{large result} \longrightarrow \begin{cases} \text{full output}, & \text{first two provider requests},\ \text{handle}+\text{size}+\text{head/tail excerpt}, & \text{later requests}. \end{cases}$$

Small results are left unchanged. The first two full transmissions preserve the model’s opportunity to inspect the output naturally, while later projections avoid repeated transmission without deleting the exact local copy.

The Evidence-Preserving Reducer runs before ObservationPack. Reducer receipts are marked so that ObservationPack does not replace them with a generic excerpt.

Evidence-Preserving Reducer

The Evidence-Preserving Reducer targets build and test logs of at least 4 KiB from a predefined command set. It does not process file reads, search results, or other non-target observations.

The exact log is archived. A lower-cost auxiliary model, specified as GPT-5.6 Luna at high, extracts a compact evidence receipt. A deterministic verifier checks the receipt schema, source hash, exit status, exact quoted evidence, and receipt size.

If verification fails, credentials are suspected, or the receipt is not smaller than the original, the harness falls back to the original log:

$$\text{invalid or non-saving receipt} \Longrightarrow \text{original output}.$$

The auxiliary model performs extraction rather than diagnosis or action selection. The main agent remains responsible for interpreting the evidence and deciding subsequent actions. Processing follows the sequence:

$$\text{tool result} \rightarrow \text{Evidence-Preserving Reducer} \rightarrow \text{ObservationPack projection} \rightarrow \text{main-agent context}.$$

This mechanism is intended to preserve verifiable information while reducing the context footprint of logs.

4. Evaluation on coding-agent benchmarks

The primary evaluation used the 51 publicly released EdgeBench tasks from a benchmark containing 134 tasks in total. The reported comparisons included GPT-5.6 Sol, Opus 5, native Codex and Claude Code harnesses, Pi, and additional third-party harnesses.

On GPT-5.6 Sol, the principal results were:

Harness	Total recorded tokens	API cost	Average score
Pi	2.1538B	$1,339	44.833
SoL-Pi [Efficiency]	1.0990B	$894	42.003
SoL-Pi [Performance]	2.0224B	$1,271	47.208

Relative to Pi, SoL-Pi [Efficiency] reduced total traffic by 49.0%, reduced API cost by 33.2%, and retained 93.7% of Pi’s average score. SoL-Pi [Performance], which used ObservationPack alone under GPT-5.6 Sol, increased the average score by 5.3%, reduced token traffic by 6.1%, and improved token efficiency by 9.8%.

The corresponding token breakdown for Pi and SoL-Pi [Efficiency] was:

Harness	Input	Cache read	Cache write	Output
Pi	0.0011B	2.1326B	0.0141B	0.0059B
SoL-Pi [Efficiency]	0.0009B	1.0605B	0.0316B	0.0061B

Thus, cache-read traffic decreased substantially while cache-write traffic increased. The total API cost nevertheless fell from $1,339 to $894, illustrating that minimizing cache reads alone is not the optimization objective.

SoL-Pi was developed using GPT-5.6 Sol trajectories and then transferred to Opus 5 without further search or adaptation:

Harness	Total recorded tokens	API cost	Average score
Pi	2.3697B	$1,741	44.756
SoL-Pi [Efficiency]	1.3101B	$1,158	42.224
SoL-Pi [Performance]	2.1016B	$1,605	50.482

On Opus 5, SoL-Pi [Efficiency] reduced recorded traffic by 44.7%, reduced API cost by 33.5%, and retained 94.3% of Pi’s score. The best individual mechanism changed: Action Fusion was the strongest single mechanism on Opus 5, whereas ObservationPack was strongest under GPT-5.6 Sol. This indicates backend dependence in mechanism activation and utility.

The component ablations showed that each mechanism reduced total token count under both backends. Under GPT-5.6 Sol, the individual configurations produced the following results:

Configuration	Total tokens	Cost	Score
Pi baseline	2.1538B	$1,339	44.833
+ Action Fusion	1.8968B	$1,235	46.664
+ Online Context Compact	1.2881B	$935	41.993
+ Evidence-Preserving Reducer	1.9375B	$1,200	44.630
+ ObservationPack	2.0224B	$1,271	47.208
Full SoL-Pi	1.0990B	$894	42.003

The full-stack results are consistent with complementarity, but the paper does not present a fully controlled interaction analysis because standalone and full-stack comparisons use different triggered-task subsets.

5. Additional applications and reported savings

On Terminal-Bench 4, the evaluation used 63 CPU-only tasks, excluding GPU-dependent tasks:

Harness	Solved	Total cost	Cost per solved task
Codex	18/63	$272.35 | $15.13
Pi	18/63	$286.45 | $15.91
SoL-Pi	15/63	$211.12 | $14.07

Relative to Pi, SoL-Pi reduced total cost by 26.3% and cost per solved task by 11.6%, but solved three fewer tasks.

On the six-problem IMO 2026 evaluation, whose problems were formalized and verified in Lean 4 with a 150-minute cap per problem, the results were:

Harness	Passed	Total cost	Cost per passed problem
Codex	5/6	$\text{tests fail before the accepted patch} \quad\land\quad \text{tests pass after the accepted patch}.$022.89</td> <td style="text-align: right"></td> </tr> <tr> <td>Pi</td> <td style="text-align: right">3/6</td> <td style="text-align: right">$\text{tests fail before the accepted patch} \quad\land\quad \text{tests pass after the accepted patch}.$125.32</td> <td style="text-align: right"></td> </tr> <tr> <td>SoL-Pi</td> <td style="text-align: right">3/6</td> <td style="text-align: right">$\text{tests fail before the accepted patch} \quad\land\quad \text{tests pass after the accepted patch}.$220.90</td> <td style="text-align: right"></td> </tr> </tbody></table></div> <p>SoL-Pi achieved the lowest cost per passed problem but did not exceed Codex in the number of passed problems.</p> <p>A separate agent-swarm experiment examined kernel optimization measured in simulated machine cycles. The configurations consisted of one Codex agent, a Codex coordinator with 20 Pi workers, and a Codex coordinator with 20 SoL-Pi workers. All runs lasted two hours and began from the same frozen starter requiring 147,734 cycles.</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Configuration</th> <th style="text-align: right">Best cycles</th> <th style="text-align: right">API cost</th> <th style="text-align: right">Thresholds passed</th> </tr> </thead><tbody><tr> <td>Single Codex agent</td> <td style="text-align: right">1,333</td> <td style="text-align: right">$39.20	8/8
Pi swarm	1,366	$82.12	7/8
SoL-Pi swarm	1,127	$60.11	8/8

Relative to the Pi swarm, the SoL-Pi swarm reduced API cost by 26.8% and obtained a better final result. The single agent remained least expensive, so the experiment does not establish that a swarm is universally preferable.

The reported API costs used prices current as of 17 August 2026 and separated uncached input, cache reads, cache writes, output, and auxiliary-model cost. Estimated hourly savings were $\text{tests fail before the accepted patch} \quad\land\quad \text{tests pass after the accepted patch}.$313.50 relative to native Codex or Claude Code harnesses and $\text{tests fail before the accepted patch} \quad\land\quad \text{tests pass after the accepted patch}.$45.71 relative to Pi. The supplied report does not provide the exact hourly workload normalization used to derive these estimates.

6. Limitations and methodological interpretation

The principal validity concern is overfitting to development environments. SoL-Pi addresses this through diverse search environments, fixed capability and efficiency metrics, isolated candidate lineages, frozen candidates, one-way held-out evaluation, and exclusion of EdgeBench results from the search loop. Nevertheless, the evaluation uses 51 of 134 EdgeBench tasks, with 11 tasks involved in frozen-candidate acceptance and 40 in final generalization evaluation. The reported results do not include confidence intervals, standard deviations, or formal statistical significance tests.

The harness is backend-dependent. It was optimized from GPT-5.6 Sol trajectories, while Opus 5 exhibited lower mechanism trigger rates and different best-performing components. The full stack transferred effectively in cost terms, but its score relative to Pi was slightly lower. This indicates that harness policies interact with model behavior, prompting, tool use, and trajectory distributions.

Efficiency and capability are not equivalent objectives. Under GPT-5.6 Sol, SoL-Pi [Efficiency] scored 42.003 versus Pi’s 44.833, while SoL-Pi [Performance] scored 47.208. Under Opus 5, SoL-Pi [Efficiency] scored 42.224 versus Pi’s 44.756, while SoL-Pi [Performance] scored 50.482. The configurations therefore represent different points on a cost–performance trade-off rather than a single total ordering.

ObservationPack can require exact retrieval through a handle. If an agent fails to retrieve relevant content, token savings may reduce information accessibility, although the original output remains locally available. The Evidence-Preserving Reducer can fail verification or fail to produce a smaller receipt, in which case it falls back to the original log. Action Fusion remains conservative because inappropriate fusion could invalidate command sequencing or obscure intermediate information.

The research scale itself does not establish a scaling law. Although the search involved hundreds of environments and thousands of runs, it did not systematically vary search breadth, lineage depth, rollout budget, candidate-pool size, or integration budget. Consequently, the relationship between research scale and harness improvement remains undetermined.

The work also does not demonstrate recursive compounding in which SoL-Pi constructs a better successor that then constructs another successor. “Recursive efficient improvement” is presented as a broader direction rather than as a demonstrated indefinitely compounding process. Similarly, the transfer evidence includes only one additional backend, Opus 5, and therefore does not establish general cross-model robustness.

SoL-Pi is best characterized as an empirically evaluated, multi-stage harness optimization system. Its contribution lies in combining automated candidate discovery with trajectory-level cost accounting, explicit capability preservation, selective context transformation, verified evidence extraction, and backend-aware evaluation. The reported results support substantial cost and token reductions in the tested settings, while leaving open questions concerning generalization, statistical robustness, scaling laws, and long-term recursive improvement.