ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents

Abstract: We introduce and release ScienceBuddy, an interactive scientific research workspace that brings continually improving scientific agents into researchers' everyday workflows. ScienceBuddy supports researchers in carrying out scientific tasks while transforming their requests, feedback, and execution evidence into tasks and evaluation rubrics for continual learning. At its core is recursive-in-recursive self-improvement, a paradigm that couples harness evolution with model reinforcement learning: the inner recursion improves the harness with the model fixed, while the outer recursion trains the model under the improved harness. Harness evolution shapes training experience, and model learning creates new opportunities for harness adaptation. We present case studies of researcher interaction, harness refinement, and model learning, with the benchmark cases spanning four scientific task families. By releasing ScienceBuddy as a research product, we make this paradigm available to the scientific community and take a step toward discovery intelligence: scientific AI that advances through sustained collaboration with researchers and evolves alongside the research it supports. Website: http://science-buddy.io

• The paper introduces recursive-in-recursive self-improvement (RinR), enhancing scientific agents through nested optimization of the agent harness and task model, which improves both procedural and parametric aspects of scientific interaction.
• The method achieves a 20-point validation gain from harness adaptation at fixed model weights and a 19.5-point increase in pass@4 problem coverage under a fixed harness, consequently enhancing the accuracy of the system.
• ScienceBuddy effectively integrates 224 research tools across 22 modules, supports multiple programming languages and database queries, while across three RinR cycles mean training reward rises between first- and second halves of RLL phases and the held-out test accuracy increases from 42.2% to 73%

Problem formulation and contribution

ScienceBuddy addresses a specific systems problem: how an interactive scientific agent can convert researcher collaboration into improvements that persist beyond the originating conversation. The paper distinguishes two adaptation targets that are often conflated. The first is the agent harness—instructions, reusable skills, context-management procedures, and other execution policies that determine how a model interacts with tools and maintains task state. The second is the task model itself, whose parameters determine the policy used to generate reasoning, tool calls, and researcher-facing responses. The central proposal is to improve these components through nested optimization rather than through a single undifferentiated training loop.

The released system combines a persistent scientific workspace with an editable agent harness and a post-training pipeline. It exposes 224 tools across 22 modules, including genomics, molecular and cancer biology, pharmacology, bioimaging, literature retrieval, and database querying. Python, R, and Bash execution are supported alongside online databases and a local biomedical data lake. Documents, tables, sequences, images, intermediate files, and generated artifacts are retained in a persistent workspace. The system therefore treats scientific assistance as an inspectable computational process rather than as a sequence of isolated text responses.

The paper’s main algorithmic contribution is recursive-in-recursive self-improvement (RinR). In the inner recursion, the task model remains fixed while a separate auxiliary model diagnoses trajectory failures and proposes bounded changes to the harness. Candidate changes are accepted only if they satisfy an edit schema and improve paired development performance. In the outer recursion, the selected harness is held fixed while the task model is updated using fresh on-policy rollouts, task-specific rubric rewards, and GRPO. The updated model is then evaluated with the inherited harness, after which the combined system returns to deployment and further researcher interaction.

This design is related to prior work on reflective prompt optimization, persistent agent skills, self-adapting LLMs, and model–harness co-evolution, including “GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning” (Agrawal et al., 25 Jul 2025), “Agentic Context Engineering: Evolving Contexts for Self-Improving LLMs” (Zhang et al., 6 Oct 2025), “Self-Adapting LLMs” (Zweiger et al., 12 Jun 2025), and “HELIX: Model-Harness Co-evolution for Recursive Self-Improvement” (Fan et al., 14 Aug 2026). ScienceBuddy’s distinctive commitment is to use scientific collaboration as the source of both task specifications and evaluation criteria, while maintaining a procedural recursion and a parameter-learning recursion with explicit information boundaries.

Scientific workspace and interaction model

ScienceBuddy implements a ReAct-style execution loop in which the task model alternates among reasoning, executable code, tool calls, observations, and researcher-facing responses. The platform separates the infrastructure for interaction, execution, storage, and event recording from the harness that constructs model context. This modularity is important experimentally: harness changes can be evaluated while the tool catalog, runtime, workspaces, evaluators, and task model remain fixed.

The researcher interface has three complementary functions. The Chat view supports questions, file attachments, model selection, and follow-up requests. The Compute and Results panels expose execution activity and generated artifacts. The Trajectory view provides an event-level record of user messages, context construction, tool calls, observations, and assistant responses. Selecting an event reveals its payload, result, and metadata, allowing the researcher to inspect the basis of an answer rather than relying exclusively on the final response.

The multimodal workflow is demonstrated with uploaded scientific diagrams. A researcher can provide an image, request target identification and related evidence, and receive a structured target–evidence table that distinguishes retrieved results from missing or unsuccessful searches. The paper explicitly preserves visibility of evidence gaps: for example, a retrieved PDE4/rolipram fragment is distinguished from searches for CD40 and AHR that returned no matching evidence. This distinction is operationally important because the system is evaluated not only on whether it produces a plausible scientific statement, but also on whether it correctly reports what was and was not established by execution.

Figure 1: ScienceBuddy integrates uploaded scientific diagrams, evidence retrieval, structured target analysis, and inspectable computation records.

Long-context interaction is illustrated through successive image-based requests concerning HMGCR, Alzheimer’s-related microglial networks, and immune signaling. The same session retains prior exchanges and execution records, enabling later requests to build on earlier analysis. The paper is careful to state that proposed analyses in these demonstrations are not executed experiments. This qualification limits the evidentiary interpretation of the demonstrations: they establish interaction and inspection capabilities, not empirical validation of the scientific hypotheses discussed in the dialogue.

Figure 2: A continuing multimodal session links uploaded diagrams, retained dialogue, evidence tables, and execution history across successive requests.

The inspection interface also supports post hoc auditing. Researchers can enlarge an uploaded image, switch from Chat to Trajectory, and inspect a specific tool event, including its input, output, and metadata. Such controls provide a mechanism for identifying unsupported claims, failed retrievals, incorrect tool usage, and discrepancies between the conversational answer and the underlying execution trace.

Figure 3: Researchers can inspect source images, execution timelines, and individual tool-event inputs and outputs beyond the final conversation.

From collaboration to executable supervision

A core methodological claim is that researcher interaction can provide more than conversational feedback. A sequence of requests, clarifications, tool observations, artifacts, and follow-up requirements can be converted into a self-contained task and a task-specific rubric. The historical answer is not directly treated as ground truth, and researcher approval is not automatically treated as a correctness label. Instead, the collaboration is reconstructed around a scientific objective, required inputs, methodological constraints, evidence requirements, and expected artifacts.

The resulting package contains four elements: an instruction, input assets, an execution environment, and an evaluation rubric. These packages are represented as Harbor tasks. Rubrics can combine executable checks with fixed-judge assessments for criteria requiring scientific interpretation. The same task packages support rejection-sampled SFT and fresh on-policy RL, although the reported experiments focus primarily on harness adaptation and GRPO-based model learning.

Figure 4: Collaboration-derived Harbor tasks package instructions, assets, execution environments, and rubric-based tests for post-training.

Two interaction cases clarify how this translation operates. In the JAK1 example, a researcher narrows a broad investigation of immunotherapy outcomes and the immune microenvironment in small-cell lung cancer into a gene-focused study plan. The resulting criteria require treatment-by-JAK1 interaction analysis, patient-level expression summaries within cell types, immune-state signatures, and a defined ordering from association analysis to mechanistic follow-up. In the ARL4C example, the researcher requests a presentation with panel selection, claim-to-evidence alignment, mechanism schematics, conclusions, and speaker notes. The derived task therefore evaluates not only scientific content, but also whether each claim is linked to an appropriate comparison or assay.

Figure 5: Researcher scope refinements and presentation requirements are translated into ordered objectives, evidence-linked criteria, and required artifacts.

This formulation separates three signals that could otherwise be confused. Researcher replies can identify unmet requirements and reveal how a task should be conducted. Diagnostic feedback can guide procedural changes to the harness. The policy-learning reward, however, is computed after a fresh rollout against a fixed rubric. In the implementation, a ternary diagnostic score used by the feedback interpreter is explicitly not substituted for the trajectory reward used by GRPO. This separation reduces leakage from private answers and prevents conversational approval from becoming an unexamined optimization target.

Inner recursion: harness evolution

At each inner step, the active harness is treated as the parent and an auxiliary model proposes one bounded revision. Permitted changes include adding, removing, or revising a scoped skill, modifying an instruction, or changing an exposed context setting, subject to the case-study restrictions. Tools, execution infrastructure, task rubrics, evaluators, and model parameters remain fixed. The auxiliary model receives trajectory evidence, rubric feedback, and edit history, but not private reference answers or evaluator internals.

The acceptance rule is deliberately conservative. A candidate must satisfy the schema and achieve a strictly higher mean normalized score than its parent on identical development tasks, seeds, and execution budgets. Ties retain the parent. Previously successful tasks are included to detect regressions, and rejected candidates are retained in the history even though they are not accepted. The procedure therefore resembles constrained local search over procedural programs rather than unconstrained prompt rewriting.

The fixed-model harness experiment uses Qwen3.5-4B as the task model. A Qwen3.8-27B helper supports bounded user simulation and feedback interpretation, while GPT-6 Astra proposes harness edits. Twenty-four adaptation batches are generated from 288 task conversations. The initial harness begins without added instruction or skill entries; the final selected harness contains four instruction entries and nine scoped skills, while the execution loop, input-inspection settings, context handling, tools, submission checks, and budgets remain unchanged.

The adaptation curve reaches a best observed batch accuracy of 75.0%. More importantly, the selected harness is evaluated on a separate validation set rather than only on the adaptation tasks. First-response accuracy increases from 31.1% with the initial harness to 51.1% with the selected harness, a gain of 20 percentage points at fixed model weights. The result supports the claim that procedural adaptation can improve task execution independently of parameter updates.

Figure 6: Bounded harness evolution improves validation accuracy from 31.1% to 51.1% while keeping model weights fixed.

The learned procedures concern execution discipline rather than task-specific answers. They include Python execution, resource and schema inspection, bounded record lookup, explicit answer submission, gene-set membership checks, cytoband lookup, and database-specific evidence extraction. Because the experiment does not isolate individual skills or feedback sources, the reported improvement should be attributed to the cumulative harness-selection process, not to any particular procedural edit.

Outer recursion: model reinforcement learning

The outer recursion fixes the selected harness and improves the task model using fresh on-policy rollouts. As harness revisions make some tasks routine, the system augments environments by varying inputs, analysis conditions, and dependencies between computational steps. Pilot execution under the current model and harness is used to calibrate these environments before they enter RL.

The reward is a weighted aggregation of task-specific rubric criteria. Criteria may cover correctness, evidence, methodological requirements, and artifact production. Executable checks are used where possible, while a fixed judge evaluates criteria requiring scientific interpretation. GRPO forms group-relative advantages from multiple trajectories generated for the same task under the same harness and rubric. The policy optimization is applied only to generated model tokens; researcher messages, tool outputs, task instructions, and deterministic harness actions are excluded from the optimized token stream.

The model-learning case isolates this component by keeping the initial harness fixed. Under the same harness and a common four-attempt evaluation budget, problem coverage measured by pass@4 increases from 48.3% before RL to 67.8% after RL, a gain of 19.5 percentage points. This metric is distinct from first-response accuracy: it measures whether a problem is solved at least once among four attempts and therefore captures breadth of solvable problem instances rather than single-attempt reliability.

Figure 7: With the harness and attempt budget fixed, RL increases pass@4 problem coverage from 48.3% to 67.8%.

The implication is specifically that parameter learning contributes capabilities not reducible to improved execution procedures. The result does not establish that RL improves calibration, scientific validity, or single-attempt reliability in general; it establishes broader coverage under the reported task distribution and evaluation protocol.

Coupled recursive-in-recursive results

The complete RinR experiment starts from Qwen3.5-4B and runs three cycles. Each cycle performs ten harness-evolution steps followed by twenty RL updates. The selected harness is inherited by the next cycle, but it is re-evaluated under the updated model because procedural effectiveness depends on the model executing the procedures.

Across the three cycles, harness validation accuracy increases from 38.9% to 44.4%, from 34.4% to 46.7%, and from 61.1% to 70.0% within the respective cycles. Mean training reward rises from 33.3% to 38.8%, 44.1% to 60.5%, and 57.8% to 69.8% between the first and second halves of the corresponding RL phases. These trajectories indicate that neither recursion becomes immediately ineffective after the first update, although the metrics are measured on the evolving experimental schedule and should not be interpreted as a single stationary learning curve.

Held-out single-attempt test accuracy increases from 42.2% to 73.3%. Of all test problems, 33.3% transition from incorrect to correct, whereas 2.2% transition from correct to incorrect. The low reverse-transition rate is consistent with broad improvement rather than improvement restricted to a small number of newly solved items. Accuracy increases across all four task families: literature reading, database judgments, protocol troubleshooting, and gene or variant assessment.

Figure 8: Three RinR cycles combine harness validation gains, model-learning gains, favorable outcome transitions, and improvements across four scientific task families.

The coupled result is the paper’s strongest numerical claim, but its causal interpretation remains bounded. The experiment demonstrates the behavior of an alternating harness–model procedure under a particular task model, auxiliary models, datasets, rubric construction process, and evaluation schedule. It does not provide a factorial ablation that compares all possible orderings of harness updates and RL, nor does it quantify the marginal contribution of each recursion within the full three-cycle system. The standalone fixed-model and fixed-harness studies provide component-level evidence, while the coupled experiment demonstrates system-level improvement.

Limitations and open questions

The empirical scope is concentrated in biomedical scientific tasks. The benchmark collection contains 895 tasks: 96 literature-reading tasks, 511 database-question tasks, 108 protocol-troubleshooting tasks, and 180 GWAS-related tasks. The workspace supports a broader tool taxonomy, but the reported quantitative evidence does not establish comparable performance in chemistry, materials science, physics, or experimental laboratory control.

The use of rubric-based evaluation introduces dependence on rubric construction and judge reliability. The paper appropriately combines executable checks with fixed judges and states that researcher approval is not automatically scientific ground truth. Nevertheless, several criteria require interpretation, and the study does not report inter-rater agreement, judge calibration, rubric sensitivity, or systematic error analysis. Consequently, improvements in rubric reward and benchmark accuracy may not fully measure scientific validity.

The harness adaptation experiment uses bounded simulated feedback rather than unrestricted human feedback for its quantitative optimization stream. The simulator receives correctness and submission-status information and selects from permitted replies, while the feedback interpreter is prevented from seeing private answers. This design improves experimental control and protects against direct answer leakage, but it leaves open whether the same revisions would emerge from heterogeneous researchers with domain-specific, inconsistent, or partially incorrect feedback.

The three-cycle experiment also leaves attribution unresolved. Harness and model updates are performed sequentially, and the system is evaluated after each cycle, but the paper does not report a full factorial comparison among harness-only, RL-only, reversed-order, and jointly optimized variants over the same number of updates. It therefore supports the feasibility of coordination more directly than the necessity of the particular RinR schedule.

Finally, the paper does not establish long-term stability under distribution shift. Although rejected harness edits and environment versions are retained, the reported studies cover three outer cycles and do not measure degradation from procedural accumulation, evaluator drift, stale skills, tool-version changes, or changing biomedical evidence. A specific open question is whether inherited harnesses remain beneficial when the task distribution changes substantially while rubrics and execution infrastructure are held fixed.

Conclusion

ScienceBuddy presents a concrete architecture for interactive scientific agents in which persistent workspaces, inspectable trajectories, researcher-derived tasks, and modular harnesses support continual improvement. Its recursive-in-recursive procedure separates bounded harness evolution from rubric-supervised model reinforcement learning and then reconnects them across deployment cycles. The experiments report a 20-point validation gain from harness adaptation at fixed weights, a 19.5-point increase in pass@4 problem coverage under a fixed harness, and an increase in held-out single-attempt accuracy from 42.2% to 73.3% across three coupled cycles. These results establish complementary procedural and parameter-level improvements, while leaving the relative causal contributions, rubric validity, human-feedback transfer, and cross-domain robustness as explicit questions for further evaluation.