2000 character limit reached
RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement
Published 28 Jul 2026 in cs.SE and cs.CL | (2607.25886v1)
Abstract: Recursive self-improvement requires turning evidence of model failures into better models. Data-centric post-training research entails diagnosing capability gaps, designing and validating training-data strategies, and learning from checkpoint feedback. Can LLM agents automate this loop? Existing benchmarks entangle research decisions with optimization, serving, evaluation, and systems implementation, obscuring agents' research capability. We introduce RSIBench-Data, a controlled benchmark of LLM agents as data-centric researchers with a fixed post-training stack. Agents iteratively revise training-data strategies for a fixed target model; training and serving use Tinker-backed services, official evaluation runs through Harbor and E2B sandboxes, and budgets are fixed across agents. We evaluate four frontier agents on six benchmarks across software engineering, terminal use, scientific question answering, and mathematics. Agents demonstrate core data-centric research capabilities: in 58.33\% of settings, they improve upon the first valid attempt by refining strategies from feedback. However, improvement is inconsistent. Among searches continuing after the best observed score, 78.26\% end with a lower-scoring final attempt, while the rest only recover the same peak. A strong candidate may therefore appear early or midway through a run even as later revisions fail. Trajectory analysis identifies four patterns in stronger runs: accurate hypotheses, validation-grounded supervision, behavior-aligned data, and preservation of strong checkpoints. These findings suggest that current agents can make useful data-centric discoveries but cannot yet translate feedback into consistent improvements. RSIBench-Data provides a measurable, auditable testbed for the research capabilities required for recursive self-improvement. We open-source our code at https://github.com/evolvent-ai/RSIBench-Data.
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Explain it Like I'm 14
1. What is this paper about?
This paper introduces RSIBench-Data, a test for seeing whether AI agents can improve another AI model by creating better training data.
The idea is connected to recursive self-improvement: an AI system looks at its mistakes, figures out what went wrong, creates useful learning examples, trains a better version of the model, and then repeats the process.
A simple analogy is a student preparing for exams:
- The student takes a test.
- They examine their mistakes.
- They practice the topics they struggled with.
- They take another test.
- They change their study plan based on the new results.
RSIBench-Data asks whether an AI agent can do something similar for another AI model.
2. Main research questions
The researchers wanted to answer several basic questions:
- Can AI agents correctly identify what another model is bad at?
- Can they create training examples aimed at fixing those weaknesses?
- Can they use test results and other feedback to improve their next training plan?
- Can they recognize when an earlier version was better and avoid replacing it with a worse version?
- How well do different AI agents perform when they are given the same tools, model, and budget?
The paper focuses especially on data-centric research. This means improving a model mainly by deciding what examples it should learn from, rather than changing its entire computer system or training algorithm.
3. How did the researchers test this?
A controlled experiment
The researchers tested four AI agents from two systems:
- Claude Code using Opus-4.8 and Sonnet-5
- Codex using gpt-5.6-sol and gpt-5.6-terra
These agents acted as “researchers.” They were given the job of improving a separate target model.
To make the comparison fair, many things were kept exactly the same for every agent:
- The starting model
- The training software
- The way the model was tested
- The available computer services
- The time and money limits
- The task examples and evaluation rules
This is like comparing four students who all receive the same textbook, classroom, exam, study time, and calculator. The main difference is their study strategy.
The improvement loop
Each agent followed a repeated cycle:
- Study the problem: The agent looked at the benchmark and available information about the target model.
- Make a hypothesis: It guessed what the model needed to learn. For example, it might decide that the model was bad at correctly using command-line tools.
- Create training data: It produced examples, conversations, tool-use records, or solved tasks designed to teach that skill.
- Train the target model: The shared training system created a new version, called a checkpoint.
- Evaluate the checkpoint: The new model attempted tasks in a controlled environment.
- Use feedback: The agent examined scores, errors, and task behavior, then decided whether to change its strategy.
- Choose a final checkpoint: At the end, it selected one of the versions it had created.
A checkpoint is simply a saved version of a model at a particular point in training, much like saving different versions of a school project.
Technical terms in simple language
- LLM agent: A LLM that can plan, use tools, run commands, inspect results, and take several actions instead of only answering one question.
- Post-training: Extra training performed after a model has already learned general language abilities.
- LoRA: A faster and cheaper way to adapt a large model by training only a small number of added parts instead of changing the whole model.
- SFT, or supervised fine-tuning: Training a model using examples that show what a good answer or action looks like.
- Benchmark: A standardized test used to compare models.
- Selection feedback: Information the agent is allowed to use while choosing what to try next.
- Official evaluation: The final test used to measure the selected model’s performance.
Tasks used in the experiments
The agents were tested on six types of tasks:
- Fixing software problems in GitHub projects
- Fixing software problems across multiple languages
- More difficult software-engineering tasks
- Using a computer terminal and tools over many steps
- Answering difficult science questions
- Solving advanced mathematics problems
Each experiment had a limit of about 16 hours and $500 in training costs.
4. Main findings
Agents sometimes discovered better training strategies
In 14 of 24 settings, a later attempt was better than the agent’s first valid attempt.
This means that in about 58.33% of cases, the agents were able to learn from their earlier experiments and create a stronger training-data strategy.
That is an important positive result. It shows that current AI agents can sometimes:
- Notice a weakness
- Design useful examples
- Test a new idea
- Discover a better way to train the model
Improvement was not reliable
The biggest problem was that agents often became worse after finding a good version.
Among the 23 experiments where agents kept searching after reaching their best score:
- 18 finished with a lower-scoring final attempt
- 5 returned only to their previous best score
- None ended with a new improvement after that peak
In other words, once an agent found a strong candidate, later changes usually did not improve it. Sometimes the researchers could protect themselves by selecting the best earlier checkpoint, but the agents did not reliably understand when to stop or how to make the next improvement.
The authors call this a discovery–reliability gap:
- Agents can sometimes make good discoveries.
- They cannot yet use feedback consistently enough to keep improving.
Results depended strongly on the task
No single agent was best at every benchmark.
For example:
- Claude Opus performed best on one software-engineering benchmark.
- Claude Sonnet performed best on another.
- Codex gpt-5.6-sol performed best on several other tasks, including difficult science, mathematics, and terminal tasks.
This suggests that success depends not only on how powerful the research agent is, but also on whether its strategy matches the kind of task.
Some training strategies made the model worse
Training was not always helpful. On some tasks, the adapted model scored lower than the original model.
For example, the original model scored:
- 61% on GPQA Diamond, while some trained versions scored below that
- 7% on SWE-bench Multilingual, while some agents produced versions scoring even lower
This shows that creating more training data does not automatically improve a model. Poorly designed examples can teach the wrong behavior or damage abilities the model already had.
Stronger runs had several common features
The researchers found four patterns in the more successful experiments:
- Accurate hypotheses: The agent correctly understood what ability was missing.
- Validation-based supervision: It checked that its training examples actually showed the desired behavior.
- Behavior-aligned data: The training examples closely matched what the model needed to do during the real test.
- Preservation of strong checkpoints: The agent kept and selected an earlier good version instead of losing it.
These are similar to good scientific research: identify the right problem, test ideas carefully, create relevant practice, and keep useful results.
More thinking sometimes helped
In one extra comparison, Claude Sonnet was allowed either a high or maximum level of reasoning effort.
The maximum-effort version:
- Found stronger candidates earlier
- Achieved a higher official score
- Created a larger and more detailed training set
However, it also completed fewer attempts because it used more resources per attempt. This shows a trade-off:
- More effort may produce a better individual experiment.
- Less effort may allow more experiments within the same budget.
5. Why are these findings important?
The paper shows that improving AI is not simply a matter of giving it more data or letting it run more experiments.
An AI researcher must make several difficult decisions:
- What problem should be fixed?
- Which examples would teach the right skill?
- How should those examples be checked?
- Did the new training actually help?
- Should the agent continue experimenting or stop?
- Which saved model is the strongest?
RSIBench-Data is useful because it separates these research decisions from other parts of AI training. Since all agents use the same training and testing systems, differences are more likely to come from the agents’ research strategies.
The benchmark also records the agent’s process, not just its final score. This lets researchers study why an agent succeeded or failed.
6. Implications and possible impact
The research suggests that AI agents are beginning to act like useful data researchers, but they are not yet dependable enough to improve models automatically over many rounds.
In the future, better systems may need to:
- Diagnose model failures more accurately
- Make training examples that better match real tasks
- Test their own training data before using it
- Avoid teaching harmful or incorrect behaviors
- Recognize when a new experiment is worse than an earlier one
- Stop spending time and money once further experiments are unlikely to help
- Test improvements on completely new tasks, not only familiar ones
If these problems are solved, AI systems could help improve other AI systems more quickly and with less human supervision. That could speed up progress in software engineering, mathematics, science, and other areas.
However, the paper also has limitations. Most experiments were run only once, and several tests used fixed subsets of tasks. Therefore, the results do not yet prove that these improvements would work reliably on completely new problems. More repeated experiments and private test sets are needed.
Overall, the paper’s message is: current AI agents can sometimes discover better ways to train models, but they still struggle to turn feedback into steady, dependable improvement. RSIBench-Data provides a way to measure and study that ability.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
The paper leaves the following issues unresolved:
- Generalization beyond the evaluated task subsets: Official checkpoint selection and evaluation use the same task subsets for several benchmarks, so it remains unclear whether improvements transfer to statistically held-out tasks, refreshed repositories, or unseen environments.
- Stability across repeated runs: Most agent–benchmark combinations are represented by one run, preventing reliable estimates of variance caused by stochastic agent behavior, data synthesis, training, decoding, and evaluation.
- Causal attribution of performance gains: The study does not isolate whether improvements arise from the researcher agent’s hypotheses, data construction, training-configuration choices, rollout quality, or interactions among these factors.
- Effect of the fixed post-training stack: Because all experiments use one target model, LoRA SFT backend, serving path, optimizer configuration, and evaluation infrastructure, it is unknown whether the findings hold for other model architectures, adaptation methods, optimization procedures, or context limits.
- Limited target-model diversity: The main matrix evaluates only Qwen3.5-35B-A3B-Base, leaving unresolved whether researcher agents behave similarly when improving instruction-tuned models, smaller models, dense models, or models with different baseline capabilities.
- Limited researcher-agent coverage: Four proprietary agent systems are evaluated, and researcher identity is bundled with the underlying LLM, scaffold, tool interface, and reasoning effort; the individual contribution of each component is therefore unknown.
- Insufficiently controlled reasoning-effort comparison: The high-versus-max reasoning experiment is a single diagnostic comparison, with different numbers and sizes of training candidates, so it does not establish whether reasoning effort itself—or the resulting resource allocation—causes the observed gains.
- Unclear reliability of selection feedback: The paper reports an average gap between selection and official scores but does not quantify how accurately selection signals predict held-out performance across tasks, agents, or candidate types.
- No principled stopping criterion: Historical-best selection protects against later regressions, but the benchmark does not determine when an agent should stop searching or how stopping policies should balance expected improvement against time and cost.
- Weak understanding of post-peak regression: The study documents that 18 of 23 continued searches end below their historical peak, but does not identify whether regressions result primarily from noisy evaluation, distributional drift, destructive fine-tuning, poor diagnosis, overfitting, or flawed checkpoint-selection heuristics.
- Unvalidated mechanism categories: The proposed mechanisms—accurate hypotheses, validation-grounded supervision, behavior-aligned data, and checkpoint preservation—are based on trajectory coding, but their coding reliability, operational definitions, and independent predictive validity are not established.
- No ablation of data-synthesis decisions: The relative importance of filtering, verification, curriculum design, data mixture, trajectory length, tool-result preservation, and training exposure is not disentangled through controlled ablations.
- Unclear contribution of configuration choices: Agents may vary whitelisted training configurations as well as data, making it difficult to determine whether the benchmark primarily measures data-centric research or bounded hyperparameter optimization.
- Potential mismatch between training experience and evaluated behavior: The paper shows failures involving malformed tool supervision and missing tool-result messages, but does not systematically measure how representation fidelity, message formatting, and trajectory completeness affect downstream performance.
- No analysis of negative transfer: Several trained checkpoints underperform the base model, yet the conditions under which training damages existing capabilities—and whether damage is localized or broad—remain unexplored.
- Limited analysis of capability retention: The benchmark focuses mainly on target-task scores and does not test whether post-training strategies preserve unrelated capabilities, safety properties, instruction following, or tool-use competencies.
- Unresolved relationship between baseline headroom and iteration value: The paper observes that iteration behaves differently across tasks with different base-model scores, but does not model how baseline capability, task difficulty, and available improvement headroom affect the likelihood of successful research.
- No evaluation of reusable research policies: The benchmark evaluates agents independently per task; it does not test whether an agent can learn a data-synthesis policy from one benchmark and transfer it to another without restarting the research process.
- No evidence of sustained recursive self-improvement: The experiments involve short, bounded post-training loops rather than multiple generations of researcher–target co-improvement, so they do not establish whether discovered strategies remain effective over repeated recursive cycles.
- Unclear scalability with larger budgets: It is unknown whether additional time, training cost, candidate attempts, or data volume eventually produce reliable gains or merely increase post-peak regression.
- Resource-efficiency conclusions lack uncertainty estimates: Reported cost and time differences are based on individual runs and are not accompanied by confidence intervals, cost-normalized performance distributions, or statistically robust efficiency comparisons.
- Benchmark representativeness is uncertain: Six benchmarks cover several domains but may not represent scientific discovery, multimodal tasks, interactive planning, safety, or real-world organizational workflows relevant to recursive self-improvement.
- Possible contamination and source-overlap risks: The paper restricts data use but does not provide a comprehensive analysis of overlap between public seed repositories, rollout data, researcher-model pretraining, and benchmark tasks.
- Incomplete characterization of evaluator noise: The effects of stochastic decoding, sandbox execution variability, verifier errors, and repeated-run aggregation on candidate rankings are not quantified.
- No formal metric for research-process quality: Official performance is primary and trajectory records are used diagnostically, but the benchmark lacks an independent, reproducible score for hypothesis quality, feedback interpretation, data validity, or research efficiency.
- Limited evidence for the early RSI experiment: The same-family Kimi experiment is exploratory and uses a different target model, budget, and setup; its results cannot be directly compared with the main matrix or used to establish general conclusions.
- Unresolved security and governance issues: The paper does not examine whether autonomous data-centric researchers can exploit evaluator artifacts, introduce unsafe training examples, leak protected information, or manipulate feedback under less restrictive service boundaries.
Practical Applications
Immediate Applications
- Automated post-training research for software-engineering agents — Software/Enterprise
- Organizations can use the RSIBench-Data workflow to test whether an AI system can identify coding-agent failures, synthesize repository-grounded training examples, train LoRA adapters, evaluate them in sandboxes, and retain the strongest checkpoint.
- A practical product would be an internal agent-improvement pipeline connected to GitHub or GitLab issue repositories, execution traces, test suites, and isolated build environments.
- The paper’s findings support several immediate controls:
- Diagnose a specific capability gap before generating data.
- Validate tool-call and tool-result pairings.
- Align supervision with the behavior measured by evaluation.
- Preserve the historical-best checkpoint rather than automatically deploying the latest one.
- Evidence: later attempts improved over the first valid attempt in 14 of 24 settings, but 18 of 23 searches that continued after their peak ended below that peak.
- Dependencies: reliable execution traces, representative repositories, secure sandboxes, sufficient compute budget, and evaluation tasks that are not included in training data.
- Checkpoint-selection and rollback systems for model development — MLOps/Cloud AI
- Training platforms can implement automatic best-checkpoint registries, score-based rollback, and early-stopping policies for iterative synthetic-data experiments.
- Rather than treating the final training attempt as the production candidate, a workflow can retain every dataset, configuration, score, trajectory, and checkpoint, then deploy the strongest historically validated model.
- This is deployable using existing experiment-tracking infrastructure and is especially relevant where additional training can cause regression or catastrophic specialization.
- Dependencies: selection scores must correlate sufficiently with production performance; otherwise, checkpoints require evaluation on private or refreshed validation sets.
- Auditable data-centric research environments — Academia/Industrial AI laboratories
- Research groups can adopt the paper’s separation of responsibilities—fixed model, training interface, serving path, evaluator, and budget—to conduct reproducible comparisons of data-generation agents.
- A practical benchmark harness could combine:
- Containerized or sandboxed task execution.
- Fixed LoRA/SFT APIs.
- Structured training-artifact validation.
- Controlled feedback channels.
- Independent final evaluation.
- This would help distinguish improvements caused by better research decisions from improvements caused by changing optimizers, infrastructure, prompts, or evaluation code.
- Dependencies: standardized APIs, reproducible environments, protected evaluation data, and sufficient infrastructure to run isolated experiments.
- Failure-driven dataset curation for coding and terminal agents — Software/Developer Tools
- Companies can mine failed tool-use traces, verifier errors, malformed patches, and execution diagnostics to create targeted supervision rather than collecting undifferentiated interaction logs.
- Potential tools include:
- A trace-to-training-example converter.
- Automatic tool-call schema validators.
- Failure clustering dashboards.
- Curriculum builders that emphasize difficult or repeatedly failed tasks.
- The paper specifically indicates that repaired tool pairing, longer multi-step traces, lower learning rates, and behavior-aligned examples can improve intermediate results.
- Dependencies: permission to use repository and user data, accurate failure attribution, privacy protection, and safeguards against learning superficial or benchmark-specific patterns.
- Cost-aware experimentation for model improvement — Cloud Computing/AI Infrastructure
- The benchmark provides a basis for selecting data strategies using both performance and resource use. Teams can define stopping rules such as “stop after no frontier improvement for n attempts” or optimize score subject to time and training-cost limits.
- This is particularly useful for hosted fine-tuning services, where iterative candidate generation can become expensive after the best checkpoint has already been found.
- The observed range—from $4.80 to$363.77 in valid-candidate training cost—shows why score-only reporting is insufficient.
- Dependencies: accurate cost accounting, stable evaluation signals, and business-specific thresholds for quality, latency, and return on investment.
- Evaluation of AI research assistants and coding copilots — Industry/Procurement
- Organizations can use RSIBench-style tests when selecting autonomous coding or research agents, evaluating not only final model quality but also:
- Quality of capability diagnoses.
- Correctness of generated supervision.
- Ability to interpret feedback.
- Checkpoint preservation.
- Search efficiency.
- This provides a more informative procurement criterion than a single benchmark score, particularly for systems expected to improve continuously.
- Dependencies: task-specific benchmarks, independent reruns, protection against benchmark contamination, and clear definitions of acceptable autonomy.
- Training and classroom exercises in reproducible machine learning — Education/Academia
- The benchmark protocol can become a practical laboratory assignment in which students design a data strategy, run bounded experiments, analyze regressions, and justify checkpoint selection.
- Students can learn experimental control, dataset construction, error analysis, reproducibility, and the difference between development feedback and official evaluation.
- Dependencies: access to affordable compute, simplified benchmark tasks, and educational versions of the sandbox and training interfaces.
Long-Term Applications
- Reliable recursive self-improvement systems — General AI Research
- The benchmark could serve as a component of systems that autonomously cycle through failure diagnosis, data synthesis, training, evaluation, and model selection.
- A mature system would need to improve not merely once, but reliably across many rounds while avoiding the paper’s discovery–reliability gap.
- Likely technical additions include:
- Bayesian or causal diagnosis of capability gaps.
- Multi-agent debate over proposed data strategies.
- Conservative policy updates.
- Uncertainty-aware stopping.
- Automatic rollback and checkpoint ensembles.
- Dependencies: repeated improvement on held-out tasks, robust safeguards, independent evaluation, resistance to reward hacking, and evidence that gains transfer beyond the benchmark.
- Self-improving software-maintenance platforms — Software Engineering
- Future development systems could monitor production failures, reproduce them in secure environments, generate targeted training data, fine-tune coding agents, and deploy only models that pass regression suites.
- Such a platform could continuously specialize an agent for a company’s programming languages, frameworks, repository conventions, and operational procedures.
- Dependencies: high-quality production telemetry, strong security isolation, human approval for code changes, prevention of sensitive-code leakage, and validation on unseen repositories.
- Domain-specific research agents for science and mathematics — Science/Education
- The closed-loop approach could be extended to research assistants that identify weak areas in scientific question answering, mathematical proof generation, experiment planning, or literature synthesis.
- For example, an agent could analyze failed answers, generate verifier-backed examples, train a domain adapter, and test it on newly authored or private problems.
- The strongest potential use cases are domains with reliable verification mechanisms, such as symbolic mathematics, code-based scientific workflows, and computational chemistry.
- Dependencies: expert-quality verifiers, legally usable scientific data, domain experts for validation, and protection against fabricated or circular supervision.
- Adaptive educational tutoring models — Education
- A tutoring system could use learner errors and interaction traces to diagnose a specific misconception, synthesize targeted exercises, fine-tune a student- or classroom-specific adapter, and evaluate learning outcomes.
- Unlike generic synthetic-data generation, this would require matching training supervision to the behavior that matters—such as explanation quality, misconception correction, or transfer to unseen problems.
- Dependencies: longitudinal evidence of actual learning, privacy-preserving student data, pedagogical validation, avoidance of overfitting to individual assessments, and human teacher oversight.
- Safety and reliability improvement for healthcare, finance, and other regulated systems — Healthcare/Finance/Policy
- RSIBench-like protocols could be used to improve specialized models for clinical documentation, financial analysis, compliance reasoning, or operational decision support while keeping the evaluation and deployment stack controlled.
- Every candidate model could be tested against private, refreshed cases and audited for regressions before release.
- The approach is more suitable for decision-support than autonomous decision-making because the paper demonstrates improvement is fragile and task-dependent.
- Dependencies: regulatory approval, clinically or financially valid evaluation, privacy and data-governance controls, human review, distribution-shift testing, and clear liability arrangements.
- Robotics and embodied-agent adaptation — Robotics/Manufacturing
- The benchmark’s treatment of executable trajectories, tool-use sequences, verifier outcomes, and environment feedback could be transferred to robots operating in simulated or real industrial environments.
- A robot-improvement loop might diagnose failures such as navigation errors or unsafe manipulation, generate targeted demonstrations, train an adapter, and test it in a digital twin before limited physical deployment.
- Dependencies: high-fidelity simulation, safe exploration, reliable physical-world verifiers, hardware repeatability, and strict safety certification.
- Policy and standards for autonomous model improvement — Government/Standards Bodies
- The findings support standards requiring autonomous improvement systems to maintain immutable experiment logs, disclose training-data provenance, report cost and time, preserve prior checkpoints, and perform independent final evaluations.
- Regulators could require evidence that a system does not use protected evaluation data as supervision and that post-training changes do not reduce performance on safety or fairness tests.
- The benchmark’s fixed service boundaries and audit trails provide a template for such governance controls.
- Dependencies: internationally compatible standards, access to auditable system records, secure handling of proprietary data, and agreement on domain-specific risk thresholds.
- General-purpose platforms for data-centric AI experimentation — AI Infrastructure
- A future commercial platform could expose standardized APIs for hypothesis generation, dataset synthesis, validation, bounded fine-tuning, sandbox evaluation, experiment tracking, and deployment approval.
- It could automatically construct Pareto frontiers over quality, cost, latency, and risk, while recommending whether to continue searching or deploy the current best checkpoint.
- This would turn data-centric post-training research into a reusable MLOps capability across sectors.
- Dependencies: scalable training services, robust cross-task evaluation, reliable metadata and provenance, reproducible environments, and algorithms that can generalize beyond the six benchmark families studied in the paper.
- Contamination-resistant evaluation of continuously evolving models — Academia/Policy
- Because autonomous systems may eventually train on their own interaction histories, future benchmarks should use private, refreshed, and statistically held-out tasks to measure genuine transfer rather than memorization.
- This would enable certification of whether a model-improvement system produces durable capability gains across new repositories, languages, environments, and problem distributions.
- Dependencies: continual benchmark maintenance, secure test-set governance, repeated trials, and methods for separating genuine generalization from evaluation leakage.
Glossary
- Agent harness: A software framework that manages an AI agent’s interaction with tools, environments, and services. “Agent Harness”
- Agentic evaluation: Evaluation of AI systems that act autonomously through multi-step interactions and tool use. “Automated post-training and agentic evaluation.”
- Agentic training: Training designed to improve an AI system’s ability to perform actions, use tools, and complete tasks autonomously. “training experience for tool-using agents”
- Auditable: Designed so that decisions, procedures, and outcomes can be traced and inspected. “This makes the loop auditable”
- Base model: A pretrained model used as the starting point for subsequent adaptation or fine-tuning. “given a fixed base model”
- Behavior-aligned data: Training examples whose contents and supervision correspond closely to the behavior evaluated by a benchmark. “behavior-aligned data”
- Capability gap: A deficiency between a model’s current abilities and the abilities required for a target task. “diagnosing a capability gap”
- Checkpoint: A saved version of a model’s parameters produced during or after training. “Each is evaluated by a fixed evaluation service”
- Checkpoint preservation: Retaining a previously strong model checkpoint instead of replacing it with a weaker later result. “preservation of strong checkpoints”
- Closed-loop evolution: An iterative improvement process in which system outputs or evaluation results guide subsequent modifications. “Closed-loop Evolution”
- Closed-loop data synthesis: Generating training data, evaluating the resulting model, and using the evaluation feedback to generate revised data. “3.4. Closed-Loop Data-Synthesis Research”
- Configuration whitelist: A restricted set of training parameters or options that an agent is permitted to modify. “a whitelisted training config”
- Contamination-resistant evaluation: Evaluation designed to reduce the possibility that test information has entered the model’s training data. “fresh contamination-resistant evaluations”
- Curriculum design: Organizing training examples by difficulty, progression, or learning sequence. “Filtering, verification, curriculum design”
- Data-centric post-training: Improving a pretrained model primarily by designing, selecting, or synthesizing training data rather than changing the entire learning system. “data-centric post-training research”
- Data contract: A specification defining the required structure and validity conditions of submitted training artifacts. “satisfy the data contract”
- Data distribution: The statistical pattern of examples, tasks, labels, or behaviors represented in a dataset. “the submitted data distribution”
- Data-synthesis policy: A strategy governing how an agent constructs, filters, validates, and submits training data. “the agent’s data-synthesis policy”
- Depth–breadth tradeoff: A resource allocation tension between exploring many candidates and developing fewer candidates more extensively. “a depth–breadth tradeoff”
- Discovery–reliability gap: The difference between an agent’s ability to discover an improved candidate and its ability to improve consistently from subsequent feedback. “Our results reveal a discovery–reliability gap.”
- Executable environment: A task setting in which an agent can run code or interact with software and receive concrete execution results. “SWE-Gym and R2E-Gym provide executable environments”
- Feedback-driven revision: Modification of a training or research strategy based on observed evaluation results. “feedback-driven revisions”
- Fixed infrastructure: A shared and unchanging set of model-serving, training, evaluation, and system components used to compare agents. “RSIBench-Data fixes the infrastructure”
- Frontier agent: A highly capable, state-of-the-art AI agent used as a research subject or benchmark participant. “We evaluate four frontier LLM researcher agents”
- Historical-best selection: Choosing the checkpoint with the highest observed score over the entire sequence of attempts. “Historical-best checkpoint selection”
- Imitation data: Training examples created by reproducing or learning from an existing model’s successful behavior. “it creates a near-no-op adapter from one 39-token example”
- Instruction-tuned model: A model further trained to follow natural-language instructions. “the target is the instruction-tuned moonshotai/Kimi-K2.6 model”
- Long-horizon interaction: A sequence of many dependent actions or decisions required to complete a task. “tasks that require long-horizon interaction”
- LoRA (Low-Rank Adaptation): A parameter-efficient fine-tuning method that trains low-rank update matrices while leaving most base-model parameters fixed. “Each attempt trains a LoRA-adapted checkpoint”
- LoRA rank: The dimensionality of the low-rank matrices used in a LoRA adaptation. “a lower learning rate and LoRA rank improve to 21%”
- Message-format supervision: Training supervision represented as structured conversational messages, such as user and assistant turns. “the agent submits message-format supervision”
- Non-monotonicity: A pattern in which successive attempts do not consistently improve and may regress to lower performance. “This non-monotonicity shows that agents do not consistently turn feedback into better training-data strategies.”
- Official evaluation: The benchmark’s designated final assessment, conducted separately from the feedback used during iterative research. “The evaluator then assesses it in fresh task environments”
- Parameter-efficient fine-tuning: Model adaptation that updates relatively few parameters instead of retraining the entire model. “LoRA-adapted checkpoint”
- Pareto frontier: The set of candidates that are not simultaneously worse than another candidate across objectives such as score and cost. “Pareto frontier”
- Post-training stack: The collection of training, optimization, serving, evaluation, and implementation components surrounding a pretrained model. “the post-training stack”
- Recursive self-improvement: An iterative process in which an AI system uses evidence about its own limitations to produce increasingly capable versions of itself. “Recursive self-improvement requires systems”
- Researcher policy: A rule or decision-making procedure that maps available evidence and prior attempts to a new training-data proposal. “a researcher policy observes and its attempt history”
- Rollout model: A model used to generate reasoning traces, tool interactions, or task trajectories for training data. “Claude Opus 4.8 is fixed as the external rollout model”
- Selection feedback: Evaluation signals available to an agent for choosing or revising candidates before official testing. “use selection feedback, not official outcomes”
- Selection score: A score returned during development and used to compare candidate checkpoints. “Valid attempt Selection score (%)”
- Serving path: The technical route through which a trained model receives inputs and produces outputs during evaluation. “the base model, optimizer, serving path”
- Shared SFT backend: A common supervised fine-tuning service used by all compared agents. “the same Tinker-backed LoRA SFT backend”
- Supervised fine-tuning (SFT): Training a model on input–output examples in which desired responses are supplied as supervision. “Shared SFT training”
- Synthetic data: Artificially generated training examples rather than examples collected directly from real-world interactions. “synthetic-data strategy”
- Target model: The model that the researcher agent attempts to improve through training. “a fixed target model”
- Task trajectory: The ordered record of an agent’s actions, tool calls, observations, and outcomes while completing a task. “the resulting trajectories make failures richly observable”
- Tool-call proxy: An intermediary component that represents or forwards an agent’s requests to external tools. “Tool-call proxy”
- Training exposure: The amount and form of training experience presented to a model. “training exposure”
- Training experience: Examples, trajectories, environments, and supervision used to adapt a model. “Training experience includes examples, executable tasks and states”
- Training-data strategy: A planned method for selecting, generating, filtering, mixing, and validating data for model training. “designing and validating a training-data strategy”
- Trajectory representation: The format used to encode a sequence of actions, observations, and tool interactions as training data. “trajectory representation”
- Verifier-grounded supervision: Training supervision supported or checked by an executable verifier. “verifier-grounded supervision”
- Whitelisted configuration: A configuration selected from a benchmark-approved set of allowable training options. “a whitelisted configuration ”