DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses
Abstract: Scientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging. We introduce an operational definition of valuable and testable research hypotheses: evidence-grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. Our DiscoveryRSI framework develops hypothesis trees through multi-agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. We also introduce DFM-Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. Across four life-science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement, with the large gains in falsifiability and data resolvability. We further validate hypothesis quality through expert review, comparing our DiscoveryRSI-27B with other research agents. Preliminary human assessments show that DiscoveryRSI-27B scores higher than GPT-6 Luna on hypothesis importance and whether claims are plausible and evidence-bounded. DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths.
First 10 authors:
Summary
- The paper introduces a new recursive learning framework, DiscoveryRSI, for discovering valuable and testable scientific hypotheses, emphasizing structured claim development in evidence-grounded investigations summarized in integer scores.
- Why-Depict Nature to-Matter curves observed-the-impression syntheses-highlights systems spanning numerical infeasibility due to algorithmic behavioural-circuitry.
- The methodology improves scientific investigations across life sciences and battery domains and automated evaluation to reveal a lack of externally independent extrapolated robustness.
- Followقتessional discover optimize problem setting credit-claim AI simulations
DiscoveryRSI proposes a framework for treating scientific hypothesis development as a recursive learning problem rather than as a one-shot generation task. Its central premise is that a research agent should learn not only from completed investigations, but also from the documented sequence of evidence gathering, proposal formation, critique, revision, task compilation, and scientific assessment that precedes execution. The paper introduces both an agent architecture and an evaluation framework intended to measure whether this process produces hypotheses that are consequential, evidence-grounded, falsifiable, and practically investigable (2610.23839).
Problem formulation and conceptual contribution
The paper defines a valuable and testable research hypothesis as a structured object comprising four components: an evidence-grounded claim, an investigation plan, explicit decision criteria, and relevant alternative explanations. This definition is deliberately stronger than a research “idea.” A claim is valuable when resolving its underlying uncertainty could change scientific understanding or a justified research decision relative to the available evidence. It is testable when the proposed investigation is feasible under stated data and resource constraints and can produce observations that support, weaken, or bound the claim.
This formulation separates scientific quality from truth. A hypothesis may be important and well designed without being correct, while a computationally executable task may still be scientifically uninformative. The distinction is reflected in the paper’s insistence that execution readiness, scientific merit, and empirical confirmation be recorded as separate properties. The operational object is therefore not merely a proposition , but a coupled structure , where is the investigation plan, the decision rule, and the set of alternative explanations.
The MIMIC-IV example makes this distinction concrete. The proposal asks whether repeated creatinine–urine-output mismatch provides information about ICU readmission or in-hospital death shortly after ICU discharge. It compares progressively richer logistic models, adds hidden Markov model summaries only where data support and stability checks permit, and specifies patient-level train, tuning, and test splits. The proposal does not assert a clinical outcome; it specifies what would count as evidence for or against incremental predictive value.
Figure 1: A MIMIC-IV hypothesis represented as a claim, comparison plan, decision criteria, and alternative explanations.
This design has an important implication for automated scientific discovery: the target of generation is not novelty in isolation, but a bounded research opportunity. The framework permits informative null results, replications, measurement checks, and uncertainty reduction to count as valuable contributions when they can alter subsequent research decisions.
DiscoveryRSI architecture
DiscoveryRSI organizes research into episodes, hypothesis trees, linked records, and recursive self-improvement rounds. Within an episode, a Lead agent coordinates Generation and Evolution agents. Generation proposes hypotheses, Evolution develops or repairs them, Critique compares candidates, and Progress evaluates whether a revision constitutes substantive advancement, necessary repair, repetition, or insufficiently supported change. Crossover can combine contributions from multiple parents, while restart opens a new research direction without creating a false ancestry relation.
The resulting “research tree” is technically a directed acyclic graph because crossover nodes may have multiple parents. Each node stores a versioned claim, investigation plan, decision criteria, alternatives, evidence, and ancestry. The broader record layer links hypotheses to literature, datasets, scripts, artifacts, assessments, agent contributions, task executions, and delivery records.
Figure 2: DiscoveryRSI links multi-agent hypothesis development, executable investigations, contribution attribution, harness adaptation, and recursive rediscovery.
This provenance structure addresses a central credit-assignment problem. A final proposal may reflect a Generation agent’s initial explanation, an Evolution agent’s methodological repair, and a Lead agent’s allocation decision. Retaining only the final text would obscure which actions produced the improvement. DiscoveryRSI instead records agent invocations and contribution links so that scientific assessments can be propagated to the specific histories that supported the proposal.
Selected investigations are compiled into Harbor tasks containing inputs, environments, required outputs, and verification checks. Compilation is intended to preserve the scientific commitments of the original hypothesis, including population, comparison, discriminating outcomes, and interpretation criteria. However, the paper explicitly maintains that a successful compilation or execution check does not establish scientific validity.
Recursive self-improvement mechanisms
DiscoveryRSI combines three improvement mechanisms: hypothesis credit optimization, harness replay, and rediscovery.
Hypothesis credit optimization
Hypothesis credit optimization transforms scientific assessments into weighted supervision for model training. Blinded pairwise Critique judgments are aggregated with a TrueSkill-style model to estimate relative preference strength and uncertainty. These scores are converted into bounded hypothesis weights using conservative ranking, effective comparison counts, uncertainty penalties, and Progress judgments.
Credit is then allocated to documented contributor histories. The allocation is provenance-based rather than role-based: Generation, Evolution, and Lead trajectories can receive credit when records establish that they proposed, supported, or coordinated the relevant work. Unsupported ancestry or mere participation does not generate credit. Local and global scientific-family caps reduce the risk that repeated revisions of one question dominate training.
The resulting objective is weighted supervised fine-tuning over supported assistant responses and tool-call arguments. Tool observations, failed actions, unsupported actions, and repeated context remain available as conditioning information but receive no direct loss. The training release contains 15,792 collected trajectories, of which 2,397 receive eligible supervision. These sequences contain 291.12 million tokens, but only 6.18 million tokens—2.12%—receive direct loss. The model is DiscoveryRSI-27B, trained with full-parameter fine-tuning for one epoch using BF16 precision, Adam, a learning rate of , and a global batch size of 16.
The paper is appropriately cautious about the interpretation of this procedure. The credit weights are not estimates of causal marginal contribution, and they do not guarantee corresponding shares of gradient magnitude. Likewise, TrueSkill scores summarize relative rubric-conditioned preference rather than the probability that a hypothesis is true. These qualifications are essential because the training signal is several transformations removed from scientific confirmation.
Harness replay
Harness replay improves the agent workflow while holding model weights fixed. The procedure diagnoses recurring failures such as repeated exploration, ineffective tool use, lost artifacts, premature closure, weak handoffs, or inappropriate research allocation. Candidate changes to coordination, evidence access, review timing, and completion checks are evaluated under matched resource limits on development questions.
The incumbent harness remains eligible, and a candidate is accepted only if development evaluation supports the change. This separates workflow adaptation from model training and avoids treating every observed failure as evidence that the model parameters should be updated. The paper nevertheless does not report a full ablation isolating the causal contributions of each harness intervention.
Rediscovery
Rediscovery applies accepted model and harness updates to fresh research questions. Model and harness candidates are evaluated separately and jointly, allowing the system to retain the original model, adopt the trained model, modify the harness, or update both. New investigations then generate additional hypotheses, assessments, contribution records, and execution outcomes.
The recursive loop is therefore not simply repeated self-training. It combines local research-tree evolution with cross-round changes to the model and research environment. The intended learning unit is the research process itself: future investigations should benefit from prior evidence about which revisions, coordination patterns, and workflow interventions produced useful scientific designs.
DFM-Bench and assessment design
The paper introduces DFM-Bench to evaluate scientific work at five levels: individual hypotheses, revisions, development paths, research runs, and scientific contribution families. This hierarchy is a substantial methodological contribution because it distinguishes local proposal quality from cumulative inquiry.
Individual hypotheses are scored across eight dimensions: importance, novelty, grounding, falsifiability, data resolvability, feasibility, alternative explanations, and actionability. Revisions are evaluated against their actual predecessors, rather than judged solely by their final absolute score. Development paths assess evidence acquisition, coherent revision, scientific continuity, and stopping decisions. Research-run evaluation considers the full graph, portfolio synthesis, exploration breadth, and budget allocation. Contribution-family analysis asks what distinct scientific increment was added beyond the starting evidence.
The framework also separates scientific quality from delivery maturity. A proposal can be proposed, selected, compiled, readiness-passed, executed, or independently reviewed, but these statuses do not collapse into a single score. This avoids treating a runnable script as evidence of a scientific finding.
The paper contrasts DFM-Bench with benchmarks that focus primarily on ideation, data-analysis correctness, executable programs, or process traces. Its stronger claim is not that existing benchmarks are ineffective, but that they do not jointly represent hypothesis merit, revision value, path progress, whole-run allocation, and contribution maturity.
Experimental scope
The principal evaluation covers four life-science domains: clinical and population health, therapeutic target prioritization, disease mechanisms, and population multi-omics. The authors additionally test a battery setting spanning electrical diagnostics, health prediction, degradation mechanisms, and usage-history-dependent aging.
Figure 3: The common evidence-to-question-to-investigation process used across life-science and battery domains.
The automated evaluation uses 241 research questions in the life-science experiments, with GPT-5.6 Sol serving as the evaluator. Behavioral analysis uses six campaigns of 20 episodes per model variant. Human evaluation covers 292 proposals from six biomedical datasets, reviewed by 27 doctors, medical students, and bioscience PhD students under three senior supervisors.
The battery-domain evaluation covers 200 episodes and 351 generated hypotheses. It reports a mean hypothesis quality of 5.61 on 165 complete scores, substantive progress on 214 of 366 transitions, a mean complete-trajectory score of 81.52 across 152 paths, and a research-round score of 71.25. Because the battery results are presented as a separate domain validation and do not receive the same detailed comparative analysis as the life-science results, they primarily establish transfer of the workflow rather than broad generality.
Automated quality improvements
The core automated result is a monotonic increase in mean hypothesis quality across two self-improvement rounds:
| System state | Mean hypothesis quality |
|---|---|
| Base model | 4.86 |
| After round 1 | 5.18 |
| After round 2 | 5.39 |
The second-round model exceeds the base model in all four life-science domains. Domain-specific gains range from 0.28 points in disease mechanisms to 0.94 points in population multi-omics. The incremental improvement decreases from 0.32 points in the first round to 0.21 points in the second, suggesting diminishing returns within the evaluated procedure.
Figure 4: Automated hypothesis quality increases across recursive self-improvement rounds, with the largest gains concentrated in testability-related dimensions.
The most pronounced improvements concern investigation specification rather than perceived scientific importance or originality. Falsifiability rises from 4.00 to 6.20, alternative-explanation handling from 4.44 to 5.98, data resolvability from 3.73 to 4.99, and feasibility from 4.79 to 5.87. Conversely, importance declines from 6.48 to 6.04 and novelty from 4.87 to 4.58.
This pattern is substantively informative. DiscoveryRSI appears to improve the operationalization of research questions—what can be measured, compared, falsified, and interpreted—more reliably than it improves the selection of more important or more original questions. The result also complicates any interpretation of the aggregate score as general scientific capability: the weighted gain is driven primarily by methodological specification.
The automated evaluator is not an independent human panel. The supplementary protocol states that the three reviewer roles use the same model family and that incomplete evidence remains unknown rather than being assigned zero. The measured improvement should therefore be interpreted as improvement under a frozen rubric and evaluator protocol, not as a fully independent estimate of scientific quality.
Behavioral changes induced by recursive learning
Self-improvement changes the allocation of research operations. Evolve and repair account for 77.5% of base-model episodes, 90.0% after the first round, and 91.7% after the second. Restart operations decline from 21.7% to 5.8% and then 5.0%. In episodes 11–20, restart frequency falls from approximately 32% for the base model to 3% after the second round.
Figure 5: Recursive improvement increases sustained evolution and repair while reducing late-stage research restarts.
Development-path scores increase most in evidential and inferential discipline, by 1.25 points, followed by technical and knowledge exploration, by 0.34 points, and scientific continuity and synthesis, by 0.24 points. Changes in cumulative progress, stopping decisions, and budget efficiency are below 0.1 points, while disciplinary connections decline by 0.28 points.
The behavioral interpretation is therefore specific: the trained system spends more effort repairing and extending existing lines of inquiry and produces stronger evidential discipline. It does not demonstrate corresponding gains in broad exploration, efficient budget allocation, or cross-disciplinary integration. Operation frequency alone is not evidence of improvement; its significance depends on whether the resulting revisions are scientifically useful.
Human assessment of revisions and proposal quality
Human review provides complementary evidence. Among assessed revisions, DiscoveryRSI-27B improves the mean human score in 41% of cases, leaves it unchanged in 47%, and worsens it in 12%. GPT-5.6 Sol achieves 31%, 37%, and 32%, respectively, while GPT-6 Luna achieves 25%, 44%, and 31%.
Mean score change per revision is:
| Model | Mean change per revision |
|---|---|
| DiscoveryRSI-27B | +0.07 |
| GPT-5.6 Sol | +0.03 |
| GPT-6 Luna | -0.01 |
Figure 6: Human reviewers identify more positive and fewer negative revision outcomes for DiscoveryRSI-27B than for the comparison systems.
The absolute criterion profile is less uniformly favorable. DiscoveryRSI-27B exceeds GPT-6 Luna on importance, 3.10 versus 2.94, and on plausible, evidence-bounded claims, 3.37 versus 3.16. Both systems score 2.31 on advance over known work. GPT-6 Luna scores higher on data fit, alternative explanations, and informative results. Thus, the stronger within-tree revision statistics coexist with weaknesses in the final proposals, particularly in data fit and handling of alternatives.
The authors illustrate revision progress with a MIMIC-IV path in which successive versions clarify the time origin, the 48-hour outcome landmark, data-field bindings, and acute kidney injury thresholds. The six-criterion mean rises from 2.83 to 3.50, while data fit and informative results each improve by two points.
Figure 7: A MIMIC-IV revision path improves through explicit temporal definitions, data bindings, and outcome thresholds, while sibling branches retain fatal design omissions.
This case also demonstrates why path-level evaluation is necessary. Two later sibling hypotheses retain fatal flaws because acute kidney injury thresholds remain unspecified. Depth or persistence in a research tree does not guarantee scientific progress, and a favorable terminal proposal does not validate every branch that preceded it.
The human comparison has important design limitations. Reviewers assessed 292 proposals, but the revision analysis concerns related parent–child pairs, which are statistically dependent through shared ancestry. The paper does not present a randomized intervention assigning identical questions to independently trained systems, nor does it establish inter-rater reliability in the reported results. The findings are therefore best understood as controlled, blinded comparative assessments of recorded research trees rather than unbiased estimates of model-wide causal improvement.
Literature correspondence and the meaning of inspiration
The final evaluation defines inspiration as a concrete correspondence between a generated hypothesis and post-cutoff literature. A qualifying publication must contribute an experimental design, comparator, measurement, or testable evidence gap relevant to developing or testing the hypothesis.
DiscoveryRSI-27B obtains 74 qualifying versions among 1,176 generated versions, for a rate of 6.3%. The comparison rates are 5.4% for DeepSeek, 2.8% for GLM, 2.5% for GPT, and 2.3% for Claude. DiscoveryRSI’s domain-specific rates are 16.9% in therapeutic target prioritization, 7.4% in population multi-omics, 6.8% in disease mechanisms, and 2.3% in clinical and population health.
Figure 8: DiscoveryRSI-27B has the highest measured rate of correspondence between generated hypothesis versions and qualifying post-cutoff literature.
The result is potentially important because it evaluates actionability rather than lexical novelty alone. Nevertheless, the paper explicitly characterizes the comparison as descriptive. Model groups differ in sample size, domain composition, cutoff specification, and retrieval coverage; multiple hypothesis versions may belong to the same research family; and duplicate judgments are propagated across 903 registered versions.
The Claude comparison particularly illustrates the sensitivity of this endpoint to representation. Only 15 of 131 Claude proposals pass the retrieval gate, compared with 355 of 1,176 DiscoveryRSI proposals. Conditional on retrieval, positive judgment rates are similar: 20.0% for Claude and 20.8% for DiscoveryRSI. Claude proposals are substantially longer, and the retrieval and semantic-audit procedures examine only initial excerpts, allowing provenance and revision material to displace scientific content. A title-based diagnostic increases Claude’s retrieval count from 15 to 95, but does not establish additional qualifying matches.
Accordingly, the 6.3% rate should not be interpreted as evidence that researchers were influenced by DiscoveryRSI, that its hypotheses were independently validated, or that the model generated discoveries before publication. It measures retrospective literature correspondence under a retrieval-and-judgment pipeline. Prospective studies would need to freeze proposals before researcher exposure and track adoption, experimental execution, and publication.
Limitations and open questions
The paper’s strongest claims are limited by several methodological dependencies. First, the principal quality gains rely substantially on automated grading by a related model family. Although the rubric is detailed and missing evidence is handled explicitly, evaluator correlation and prompt sensitivity remain possible sources of bias. Independent expert calibration on unseen cases is identified as pending.
Second, the training procedure lacks the ablations needed to separate the contributions of credit attribution, family-level credit caps, domain balancing, model fine-tuning, harness replay, and rediscovery. The observed gains establish that the complete protocol improves the evaluated system, but not which component is necessary or sufficient.
Third, human revision outcomes are not independent observations. Shared ancestry, repeated research families, and model-specific proposal distributions complicate significance testing and causal interpretation. The results demonstrate a favorable pattern of assessed revisions, not a definitive estimate of improvement on an independently sampled scientific workload.
Fourth, the framework evaluates hypothesis quality more directly than completed scientific contribution. Many investigations remain proposed or compiled rather than executed, and unavailable data or wet-lab dependencies remain explicit unresolved requirements. The paper therefore does not show that DiscoveryRSI produces confirmed empirical discoveries.
Finally, the literature-inspiration metric is sensitive to retrieval format, temporal cutoffs, semantic auditing, and duplicate propagation. The Claude diagnostic demonstrates that measured rates can change substantially when proposal representation changes. A remaining open question is whether DiscoveryRSI’s advantage persists under a common structured representation, identical retrieval coverage, independent expert adjudication, and prospective tracking of which hypotheses lead to actual investigations.
Conclusion
DiscoveryRSI formalizes scientific hypothesis development as a recursive, provenance-preserving learning process. Its principal technical contributions are the structured definition of valuable and testable hypotheses, research trees linked to executable investigations, contribution-aware supervision, harness replay, rediscovery, and the multi-level DFM-Bench evaluation framework.
Across two self-improvement rounds, automated hypothesis quality increases from 4.86 to 5.39, with the largest gains in falsifiability, alternative explanations, data resolvability, and feasibility. Human assessments report a mean revision gain of +0.07 for DiscoveryRSI-27B, compared with +0.03 and -0.01 for the comparison systems, while the literature-correspondence rate reaches 6.3%. These results support the narrower conclusion that documented research histories can improve the specification and revision of testable investigations. They do not yet establish confirmed scientific discovery, causal researcher influence, or general superiority across scientific domains and evaluation protocols.
Whiteboard
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Explain it Like I'm 14
1. What is this paper about?
This paper introduces DiscoveryRSI, an artificial intelligence system designed to help with scientific research.
The system does not simply generate random ideas. Instead, it tries to create useful scientific hypotheses—possible explanations or questions that:
- Matter for an important problem,
- Are supported by existing evidence,
- Can actually be tested,
- Include a clear plan for investigation,
- Explain what different results would mean, and
- Consider other possible explanations.
The main idea is that an AI system can become better at research by learning from its earlier attempts, mistakes, and improvements.
2. What questions are the researchers asking?
The paper focuses on one main question:
Can an AI research system learn from its own research history and gradually become better at creating and improving scientific hypotheses?
The researchers also ask several smaller questions:
- Does the system produce better hypotheses after learning from previous research?
- Does it become better at designing experiments and tests?
- Can it improve an idea through several rounds of revision?
- Do human experts think its revised hypotheses are better?
- Do the AI’s ideas connect to real scientific work published later?
3. How does DiscoveryRSI work?
DiscoveryRSI uses several AI “research agents.” Each agent has a different job, much like a research team.
For example:
- A Lead agent organizes the work.
- A Generation agent suggests new hypotheses.
- An Evolution agent improves existing hypotheses.
- A Critique agent looks for weaknesses.
- A Compiler turns a research plan into a computer-executable task.
Hypothesis trees
The system stores ideas in a structure called a hypothesis tree. Each new version of an idea is connected to the version before it.
This is similar to how a rough school project might improve over time:
- Someone suggests an idea.
- Another person notices a problem.
- The plan is repaired.
- Someone adds a better comparison.
- The final version becomes clearer and easier to test.
The system records these changes instead of saving only the final answer. This helps it learn which changes were useful.
Turning ideas into tasks
When possible, the system converts a hypothesis into an executable computer task called a Harbor task. This task includes:
- The data needed,
- The computer environment,
- The expected results,
- The comparisons to make, and
- Checks to see whether the task worked correctly.
For example, a medical hypothesis might ask whether a certain pattern in a patient’s data helps predict hospital readmission. The system would need to specify which patients to study, which measurements to use, what models to compare, and how to interpret the results.
Three ways the system learns
DiscoveryRSI improves through three main methods.
1. Hypothesis credit optimization
The system tries to determine which actions helped create a good hypothesis.
For example, perhaps one agent suggested the main idea, another added an important control group, and a third corrected a confusing measurement. Instead of rewarding every action equally, DiscoveryRSI gives more credit to the actions that made a real contribution.
This is similar to giving credit in a group project to the people who made useful improvements, rather than simply rewarding everyone who was present.
2. Harness replay
A harness is the set of tools and rules that controls how the AI works.
The researchers examine where the system often fails, such as:
- Repeating the same search,
- Losing important files,
- Using tools incorrectly,
- Ending research too early, or
- Failing to compare different explanations.
They then adjust the workflow. This is like changing the rules or equipment of a science laboratory after discovering that the original setup causes problems.
3. Rediscovery
After improving the AI and its workflow, the researchers give it new research questions.
The system then uses what it learned on these fresh problems. The process repeats:
- Research new questions.
- Record useful ideas and mistakes.
- Improve the AI or its workflow.
- Test it on new questions again.
This repeated cycle is why the paper uses the term recursive self-improvement.
4. How was the system tested?
The researchers tested DiscoveryRSI mainly on life-science topics, including:
- Clinical and population health,
- Medical treatment targets,
- Disease mechanisms, and
- Genetic and molecular data.
They also tested it on battery research, such as battery aging and health prediction.
The researchers created DFM-Bench, a test system for judging scientific research ideas. It looks at several levels:
| What is judged? | Simple meaning |
|---|---|
| Hypothesis quality | Is the idea important and testable? |
| Revision value | Did the new version improve the old one? |
| Development path | Did the idea become stronger over time? |
| Research run | Did the AI use its time and effort wisely? |
| Scientific contribution | Did the work add something useful to knowledge? |
The researchers used both computer-based grading and human review.
For the human review, 27 people with medical, biomedical, or clinical research experience examined 292 proposals. They did not know which AI system had created each proposal.
5. What did the researchers find?
The hypotheses improved after repeated learning
The average automated quality score increased as the system learned:
- Base model: 4.86
- After one improvement round: 5.18
- After two improvement rounds: 5.39
The biggest improvements were in designing investigations that could actually test the claims.
For example, the later hypotheses were better at:
- Explaining what evidence would prove or weaken an idea,
- Considering alternative explanations,
- Identifying what data were needed, and
- Describing how an investigation could be carried out.
The system became better at improving ideas instead of abandoning them
At first, the AI often restarted with a completely new question. After training, it spent more time repairing and developing existing ideas.
The percentage of episodes involving evolution or repair increased from 77.5% to 91.7%. At the same time, complete restarts decreased from 21.7% to 5.0%.
This is important because real scientific research usually requires improving an idea step by step rather than constantly starting over.
Human reviewers saw more improvement in its revisions
Human reviewers found that DiscoveryRSI’s revisions improved more often than those of the comparison systems.
The average change in score per revision was:
- DiscoveryRSI-27B: +0.07
- GPT-5.6 Sol: +0.03
- GPT-6 Luna: −0.01
DiscoveryRSI also had more revisions that improved a proposal and fewer that made it worse.
However, the system was not better at everything. Compared with GPT-6 Luna, it scored higher on:
- The importance of the research question, and
- Whether the claims were reasonable and supported by evidence.
GPT-6 Luna scored higher on some areas, including data fit, alternative explanations, and whether the results would be informative.
Some ideas matched later scientific publications
The researchers also checked whether the AI’s hypotheses were connected to research published after the AI’s knowledge cutoff.
They counted a connection when a later paper provided something concrete, such as:
- A useful experimental design,
- A comparison,
- A measurement, or
- A testable gap in evidence.
DiscoveryRSI had a 6.3% connection rate, compared with:
- GPT-6: 2.5%
- Claude Opus 5: 2.3%
This suggests that some DiscoveryRSI ideas were relevant to real research directions that later appeared in scientific literature.
6. Why are these findings important?
Scientific research is not only about having creative ideas. A good idea must also be:
- Clear,
- Important,
- Possible to investigate,
- Careful about uncertainty, and
- Able to distinguish between different explanations.
The study suggests that AI can improve in these areas when it remembers how ideas developed, which changes helped, and which mistakes happened repeatedly.
One especially important result is that the system improved most in testability, rather than simply becoming more imaginative. In other words, it became better at answering questions such as:
“What exactly should we measure, and what result would support or weaken this idea?”
That kind of careful planning is valuable because it can help scientists decide which projects are worth pursuing.
7. Possible impact and limitations
If systems like DiscoveryRSI continue to improve, they could help researchers:
- Find promising research questions,
- Design clearer experiments,
- Identify weaknesses before experiments begin,
- Compare competing explanations,
- Organize large amounts of scientific evidence, and
- Develop ideas more efficiently.
However, the system does not prove that its hypotheses are true. A hypothesis can be well designed and still turn out to be wrong. Also, some results in the paper were produced by automated judges, and the human evaluation was relatively limited. The paper itself shows that DiscoveryRSI still has weaknesses, especially in data fit, alternative explanations, and predicting how informative results will be.
Overall, the paper presents DiscoveryRSI as a step toward AI systems that do more than generate scientific ideas. Its goal is to create AI researchers that can develop, test, learn from, and improve scientific ideas over time.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
- Causal contribution of each RSI mechanism remains unresolved. The paper reports improvements after combining hypothesis-credit optimization, harness replay, and rediscovery, but does not isolate the effects of each component through a full factorial ablation.
- The source of performance gains is unclear. It is not established whether improvements arise primarily from model fine-tuning, workflow changes, increased revision depth, better evidence access, altered operation selection, or interactions among these factors.
- The effectiveness of hypothesis-credit attribution is not independently validated. The proposed contribution weights rely on documented histories and heuristic allocation rules, but the paper does not compare them with alternative credit-assignment methods or measure attribution accuracy against expert judgments.
- The training signal may be vulnerable to evaluator bias. Credit is derived from automated or supervised assessments, yet the paper does not quantify how grader errors, rubric interpretation, preference noise, or systematic domain biases affect model updates.
- The generalization of automated DFM-Bench scores is uncertain. Most large-scale quality results are produced by GPT-5.6 Sol, and the paper does not establish strong agreement between this evaluator and independent experts across all DFM-Bench dimensions.
- Potential evaluator–model contamination is not fully addressed. The paper does not clarify whether the automated judge, reference materials, or evaluation prompts overlap with training data or development trajectories used by DiscoveryRSI.
- Human validation is limited in scale and scope. The expert study includes 292 proposals from 27 biomedical or clinical reviewers, but it does not demonstrate that findings generalize to larger expert populations, non-biomedical disciplines, or researchers with different levels of subject-matter expertise.
- Inter-rater reliability is not reported in the provided results. Agreement statistics, calibration effects, handling of reviewer disagreements, and the reliability of the “can't judge” and fatal-flaw judgments remain unclear.
- The human comparison design may not fully control for model or tree differences. The paper compares revision outcomes across model-generated research trees, but does not establish that models received matched questions, evidence, search budgets, context windows, and opportunities for revision.
- The interpretation of higher revision gains is ambiguous. DiscoveryRSI has better mean change per revision, but it scores lower than GPT-6 Luna on data fit, alternative explanations, and informative results; the paper does not determine whether the improvement metric reflects genuine scientific progress or selective revision behavior.
- The reported gains may partly reflect reduced exploration. Restarts fall sharply while evolve and repair operations increase, yet the paper does not determine whether this improves discovery or merely causes the system to exploit existing directions and overlook better research questions.
- Research-question novelty declines after self-improvement. Novelty falls from 4.87 to 4.58, but the paper does not investigate whether this reflects useful specialization, mode collapse, training-data imitation, or reduced ability to generate genuinely new hypotheses.
- Broader scientific exploration remains underdeveloped. Improvements in exploration breadth, stopping decisions, budget efficiency, and disciplinary connections are small or negative, leaving unresolved whether DiscoveryRSI can discover valuable directions rather than mainly improve proposal specification.
- Long-term recursive improvement is not demonstrated. Results cover only two self-improvement rounds, so it is unknown whether performance continues to improve, plateaus, oscillates, or degrades over additional cycles.
- The system’s resistance to recursive error amplification is untested. Reusing model-generated hypotheses, assessments, and trajectories as training data could reinforce systematic mistakes, unsupported assumptions, or narrow research preferences.
- Out-of-distribution generalization remains uncertain. The main experiments focus on four life-science domains, and the battery setting is reported separately; broader transfer to other scientific fields, languages, data modalities, and experimental paradigms is not established.
- Physical and wet-lab research capability is largely unresolved. Harbor tasks primarily support computational investigations, while unavailable measurements and physical experiments remain dependencies rather than executed tests. The framework’s usefulness for laboratory, field, or clinical studies is therefore not demonstrated.
- Executable-task success is insufficiently connected to scientific validity. The paper distinguishes Harbor contract compliance from scientific assessment, but does not report how often technically successful tasks produce valid, reproducible, or decision-relevant scientific evidence.
- Reproducibility of generated investigations is not established. The paper does not provide systematic results on rerun stability, sensitivity to random seeds, data-split choices, software environments, or changes in external literature and tool availability.
- The reference-actionability metric is only a proxy for discovery. A post-cutoff publication’s correspondence with a generated hypothesis does not show that the hypothesis caused, enabled, or independently anticipated the research, and the paper does not validate this metric against prospective scientific impact.
- The 6.3% correspondence rate may be sensitive to matching decisions. The paper does not fully establish how semantic matches, partial overlaps, publication selection, temporal cutoffs, duplicate ideas, and evaluator judgments affect the reported comparison with other models.
- Actual scientific outcomes are not measured. The study evaluates hypothesis quality, revision progress, and literature correspondence, but does not show whether generated hypotheses lead to successful experiments, published findings, improved clinical decisions, or durable reductions in scientific uncertainty.
- Resource efficiency is incompletely characterized. Although the framework claims to consider cost, the reported results do not establish comparative compute, token, tool-use, wall-clock, or human-review costs per useful hypothesis or validated contribution.
- The impact of the 256K-token context and full-parameter fine-tuning is unclear. No comparison with smaller contexts, parameter-efficient updates, or alternative training budgets is provided, limiting understanding of which resources are necessary for the gains.
- The high degree of selective supervision raises scalability questions. Only 2.12% of training tokens receive direct supervision, but the paper does not analyze whether this sparse masking improves learning, causes unintended behavioral changes, or scales to much larger and more heterogeneous research histories.
- The operational definition of “valuable” remains domain- and evaluator-dependent. The framework defines value relative to evidence and constraints, but does not resolve how consequentiality, novelty, replication value, or decision relevance should be calibrated across disciplines.
- The relationship between hypothesis quality and truth remains unresolved. The paper explicitly distinguishes comparative quality from truth, yet it does not evaluate calibration, empirical correctness, or the rate at which high-scoring hypotheses are later supported or falsified.
- Alternative explanations may be specified without being meaningfully discriminated. The framework rewards explicit alternatives and decision criteria, but does not test whether proposed investigations possess sufficient statistical power or identifiability to distinguish them.
- Statistical validity of proposed analyses is not comprehensively assessed. Data resolvability and feasibility scores do not establish adequate sample size, power, missing-data handling, multiple-testing control, causal identification, or robustness to model misspecification.
- Potential data leakage and benchmark memorization are not fully ruled out. The paper does not show whether generated proposals reproduce known dataset-specific research patterns or exploit information contained in public corpora rather than deriving novel, evidence-grounded insights.
- The stability of research trees under different agent configurations is unknown. The effects of agent count, role specialization, critique frequency, operation ordering, and crossover policy are not systematically evaluated.
- The benefits and risks of crossover remain unclear. Combining multiple parent hypotheses may enable synthesis, but the paper does not measure whether crossover produces genuinely integrative insights or merely concatenates incompatible claims and methods.
- Stopping and selection policies are insufficiently validated. The framework records stopping decisions and research allocation, but does not establish whether the Lead reliably selects the best branches or whether early selection suppresses promising but initially immature ideas.
- The framework’s human oversight requirements are unspecified. It remains unclear how much expert intervention is needed to define questions, verify evidence, resolve conflicts, approve tasks, and assess contributions for reliable deployment.
- Safety and ethical risks are not systematically examined. Biomedical and clinical hypothesis generation can introduce privacy, fairness, misuse, or harmful-treatment risks, but the paper does not evaluate safeguards for these failure modes.
- The durability of learned behavior is unknown. The paper does not test whether self-improvement persists when the system encounters new tools, updated evidence, unfamiliar datasets, or research questions outside the training distribution.
- Comparisons with competing agents may be affected by model-version uncertainty. The paper names models such as GPT-5.6 Sol, GPT-6 Luna, and Claude Opus 5, but does not provide sufficient information about exact versions, prompts, system instructions, tool configurations, or reproducibility of these baselines.
- The battery-domain results are not integrated into the main claims. Although battery experiments are reported, the paper does not clearly compare their recursive-learning effects, human validity, or literature correspondence with the life-science results.
- The statistical significance and uncertainty of several behavioral findings are unclear. Operation-frequency changes and small shifts in budget efficiency, stopping, and disciplinary connections are reported descriptively, without consistently showing episode-level variance, corrected significance tests, or effect sizes.
- It is unknown whether gains transfer to independent research questions rather than the same question families. The evaluation uses fixed domain batteries and research-question sets, so performance on independently constructed, temporally later, or adversarially selected questions remains unresolved.
Practical Applications
Immediate Applications
The paper’s methods are sufficiently operational for deployment as decision-support, research-management, and evaluation workflows, provided that outputs are treated as candidate hypotheses rather than validated scientific conclusions.
- Biomedical and clinical research prioritization — healthcare, academia
- Use DiscoveryRSI to convert clinical datasets and literature into ranked, evidence-grounded research proposals containing a claim, investigation plan, decision criteria, and alternative explanations.
- Suitable inputs include MIMIC-IV, eICU, UK Biobank, GEO, and institutional datasets.
- A practical workflow is: dataset profiling → hypothesis generation → multi-agent critique → revision tree → feasibility screening → researcher review.
- The system could help identify studies worth pursuing, such as whether a repeated creatinine–urine output mismatch provides predictive information after ICU discharge.
- Dependencies: access to properly governed clinical data, domain-expert review, privacy protections, and explicit separation between research hypotheses and clinical recommendations. The paper does not demonstrate improved patient outcomes or clinical safety.
- Automated research-protocol drafting — healthcare, life sciences, pharmaceutical research
- Generate preliminary study protocols that specify populations, measurements, controls, model comparisons, outcome windows, decision thresholds, and alternative explanations.
- This can support retrospective cohort studies, biomarker discovery, therapeutic-target prioritization, and molecular-mechanism investigations.
- The system’s emphasis on falsifiability, data resolvability, and feasibility can be incorporated into protocol templates or electronic research notebooks.
- Dependencies: accurate data dictionaries, reliable literature retrieval, correct mapping between proposed variables and available measurements, and expert validation before execution.
- Research-plan quality assurance — universities, laboratories, grant offices
- Deploy DFM-Bench-style assessments as a pre-registration or internal review checklist.
- Proposals can be scored on importance, novelty, evidence grounding, falsifiability, feasibility, alternative explanations, data fit, and actionability.
- Research groups could use the resulting report to flag missing controls, ambiguous endpoints, unsupported causal claims, or experiments that cannot distinguish competing explanations.
- Dependencies: calibrated rubrics and reviewers; automated scores should be used for triage rather than replacing peer review. The reported automated evaluations rely substantially on model-based judging.
- Versioned hypothesis and provenance management — research software, knowledge management
- Implement research trees or directed acyclic graphs linking hypotheses to evidence, datasets, code, drafts, critiques, execution records, and contributor actions.
- This would make it possible to answer practical questions such as: Which revision introduced the control? Which evidence caused the hypothesis to change? Which contributor supplied the useful comparison?
- Potential products include a “Git for research hypotheses,” extensions to electronic laboratory notebooks, and provenance-aware grant or manuscript systems.
- Dependencies: stable identifiers, reproducible file storage, contribution metadata, access-control policies, and agreed semantics for ancestry, shared evidence, and scientific contribution.
- Computational experiment compilation — software engineering, data science
- Convert selected research plans into executable task specifications similar to Harbor tasks, including inputs, environments, required outputs, validation checks, and interpretation criteria.
- This can support reproducible machine-learning experiments, statistical analyses, battery diagnostics, and computational biology pipelines.
- A generated task might automatically create a container specification, dataset split, baseline models, evaluation script, and expected artifact list.
- Dependencies: available data, executable code, compatible environments, secure sandboxing, and separate checks for technical completion versus scientific validity.
- Agent workflow optimization — AI engineering, software agents
- Apply harness replay to diagnose recurring agent failures such as repeated exploration, lost artifacts, premature stopping, ineffective tool use, or poor task allocation.
- Teams can compare alternative prompts, tool permissions, review schedules, memory policies, and agent-role assignments under matched budgets.
- This is immediately useful for coding agents, data-analysis agents, literature-review systems, and internal research assistants.
- Dependencies: detailed execution logs, reproducible session state, controlled development sets, cost monitoring, and safeguards against optimizing for superficial benchmark scores.
- Battery and energy-system diagnostics — energy, automotive, hardware
- Use the same hypothesis-development process for battery capacity loss, impedance changes, degradation mechanisms, state-of-health prediction, and path-dependent aging.
- Candidate investigations can explicitly separate measured quantities from fitted degradation labels and enforce cell-, condition-, and protocol-level splits.
- Potential outputs include experiment prioritization tools, battery diagnostic protocols, and research assistants for battery-management-system development.
- Dependencies: sufficiently representative aging data, controlled operating conditions, sensor quality, physical plausibility checks, and validation on unseen cells and protocols.
- Literature-to-experiment opportunity discovery — academia, R&D, technology scouting
- Search for links between generated hypotheses and later publications that provide a design change, comparator, measurement, or testable evidence gap.
- Organizations could use this to identify research opportunities, update internal roadmaps, and detect when an apparently novel question has already acquired relevant evidence.
- The reported 6.3% post-cutoff correspondence rate suggests potential for structured literature scouting, but not reliable autonomous discovery.
- Dependencies: comprehensive and current literature access, temporal cutoff control, high-quality citation matching, and human verification of whether the correspondence is scientifically meaningful.
- Education and researcher training — universities, graduate programs
- Use hypothesis trees and revision records to teach students how to turn observations into falsifiable questions, define controls, identify alternative explanations, and establish decision criteria in advance.
- Instructors could provide an initial proposal and ask students to perform evidence-based repairs, with DFM-Bench-like feedback.
- Dependencies: discipline-specific rubrics, instructor oversight, and safeguards against students treating automated scores as definitive assessments of scientific quality.
- Policy and public-sector research portfolio management — government, funding agencies
- Use structured hypothesis assessments to compare proposed studies by feasibility, expected information value, evidence gaps, and resource requirements.
- Research programs could maintain a portfolio of competing hypotheses and track whether successive investments produce substantive revisions or merely repeated proposals.
- Dependencies: transparent scoring policies, democratic and domain-specific priorities, protection against bias in automated ranking, and independent review for high-impact funding decisions.
- Everyday structured decision support — personal analytics and professional planning
- Adapt the four-part hypothesis format for non-scientific decisions: claim, evidence, test or information-gathering plan, decision criteria, and alternative explanations.
- Examples include testing productivity interventions, comparing household energy-saving measures, evaluating a fitness routine, or deciding whether a software tool improves work output.
- Dependencies: low-stakes use, reliable personal measurements, awareness of confounding factors, and avoidance of medical, legal, or financial decisions without qualified advice.
Long-Term Applications
The following uses would require additional validation, broader data, integration with physical experimentation, or stronger governance before they could be deployed reliably at scale.
- Closed-loop autonomous scientific laboratories — robotics, life sciences, materials, chemistry
- Integrate DiscoveryRSI with robotic laboratories so that the system proposes hypotheses, selects experiments, executes them, interprets results, and recursively improves future investigations.
- Research trees could guide experiment selection while decision criteria determine whether to continue, revise, abandon, or branch into alternative explanations.
- Potential products include autonomous platforms for drug screening, materials synthesis, battery aging, and biological assay optimization.
- Dependencies: reliable laboratory robotics, safe chemical and biological handling, accurate sensors, experiment reproducibility, robust causal interpretation, and human authorization for consequential actions.
- Clinical trial and translational-research design — healthcare and pharmaceuticals
- Use the framework to generate and compare candidate trial designs, eligibility criteria, endpoints, subgroup analyses, controls, and stopping rules.
- A mature system might identify which uncertainties are most decision-relevant before a trial begins and propose designs that distinguish treatment effects from confounding explanations.
- Dependencies: prospective validation, regulatory approval, statistical guarantees, patient-safety oversight, fair representation of populations, and strict separation between automated proposal generation and clinical decision-making.
- AI systems that learn from scientific contribution rather than final answers — machine learning
- Extend hypothesis credit optimization into a general training paradigm in which model updates are weighted by documented contributions, evidence quality, and downstream usefulness.
- This could improve research agents by reinforcing useful controls, measurements, revisions, and coordination decisions rather than simply rewarding final text.
- Potential applications include continual learning for coding agents, data-science assistants, robotics planners, and multi-agent scientific systems.
- Dependencies: reliable attribution, resistance to reward hacking, protection against biased or incorrect expert judgments, prevention of catastrophic forgetting, and independent evaluation on unseen research questions.
- Scalable scientific knowledge graphs and research memory — academia, enterprise R&D
- Build persistent knowledge systems that connect claims, evidence, methods, failed attempts, alternative explanations, datasets, code, and experimental outcomes across institutions and research programs.
- Such systems could identify recurring failures, transfer useful investigation strategies between fields, and expose unresolved evidence gaps.
- Dependencies: interoperable metadata standards, intellectual-property agreements, privacy controls, provenance integrity, long-term storage, and mechanisms for updating or retracting unreliable evidence.
- Cross-domain discovery beyond life science and batteries — engineering, climate, finance, social science
- Apply the framework to energy-grid reliability, semiconductor failure analysis, climate intervention assessment, financial risk modeling, public-policy evaluation, and robotics.
- The paper’s battery experiments suggest some transferability, but each domain would require tailored constraints, measurement models, safety checks, and evaluation criteria.
- Dependencies: domain-specific evidence bases, appropriate causal assumptions, valid simulations or experiments, and benchmarks that measure real-world utility rather than textual plausibility.
- Research-resource allocation and portfolio optimization — funding agencies, corporate R&D
- Combine run-level DFM assessments with budget records to decide which research branches deserve additional computation, laboratory time, data collection, or personnel.
- The system could prioritize investigations that maximize expected reduction in consequential uncertainty rather than merely generating novel ideas.
- Dependencies: reliable estimates of experiment cost and information value, explicit organizational priorities, uncertainty-aware decision models, and protection against systematically underfunding unconventional or minority research directions.
- Autonomous replication and evidence auditing — academia, publishing, research integrity
- Use executable Harbor-style tasks to reproduce published analyses, verify data-processing steps, test alternative specifications, and record whether claims remain supported.
- Hypothesis trees could distinguish replication, extension, informative negative results, and genuine new contributions.
- Dependencies: access to original data and code, reproducible environments, publication cooperation, legal permissions, and methods for handling confidential or proprietary datasets.
- Policy foresight and evidence-based regulation — government and international organizations
- Represent policy claims as hypotheses with measurable outcomes, competing explanations, intervention plans, and pre-specified decision criteria.
- This could support evaluation of public-health interventions, energy policies, education reforms, or labor-market programs.
- Dependencies: high-quality longitudinal data, causal identification, stakeholder participation, fairness assessments, and safeguards against using automated recommendations to bypass democratic accountability.
- General-purpose scientific co-pilots with calibrated autonomy — professional and daily life
- A mature system could act as a continuously learning assistant that maintains a user’s evidence base, proposes tests, tracks outcomes, and updates recommendations.
- In daily life, this might support household energy management or personal learning; in professional settings, it could manage engineering investigations or operational experiments.
- Dependencies: calibrated uncertainty, user control, data privacy, protection against overconfident recommendations, and strong boundaries for health, legal, financial, and safety-critical domains.
- Formal standards for evaluating scientific AI — academia, policy, industry
- DFM-Bench could evolve into a broader standard for assessing not only whether an AI generates plausible ideas, but whether it improves hypotheses, uses evidence, allocates research effort, preserves provenance, and produces independently supported contributions.
- Such standards could be used in procurement, model release evaluation, grant review, and regulatory audits.
- Dependencies: independent benchmark governance, resistance to benchmark overfitting, expert consensus on rubrics, cross-domain validation, and evaluations based on completed research rather than generated proposals alone.
Glossary
- Alternative explanation: A competing account that could explain the same observation or result. “The alternative explanations identify competing accounts, confounding, or measurement limitations”
- Ancestor-aware progress: Assessment of progress that accounts for a hypothesis’s position and predecessors in a research lineage. “ancestor-aware Progress assesses substantive progress, necessary repair, repetition, or insufficient evidence.”
- Artifact: A file, dataset, output, or other research object produced or used during an investigation. “Files preserve the research material; registry entries, identifiers, paths, and available content hashes preserve its relationships.”
- Bayesian posterior: A probability distribution updated using observed evidence. “$p(z_h\mid\mathcal D_{\mathrm{cmp}) \approx \mathcal N(\mu_h,\sigma_h^2)$.”
- Benchmark: A standardized dataset, task, or evaluation framework used to compare systems. “We introduce DFM-Bench to evaluate hypothesis quality and cumulative research progress.”
- BF16 precision: A 16-bit numerical format commonly used to reduce memory and computational requirements during neural-network training. “We train for one epoch using Adam with a learning rate of , a global batch size of 16, and BF16 precision.”
- Blinded critique: Evaluation in which reviewers assess scientific merit without access to potentially biasing information. “Blinded Critique compares scientific merit, while ancestor-aware Progress assesses substantive progress, necessary repair, repetition, or insufficient evidence.”
- Causal confounding: A distortion in an observed relationship caused by an unmeasured or uncontrolled variable. “The alternative explanations identify competing accounts, confounding, or measurement limitations”
- Censoring: Partial observation of an outcome because measurement ends before the event occurs or becomes observable. “A supported lack of added information would also inform whether more complex models merit further investigation.”
- Claim calibration: Restricting a scientific claim so that its scope matches the available evidence and uncertainty. “DiscoveryRSI-27B scores higher than GPT-6 Luna on importance (3.10 versus 2.94) and plausible, evidence-bounded claims (3.37 versus 3.16).”
- Crossover: An evolutionary-search operation that combines information from multiple candidate solutions. “evolution develops a hypothesis, repair addresses an identified defect, and crossover combines contributions from multiple parents.”
- Data resolvability: The extent to which available data can distinguish among competing hypotheses or answer a research question. “with the large gains in falsifiability and data resolvability.”
- Directed acyclic graph (DAG): A directed graph containing no cycles, so its edges impose an acyclic dependency structure. “The ancestry can therefore form a directed acyclic graph, although we use research tree as an intuitive description.”
- Domain shift: A change in the data domain or task distribution between training and evaluation. “To test transfer beyond life science, we add a battery setting with four domains”
- Evidence-bounded claim: A claim whose scope and certainty do not exceed what the supporting evidence warrants. “DiscoveryRSI-27B scores higher than GPT-6 Luna on importance (3.10 versus 2.94) and plausible, evidence-bounded claims (3.37 versus 3.16).”
- Evolutionary search: Search that iteratively improves candidate solutions through operations such as mutation, repair, selection, and recombination. “It develops research trees of testable hypotheses through multi-agent evolutionary search”
- Falsifiability: The extent to which a claim could be shown to be false by an appropriate observation or experiment. “with gains in falsifiability, alternative explanations, and data resolvability.”
- Fine-tuning: Additional training of an existing model on a task- or domain-specific dataset. “We perform full-parameter fine-tuning of DiscoveryRSI-27B with a 256K-token context window using hypothesis-credit-weighted supervision.”
- Full-parameter fine-tuning: Fine-tuning in which all model parameters, rather than only a subset, are updated. “We perform full-parameter fine-tuning of DiscoveryRSI-27B”
- Harness replay: Re-executing recorded agent interactions to diagnose failures and evaluate workflow modifications. “Harness replay examines these histories to identify productive strategies and recurring failures”
- Hidden Markov model: A probabilistic model in which an unobserved state sequence generates observable data. “L1 adds hidden Markov model summaries, subject to data-support and stability checks”
- Hypothesis credit optimization: A method for assigning scientific assessment credit to documented agent contributions and using it as training supervision. “Hypothesis credit optimization links assessments of research quality to documented contributions and assigns bounded credit to the contributing histories”
- Impedance: A measure of a system’s opposition to the flow of alternating electrical current, often used to characterize batteries. “The evidence pack covers capacity and impedance aging measurements”
- Interquartile range: The interval between the 25th and 75th percentiles of a distribution. “Error bars show 95\% confidence intervals and shading shows the interquartile range.”
- Logistic model: A statistical model used to estimate the probability of a binary outcome. “B0--B2 compare clinical, current-mismatch, and repeated-mismatch logistic models”
- Multi-agent evolutionary search: Search in which multiple agents collaboratively generate, critique, revise, and select candidate hypotheses. “DiscoveryRSI develops research trees of testable hypotheses through multi-agent evolutionary search”
- Multi-omics: Integrated analysis of multiple types of biological molecular data, such as genomics, transcriptomics, and proteomics. “population multi-omics”
- Negative result: A finding that fails to support a proposed effect or hypothesis but may still provide useful scientific information. “Scientific value can arise from a new explanation, a useful replication, a measurement check, or an informative negative result”
- Operational definition: A precise definition specifying how an abstract concept is represented or measured in practice. “We introduce a joint operational definition of valuable and testable research hypotheses”
- Post-cutoff research: Research published after the date up to which a model’s knowledge or reference data were available. “DiscoveryRSI-27B also achieves a 6.3\% rate of hypothesis correspondence with post-cutoff research”
- Provenance: Information recording the origin, history, and transformations of data or research artifacts. “Appendices~\ref{app:credit} and \ref{app:record-attribution} give the allocation rules and provenance requirements.”
- Recursive self-improvement: Iterative improvement in which an AI system uses experience from one cycle to modify the model or workflow used in later cycles. “To address these challenges, we introduce DiscoveryRSI, a general framework that turns hypothesis-driven research into a recursive learning process”
- Rediscovery: Applying accepted model and workflow updates to new investigations to generate additional experience. “Rediscovery applies accepted model and harness updates to fresh investigations.”
- Reference-actionability: The extent to which a generated hypothesis is connected to later research providing a concrete design or evidential contribution. “We define ``inspiration'' as a concrete connection between a generated hypothesis and subsequent literature.”
- Research trajectory: A recorded sequence of actions, decisions, and outputs produced during an investigation. “From 15,792 collected research trajectories, attribution, filtering, and tokenization yield 2,397 training trajectories”
- Scientific restart: Beginning a new research question without assigning it parent hypotheses in the existing research tree. “Scientific restart opens a new question without assigning parents”
- Self-supervised improvement: Improvement driven by experience or generated feedback rather than exclusively by manually labeled examples. “hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement”
- Supervision signal: Information used to guide model training toward desired outputs or behaviors. “We use this supervision to train DiscoveryRSI-27B.”
- Testability: The extent to which a hypothesis specifies a feasible investigation capable of producing evidence that bears on the claim. “It is testable when specifies an investigation feasible under ”
- TrueSkill: A Bayesian rating system that estimates the relative skill or preference strength of competing entities while representing uncertainty. “TrueSkill \citep{trueskill} summarizes relative preference strength and uncertainty”
- Uncertainty quantification: The representation or estimation of uncertainty associated with a prediction, measurement, or assessment. “The resulting describe comparative assessment, not the probability that a hypothesis is true.”
- Verifiability: The extent to which a computational result or task can be checked against explicit criteria. “Selected computational investigations are compiled into executable Harbor tasks with concrete inputs, environments, required outputs, and verification checks.”
- Zero-shot transfer: Applying a model or method to a new domain or task without additional task-specific training. “To test transfer beyond life science, we add a battery setting with four domains”