EvoScientist: Evolving AI Discovery

• EvoScientist is a memory-centric multi-agent framework that defines scientific discovery as an evolving process over trajectories linking goals, ideas, proposals, experiments, and results.
• It employs specialized agents for idea generation, experiment execution, and memory evolution, ensuring that both successful strategies and failed attempts inform future research.
• Demonstrated through improved innovation metrics and successful ML paper generation, EvoScientist enhances execution success rates and reusability in code-centric AI research.

EvoScientist is an evolving multi-agent AI scientist framework for end-to-end scientific discovery in code-centric domains, especially machine learning and AI experimentation. It was introduced to address a recurrent limitation of contemporary AI scientist systems: most rely on static, hand-designed pipelines and do not adapt from accumulated interaction histories, so they overlook promising directions, repeat failed experiments, and pursue infeasible ideas. Its central design move is to treat scientific discovery as a learning problem over trajectories of the form goal $\rightarrow$ idea $\rightarrow$ proposal $\rightarrow$ experiments $\rightarrow$ results, and to improve future trajectories through persistent memory and self-evolution (Lyu et al., 9 Mar 2026).

1. Conceptual scope and problem setting

EvoScientist comprises three specialized agents: a Researcher Agent (RA) for scientific idea generation, an Engineer Agent (EA) for experiment implementation and execution, and an Evolution Manager Agent (EMA) that distills insights from prior interactions into reusable knowledge. It also contains two persistent memory modules: an ideation memory, which summarizes feasible research directions from top-ranked ideas while recording previously unsuccessful directions, and an experimentation memory, which captures effective data processing and model training strategies derived from code search trajectories and best-performing implementations (Lyu et al., 9 Mar 2026).

The framework is motivated by a specific failure mode of earlier AI scientist systems. In static pipelines, interaction history is largely transient: failed ideas do not become explicit negative knowledge, and successful implementation patterns are not turned into reusable strategy. EvoScientist therefore treats prior trajectories as first-class scientific artifacts. The RA and EA do not merely consult literature or the current task description; they retrieve distilled knowledge from earlier runs and condition their next decisions on that retrieval (Lyu et al., 9 Mar 2026).

A plausible broader reading is that “EvoScientist” also names a family resemblance across newer AI-discovery systems: systems that evolve something more abstract than a single hypothesis. In adjacent work, the evolving object may be scientific principles rather than hypotheses, as in PiEvo’s principle space (Pu et al., 6 Feb 2026), research ideas as a population in EvoGens (Li et al., 29 May 2026), problem clusters in EvoSci (Xiong et al., 20 May 2026), or even the search mechanism itself in Bilevel Autoresearch, which explicitly cites “EvoScientist’s persistent memory” as a prior structural improvement (Qu et al., 24 Mar 2026). Within that broader landscape, the distinctive feature of EvoScientist proper is its coupling of multi-agent role specialization with persistent cross-task memory (Lyu et al., 9 Mar 2026).

2. Multi-agent architecture and end-to-end workflow

Given a user goal $G$, EvoScientist runs a two-stage pipeline. In Stage 1, the RA retrieves relevant knowledge from ideation memory,

$K_I = \text{Retrieve}_I(M_I, G),$

and performs an idea tree search conditioned on the goal, retrieved literature, and the retrieved memory. Candidate ideas are generated and critiqued,

$$\{ (I_1, \mathrm{rev}_1), \dots, (I_{N_I}, \mathrm{rev}_{N_I}) \} = \text{IdeaTreeSearch}(G, L, K_I),$$

then ranked by an Elo-style tournament,

$$\{r_1, \dots, r_{N_I}\} = \text{EloRank}(I_{1:N_I}),$$

after which the top-$3$ ideas are retained as direction evidence and the top-$1$ idea is expanded into a proposal $\rightarrow$0 (Lyu et al., 9 Mar 2026).

In Stage 2, the EA retrieves relevant execution knowledge from experimentation memory,

$\rightarrow$1

and runs a four-stage experiment tree search: initial implementation, hyperparameter tuning, proposed method implementation, and ablations. For each stage $\rightarrow$2,

$\rightarrow$3

where $\rightarrow$4 is a code version and $\rightarrow$5 is its structured execution record. The best code per stage is selected,

$\rightarrow$6

and the resulting histories are summarized into an execution report $\rightarrow$7 (Lyu et al., 9 Mar 2026).

The architecture is therefore not a flat prompt chain but a role-differentiated pipeline in which ideation and experimentation are explicitly separated and both are history-aware. The RA operates on research directions and proposal structure; the EA operates on executable artifacts and debugging trajectories; the EMA turns the entire trajectory into memory updates. This division mirrors a laboratory workflow in which conception, implementation, and post hoc methodological reflection are related but not identical activities (Lyu et al., 9 Mar 2026).

Component	Primary function	Retrieval or update object
Researcher Agent (RA)	Idea generation and proposal writing	$\rightarrow$8
Engineer Agent (EA)	Experiment implementation and execution	$\rightarrow$9
Evolution Manager Agent (EMA)	Distill reusable knowledge from history	$\rightarrow$0
Ideation memory $\rightarrow$1	Store feasible and unsuccessful directions	Updated by IDE and IVE
Experimentation memory $\rightarrow$2	Store execution and training strategies	Updated by ESE

3. Persistent memory and self-evolution

The EMA implements three evolution mechanisms. The first is Idea Direction Evolution (IDE), which abstracts high-level directions from the top-ranked ideas:

$\rightarrow$3

followed by

$\rightarrow$4

The second is Idea Validation Evolution (IVE), which records directions that turned out to be unsuccessful when judged against proposal execution and outcomes:

$\rightarrow$5

with the corresponding ideation-memory update

$\rightarrow$6

The third is Experiment Strategy Evolution (ESE), which summarizes full code-search trajectories into reusable execution strategies:

$\rightarrow$7

and updates experimentation memory through

$\rightarrow$8

These definitions make the framework’s “evolution” literal: top-ranked directions are inherited, failed directions become negative knowledge, and robust implementation patterns are retained for later reuse (Lyu et al., 9 Mar 2026).

The two memories store different epistemic objects. Ideation memory contains feasible direction summaries extracted from top ideas and unsuccessful directions derived from proposal validation. Experimentation memory contains strategy summaries extracted from complete code-search trajectories, including data preprocessing, model training, implementation patterns, and debugging fixes. Both memories are embedded using mxbai-embed-large via Ollama; retrieval uses cosine similarity with top-$\rightarrow$9 for ideation memory and top-$\rightarrow$0 for experimentation memory (Lyu et al., 9 Mar 2026).

This design changes the status of failure. In a non-evolving pipeline, a failed proposal or a broken training loop is usually just an exhausted branch. In EvoScientist, it becomes a memory item that can suppress future repetition. The same applies positively to successful trajectories: a useful experimental pattern is no longer tied to the task that first produced it, but becomes transferable knowledge for later tasks (Lyu et al., 9 Mar 2026).

4. Idea generation, experiment execution, and empirical performance

EvoScientist was evaluated on 30 research queries from real AI researchers spanning areas such as machine translation, speech, software engineering, healthcare agents, retrieval-augmented generation, text-to-SQL, multimodal models, efficiency, safety, and alignment. For idea generation, it was compared against 7 open-source and commercial systems: Virtual Scientist, AI-Researcher, InternAgent, AI Scientist-v2, Hypogenic, Novix, and K-Dense (Lyu et al., 9 Mar 2026).

Under automatic evaluation with Gemini-3-flash, EvoScientist showed strong pairwise win rates. Against Virtual Scientist, it achieved Novelty win 96.67%, Feasibility win 93.33%, Relevance win 90.00%, and Clarity win 96.67%, with Avg. gap $\rightarrow$1. Against AI Scientist-v2, it achieved Novelty win 63.33%, Feasibility win 53.33%, Relevance win 36.67%, and Clarity win 56.67%, with Avg. gap $\rightarrow$2. Against Hypogenic, it achieved Novelty win 93.33%, Feasibility win 83.34%, Relevance win 70.00%, and Clarity win 96.67%, with Avg. gap $\rightarrow$3 (Lyu et al., 9 Mar 2026).

Human evaluation by 3 PhD-level annotators on 120 idea pairs showed a similar pattern. Against InternAgent, EvoScientist achieved Novelty win 66.67%, Feasibility win 96.67%, Relevance win 90.00%, and Clarity win 93.33%, with Avg. gap $\rightarrow$4. Against AI Scientist-v2, the Avg. gap was $\rightarrow$5; against Novix, $\rightarrow$6; against K-Dense, $\rightarrow$7. Agreement between LLM and human judgments was reported as approximately 87.3% across dimensions (Lyu et al., 9 Mar 2026).

On the experimentation side, Figure 1 reports that the overall average execution success rate increased from 34.39% before Experiment Strategy Evolution to 44.56% after it. On the hardest stage, proposed method implementation, the success rate increased from 20.33% to 21.57% (Lyu et al., 9 Mar 2026). The ablations also isolate the role of memory evolution. Removing Idea Direction Evolution harmed novelty and feasibility; removing Idea Validation Evolution most strongly harmed feasibility; removing both caused clear degradation, including Novelty lose 80.00% and Feasibility lose 83.33% in pairwise comparisons (Lyu et al., 9 Mar 2026).

The most visible end-to-end demonstration was the generation of six full ML papers submitted to the ICAIS 2025 AI Scientist Track. The track received 82 submissions and accepted 26, for a 31.71% acceptance rate; all six EvoScientist papers were accepted. Two received major awards: “Adaptive Evidential Meta-Learning with Hyper-Conditioned Priors for Calibrated ECG Personalisation” won the Best Paper Award, and “Hierarchical Change Signature Analysis: A Framework for Online Discrimination of Incipient Faults and Benign Drifts in Industrial Time Series” won the AI Reviewer’s Appraisal Award (Lyu et al., 9 Mar 2026).

5. Position within the evolving AI-scientist literature

EvoScientist belongs to a rapidly diversifying literature on AI systems that cast scientific discovery as an evolutionary process, but different systems choose different evolving substrates. PiEvo treats scientific discovery as Bayesian optimization over an expanding principle space, using Information-Directed Hypothesis Selection via Gaussian Process and anomaly-driven augmentation; across four benchmarks it reported average solution quality of up to 90.81%–93.15%, a 29.7%–31.1% improvement over the state of the art, and an 83.3% speedup in convergence step (Pu et al., 6 Feb 2026). EvoGens instead recasts scientific idea generation as an evolutionary search over a population of ideas, with rank-based mutation and semantic-aware crossover; under its automatic protocol it improved Novelty from 0.1 to 0.4 and Diversity from 0.24 to 0.55 while maintaining comparable Relative Quality at 0.21 (Li et al., 29 May 2026).

Other systems broaden the same theme in different directions. ResearchEVO instantiates a discover-then-explain paradigm: an Evolution Phase performs LLM-guided bi-dimensional co-evolution over code, and a Writing Phase produces a compilable LaTeX paper through sentence-level retrieval-augmented generation with explicit anti-hallucination verification; its case studies reported zero fabricated citations in the generated manuscripts (Zhao et al., 7 Apr 2026). EvoSci emphasizes bio-inspired multi-agent collaboration and knowledge-graph evolution, reporting the highest overall peer-review score of ICLR 4.90 and Top-10 = 54 in tournament-style ranking (Xiong et al., 20 May 2026). Bilevel Autoresearch moves the evolutionary target one level upward: its outer loop meta-optimizes the inner autoresearch loop by generating and injecting new search mechanisms as Python code at runtime, and it explicitly places “EvoScientist’s persistent memory” alongside Karpathy’s single-track loop and AutoResearchClaw’s multi-batch search as prior stages in the autoresearch lineage (Qu et al., 24 Mar 2026).

Against that backdrop, EvoScientist is most naturally characterized as a memory-centric evolving AI scientist. It does not primarily evolve principles, code search mechanisms, or a population of idea texts. It evolves the reusable summaries that shape future ideation and execution. That makes it especially relevant for settings in which the main bottleneck is not one-shot generation quality, but repeated exposure to similar research tasks where accumulated experience can be leveraged (Lyu et al., 9 Mar 2026).

6. Limitations, interpretation, and open problems

EvoScientist’s strongest results are in computational domains where experiments are code-executable. The paper explicitly notes that the current scope is computational and ML-style research; physical sciences with laboratory experiments would require integration with lab hardware, experimental design tooling, and safety constraints (Lyu et al., 9 Mar 2026). The framework also inherits the limitations of its underlying LLMs: idea novelty, literature understanding, code correctness, and memory summarization quality are all bounded by model quality, and biases or hallucinations can in principle propagate into persistent memory (Lyu et al., 9 Mar 2026).

Evaluation remains a live issue. Although the reported agreement between LLM and human judgments is high, idea quality assessment still partly relies on LLM-as-judge protocols, and the end-to-end paper-generation evidence is based on six accepted papers rather than a large-scale controlled benchmark (Lyu et al., 9 Mar 2026). The qualitative analysis of those papers is also revealing: the system excelled at empirically testable ideas and comprehensive experiments, but some submissions lacked deeper theoretical formalization and sometimes had protocol or reporting weaknesses. This suggests that EvoScientist is especially strong as an empirical discovery assistant rather than as a fully autonomous theorist (Lyu et al., 9 Mar 2026).

A common misconception is to treat such systems as autonomous scientific authorities. The stated interpretation is narrower: EvoScientist should be used as decision support, not as an authority, and human experts remain necessary for verification, especially in high-risk domains (Lyu et al., 9 Mar 2026). Within the broader “EvoScientist” trend, the central open problem is therefore not just better idea generation or higher execution success, but better integration of memory, evaluation, theory formation, and domain-specific empirical constraints. The existing literature already points toward several nonexclusive directions: evolving principles rather than hypotheses (Pu et al., 6 Feb 2026), evolving search mechanisms themselves (Qu et al., 24 Mar 2026), strengthening collaborative multi-agent problem-space evolution (Xiong et al., 20 May 2026), and coupling blind algorithmic discovery with literature-grounded explanation (Zhao et al., 7 Apr 2026).