ScientistTwo: An AI System That Does Research Autonomously
ScientistTwo is a multi-agent AI system that autonomously conducts empirical research from problem to published paper. Given a scientific problem and baseline implementation, it identifies limitations, generates hypotheses, implements and tests them across multiple datasets, performs ablation studies, drafts manuscripts, and responds to simulated peer review—all without human intervention. Evaluated on 107 research problems from top ML conferences, ScientistTwo achieved an 80.4% success rate with an average 25.2% improvement over human-designed baselines, generating papers that matched or exceeded acceptance scores of human-authored conference papers.Script
A research team at a top AI lab recently published work that isn't just impressive for its results. What makes it startling is that the research team consisted entirely of AI agents.
ScientistTwo operates as a staged pipeline where specialized agents tackle each phase of research. The system starts by extracting limitations from existing work, generates and ranks hypotheses, implements the most promising ideas on representative subsets before full evaluation, and then subjects successful candidates to ablation studies that verify each component actually contributes to the improvement.
What separates this from automated hyperparameter search is the review loop. After drafting a paper, simulated peer reviewers assign scores and identify weaknesses. If the score falls below threshold, the system doesn't just revise the writing—it designs new experiments, modifies the method itself, reruns ablations, and redrafts. Papers that underwent two review iterations saw acceptance rates climb from 47% to 94% on one automated reviewer.
Across 107 research problems drawn from accepted conference papers, ScientistTwo reported successful advances on 86 tasks, with an average relative improvement of 25.2% over human baselines. Under automated review, its generated papers matched or exceeded the average acceptance scores of human authors at ICLR and NeurIPS, though they did not reach the level of spotlight presentations.
In one concrete case study, the system tackled time-series forecasting and invented a method called DynaSpec-RAG. It decomposes input signals into frequency bands, retrieves relevant patterns separately for each band, applies learned gates that suppress unhelpful retrievals, and includes a safety switch that disables retrieval entirely when it degrades validation performance. The method achieved best performance on six of seven datasets while adding only 0.27 million trainable parameters.
ScientistTwo demonstrates that autonomous research can extend beyond configuration tuning to hypothesis generation, ablation, and iterative refinement within well-defined empirical settings. Whether similar systems can handle problems requiring theoretical invention or new experimental protocols remains an open frontier—one you can explore further at EmergentMind.com, where you can create your own videos on the latest research.
