Learning the ARTS of Search for Automated Discovery

Abstract: Scientific discovery can be formulated as an iterative search process over the space of hypotheses and experiments. Contemporary methods navigate this space using heuristics such as MCTS. These algorithms conflate the merit of a hypothesis with the quality of its experimental execution. A promising hypothesis with preliminary execution is therefore ranked below a modest hypothesis whose execution is refined. Moreover, prior methods prune the search logs as the search progresses because the accumulated history outgrows the context window. We propose Agentic Reasoning for Tree Search (ARTS), where we deploy a reasoning LLM to navigate this space. The model inspects prior execution logs, diagnoses whether earlier failures arose from faulty implementations or bad hypotheses, and selects the hypothesis to build on next. To mitigate challenges with context length, ARTS uses test-time training to instill the knowledge of search tree in the model weights. Across 22 tasks from MLGym and MLEBench, we show that ARTS outperforms leading algorithms, with over 15.3% relative improvement in the normalized score. With test-time training we show that a Qwen3-4B agent can match performance with closed-source frontier models like Gemini-3 Pro and GPT o3-reasoning with upto 5x lower inference cost. We further observe that on partially observable RL tasks, the test-time trained Qwen3-4B scientist surpasses ARTS with the o3 scientist by rediscovering the human-best recurrent-memory solution that heuristic methods prune away.

• The paper presents ARTS, a reasoning-based tree search algorithm that distinguishes between hypothesis merit and execution quality.
• It employs a verbalized sampling mechanism to maintain hypothesis diversity and counteract the limitations of traditional score-based approaches.
• The approach integrates test-time training to distill search history into model weights, achieving improved performance across various ML benchmarks.

Agentic Reasoning for Tree Search (ARTS): Advancing Automated Scientific Discovery

Formulation of Automated Scientific Discovery as Search

"Learning the ARTS of Search for Automated Discovery" (2606.21891) approaches scientific discovery as an iterative search across hypotheses and experiments. Each node in the search space encapsulates a hypothesis, its implementation, execution traces, and the observed outcome. The canonical challenge in existing search algorithms—linear, tree-based (e.g., MCTS), or evolutionary—is the conflation between hypothesis merit and the quality of execution. Score-based heuristics can penalize promising hypotheses with poor preliminary executions, favoring well-implemented but modest hypotheses. Moreover, as search progresses, accumulated history rapidly outgrows context windows, leading prior approaches to prune or summarize past nodes, causing loss of critical information.

ARTS: Agentic Reasoning for Tree Search

ARTS proposes a fundamentally different search paradigm by leveraging a Large Reasoning Model (LRM) as an agentic scientist. This scientist inspects the search tree, reads prior logs and code, and diagnoses the failure mode (conceptual vs. executional). Parent selection becomes a reasoning task rather than a heuristic. The agentic scientist, equipped with tools for log and code inspection, proposes the next hypothesis based on explicit diagnostic reasoning.

Diversity collapse—a typical pathology in computational search—is mitigated via an adapted "verbalized sampling" mechanism. Here, the scientist generates a probabilistic distribution over multiple hypotheses. Rather than greedily selecting the most probable, sampling occurs from this distribution, preserving the exploration-exploitation balance and hypothesis diversity.

ARTS's modular design separates the scientist (reasoner) from the executor (coder), enabling independent advances in reasoning and code synthesis models. The executor, chosen for its strengths in code synthesis (e.g., Gemini 3. Flash), implements the scientist's proposal, validates it within a sandboxed workspace, and feeds the outcomes back to the search tree.

ARTS*: Addressing Context Length and Knowledge Retention via Test-Time Training

Long-running searches face context length limitations as the search tree grows exponentially. ARTS* introduces test-time training (TTT) for the scientist, employing task-specific fine-tuning of model weights (LoRA adapters) using the current search tree and rollout data. This process distills the search history into the scientist’s parameters, functioning as a memory mechanism and reducing dependence on explicit context length. Rollouts and rewards are structured to incentivize both effective parent selection and hypothesis generation, leveraging percentile-based adaptive reward signals for granular credit assignment.

Empirical Evaluation and Comparison to Existing Methods

ARTS was evaluated on 22 tasks drawn from the MLGym and MLEBench benchmarks, encompassing regression, classification, vision, RL, and game theory. Key baselines included linear search (e.g., AutoResearch), tree search via heuristic node selection (e.g., AIRA/MCTS), and evolutionary populations (e.g., MLEvolve).

ARTS demonstrated a mean normalized score improvement of 15.3% over the strongest baseline, achieving reliable gains (highest IQM) and narrowing the optimality gap to human-best results in challenging tasks with expensive training runs. Notably, ARTS achieved human-best or superhuman scores in medical image analysis and RL navigation tasks—a direct consequence of reasoning-based failure diagnosis and hypothesis diversity.

Test-time training raised the normalized score of Qwen3-4B from 0.72 to 1.01, and enabled the scientist to attain parity with closed-source frontier models at 5x lower inference cost. For partially observable RL tasks, ARTS* with a trained Qwen3-4B returned to and refined previously pruned recurrent-memory solutions, surpassing prior search algorithms and demonstrating practical retention and reuse of search experience.

Analysis of ARTS Performance Gains

Performance gains from ARTS were attributed to three core factors:

1. Failure Attribution: Reasoning-based diagnosis via log and code inspection allowed ARTS to distinguish between conceptual and executional errors. Promising hypotheses outperformed when maintained beyond preliminary failures, counteracting premature pruning observed in score-only methods.
2. Hypothesis Diversity: Verbalized sampling enforced broad exploration across multiple axes, with ARTS achieving higher coverage and entropy in the axis distribution compared to baselines. 45% of expansions in ARTS opened new hypothesis axes—orders of magnitude above baselines, which concentrated on incremental tweaks.
3. Quality of Proposals: The reasoning scientist generated high-quality proposals, addressing explicit failure modes from previous experiments. This resulted in more productive utilization of runtime budget, as each hypothesis was contextually targeted.

Ablations indicated significant drops in performance when audit or verbalized sampling components were removed, corroborating their necessity. Token usage and compute analysis confirmed ARTS’s efficiency despite enhanced search breadth and reasoning capability.

Theoretical and Practical Implications

ARTS presents a principled framework for automated discovery, shifting the search paradigm towards explicit reasoning, diagnosis, and targeted exploration. The modularity of scientist and executor components allows for independent improvement and compositional scalability as better models emerge. Test-time training can serve as a generalized adaptation mechanism, distilling long-range search experience into compact in-weight memory.

Practically, ARTS enables efficient automated research agents capable of tackling ML benchmarks, scientific tasks, and RL environments with limited compute resources. The methodology serves as a blueprint for integrating reasoning agents in complex, iterative discovery settings, where the distinction between idea and implementation is paramount.

Theoretically, ARTS's approach raises new questions about search in high-dimensional, heterogeneous domains. The separation of diagnostic reasoning from score-based heuristics aligns with cognitive theories of scientific method, and may inform future architectures in automated hypothesis generation and experiment design.

Outlook and Future Directions

ARTS is extensible to domains beyond ML, including algorithmic and scientific discovery in chemistry, combinatorics, and other experimental fields. Integrating more advanced scientists with memory architectures or leveraging unsupervised or meta-learning for knowledge generalization represents promising research avenues. The interaction of agentic reasoning, modular execution, and adaptive search policies suggests a path forward for scalable, autonomous research systems.

Provisions for ethical deployment—including transparency, logging, and oversight—are necessary. As research agents become more capable and computation-concentrated, the risk of asymmetric access and malicious use increases. Task constraints and human-in-the-loop mechanisms should accompany deployment in sensitive domains.

Conclusion

ARTS introduces a robust, reasoning-centric search paradigm for automated discovery, outperforming contemporary score-driven methods through diagnostic attribution and hypothesis diversity. Test-time training enables compact knowledge retention, making search efficient and scalable. The approach offers both practical utility in ML experimentation and theoretical insight into the future design of agentic research systems.