AiScientist: Autonomous Research System

• AiScientist System is an autonomous, modular computation framework that integrates multi-agent LLM-powered agents for end-to-end research automation.
• It decomposes research into specialized roles—hypothesis generation, experiment design, execution, evaluation, and iterative refinement—with rigorous context tracking.
• The system leverages human-in-the-loop feedback and advanced optimization techniques to ensure reproducibility, efficiency, and scalability in scientific inquiry.

An AiScientist System is an autonomous, modular computation framework designed to emulate and, in some domains, exceed the capabilities of human-driven scientific research. By orchestrating specialized LLM-powered agents into rigorously structured closed loops, these systems execute the complete research cycle—from automated literature synthesis and hypothesis formation to experiment design, evaluation, and iterative refinement—while integrating both automated and human expert feedback at every step. The AiScientist paradigm represents a convergence of advances in multi-agent AI, program synthesis, reinforcement learning, optimization theory, and workflow automation, enabling not only extraordinary speed and efficiency gains but also a new level of systematization and reproducibility in the scientific process (Team et al., 22 May 2025, Tie et al., 27 Oct 2025).

1. Formal System Architectures and Research Loop Dynamics

At the core of leading AiScientist systems such as InternAgent is a closed-loop multi-agent architecture that decomposes end-to-end research into a set of interoperable roles. A canonical instantiation (as in InternAgent) includes five primary modules: the Hypothesis Generator, Experimental Planner, Execution/Simulation Engine, Evaluator, and Iteration Controller. Each agent is mapped onto a concrete protocol layer—typically enforced via strict input/output schemas (e.g., structured JSON or custom Graph/Message protocols)—and coordinated by an orchestrator that maintains the global research context and drives cyclical progression (see formulas below) (Team et al., 22 May 2025).

The iterative pipeline at research iteration $t$ is defined as:

$$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$

where $G$ is the generative LLM agent, $T$ the transformation from hypothesis to methodology, and all context (including task descriptors $T$, baseline code $B$, literature $L$, and outcomes $O_t, S_t$) is centrally versioned.

This pipeline is instantiated as a modular pseudocode loop:

$G$2

The mathematical closure and rigorous context tracking in such systems ensure not only efficient search, but also auditable and reproducible experiment logs (Team et al., 22 May 2025, Tie et al., 27 Oct 2025).

2. Multi-Agent Specialization, Human-in-the-Loop, and Interactive Feedback

InternAgent and its successors enforce division of cognitive labor via agent specialization. For each phase of the research loop, outputs are produced and consumed by specialized agents, which communicate via a star topology around the orchestrator. Agent outputs are formatted for programmatic merging and critique (e.g., the IdeaInnovation Agent emits $\{idea\_id, text, confidence\}$ tuples; the Assessment Agent attaches automated metric vectors).

Human expert oversight is natively supported as a feedback vector $f(h)\in\mathbb{R}^d$—with axes such as novelty, feasibility, and domain fit. Feedback integrates with automated critique as

$$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$0

where $$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$1 controls the weighting between automated and human input. Practically, this is manifested as UI affordances for experts to select, annotate, pin, or prune hypotheses, with each intervention logged in the system’s provenance ledger for downstream learning and auditing (Team et al., 22 May 2025).

3. Core Algorithmic Engines and Optimization Objectives

AiScientist systems implement a diverse family of decision-making strategies and optimization engines across hypothesis selection, experiment execution, and policy improvement:

• Hypothesis Pool Management: Candidate ideas are ranked or selected via multi-armed bandit policies, often with an Upper Confidence Bound (UCB) objective:

$$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$2

where $$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$3 and $$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$4 are the empirical mean and uncertainty of assessment, and $$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$5 tunes exploration.

• Experiment Design: For hyperparameter tuning, Bayesian optimization is employed, with acquisition functions such as Expected Improvement or Probability of Improvement.
• Genetic Algorithms: Where applicable (e.g., model architecture search), genetic operators mutate and recombine code or configurations, with direct evaluation of “fitness” via automatic metric scripts.
• Reinforcement Learning: RL policies are trained to maximize expected reward over research iterations:

$$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$6

with reward signals given by successive performance deltas, e.g., $$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$7.

Debugging and code generation modules can incorporate joint loss functions

$$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$8

where exception minimization is balanced against test coverage maximization (Team et al., 22 May 2025).

4. Empirical Performance, Evaluation Metrics, and Cost Analysis

InternAgent and comparable systems are quantitatively benchmarked on domain-diverse tasks. Three representative results for InternAgent are:

Task	Metric	Baseline	InternAgent	Time
AutoRYP	$$\begin{align*} H_t &= G(H_{t-1}, C_{t-1}, L) \ M_t &= T(H_t, C_{t-1}, B, L) \ E_t &= \mathrm{Plan}(M_t) \ O_t &= \mathrm{Exec}(E_t) \ S_t &= \mathrm{Eval}(O_t) \ C_t &= \mathrm{Update}(C_{t-1}, H_t, O_t, S_t) \end{align*}$$9	27.6%	35.4%	12 h
AutoEAP	Pearson	0.65	0.79	4 h
Auto2DSeg	mIoU	78.8%	81.0%	30 h

Notably, InternAgent executed 60–90% of its self-generated ideas successfully, with improvement rates ($G$0 run metric) on 70–80% of those cases, substantially outpacing prior systems (which achieved 30–50% success rates). Financial cost per idea-generation was $G$10.7–1.2 for coder-debug cycles (Claude 3.7), reflecting an order-of-magnitude advantage over human-in-the-loop timelines (Team et al., 22 May 2025).

5. Robustness, Reproducibility, and Domain Transfer

To ensure scientific rigor and transferability, AiScientist systems incorporate:

• Strict version-control over input corpora, code, and configuration
• Comprehensive random seed management (affecting LLM generation, optimization, and simulation)
• Immutable “research ledger” tracking all hypotheses, methods, experiment runs, and evaluations

Domain adaptability is realized by parameterization: providing a new task description and baseline context automatically seeds SurveyAgent queries for literature and code, while core innovation and assessment modules remain unchanged. This modularity and systematic context updating support hardening against performance drift and support reuse across divergent scientific domains (Team et al., 22 May 2025).

6. Integration with Human Workflows and Automation Interfaces

AiScientist deployments incorporate advanced workflow integration:

• RESTful APIs for programmatic triggering, data uploads, and re-running closed-loop cycles
• UI visualizations representing hypothesis/experiment trees with facilities for real-time expert intervention (“pin/killed branches”)
• Dynamic reporting, with auto-generated methods/results sections compatible for manuscript integration

This deep integration aligns with broader trends in the field, including mixed-initiative research groups, agentic tree-search for parallel hypothesis exploration, and provenance-driven collaboration to ensure both transparency and efficiency (Team et al., 22 May 2025, Tie et al., 27 Oct 2025).

7. Future Trajectories and Open Problems

Current systems highlight several open research frontiers:

• Scalability to full self-driving labs and multi-user environments
• Formal verification and ethical governance, e.g., via risk-gated execution for high-stakes research
• Advanced collaboration protocols that unify audit-friendly agentic and human participation, as exemplified by open evaluation platforms and collaborative research protocols in emerging AiScientist ecosystems (Tie et al., 27 Oct 2025, Shao et al., 21 Nov 2025).
• Incorporation of stronger domain priors and theoretical constraints to further elevate the quality and interpretability of hypotheses and experimental designs

Ongoing work is focused on modularizing functionally distinct components, enhancing robustness via cryptographic provenance, and formalizing best-practice standards for authorship and AI involvement in scientific publications (Team et al., 22 May 2025, Tie et al., 27 Oct 2025).

---

The AiScientist System, thus, is characterized by a tightly orchestrated, multi-agent, research-centric architecture, rigorous optimization-driven workflows, explicit provenance and reproducibility mechanisms, and interfaces for high-integrity human–AI co-discovery. As a new paradigm for scientific inquiry, it enables closed-loop, scalable, and extensible research pipelines in both digital and physical domains (Team et al., 22 May 2025, Tie et al., 27 Oct 2025).