AgentBuilder Systems

• AgentBuilder systems are frameworks that automate agent construction as a core computational workflow, emphasizing role decomposition and dynamic design.
• They utilize sequential and hierarchical processes to generate, test, and optimize agent pipelines, ensuring artifact-centered orchestration.
• These systems bridge manual engineering with autonomous design by producing deployable agents, toolchains, and validated service artifacts.

Searching arXiv for papers on AgentBuilder-style systems. AgentBuilder denotes a class of systems in which agent construction itself becomes an explicit computational workflow rather than a purely manual engineering activity. Across recent arXiv work, the term covers several related forms: multi-agent systems that design, generate, test, optimize, and deploy other multi-agent systems; contract-driven frameworks that compile a scientific agent from an authored rubric and curriculum; workflow compilers that transform underspecified user requests into validated AI model pipelines; and agent development kits that let models create their own topology, tools, and memory structures at runtime (Harper, 2024, Shin et al., 11 Jun 2026, Kim et al., 2024, Li et al., 18 Feb 2026). The unifying theme is that the artifact being produced is not merely an answer to a task, but an agent, toolchain, pipeline, repository, or deployable service whose structure, interfaces, and operational behavior are themselves the primary object of design.

1. Conceptual scope and definitional boundaries

In the recent literature, AgentBuilder is not a single standardized architecture but a family of systems that elevate agent creation to a first-class stage of computation. In "AutoGenesisAgent" (Harper, 2024), the central idea is recursive: use a multi-agent system to create another multi-agent system. In "AgentBuild" (Shin et al., 11 Jun 2026), agent construction is defined as a scientific workflow stage with a declared interface, an acceptance test, a build step, typed outputs, and a promotion gate. In "Bel Esprit" (Kim et al., 2024), the target artifact is not a conversational agent per se but an executable AI model pipeline graph synthesized from a clarified user request. In "OpenSage" (Li et al., 18 Feb 2026), the focus shifts to an ADK in which LLMs automatically create agents with self-generated topology and toolsets while using structured memory support.

This variation reveals a useful conceptual distinction. Some systems build agents as runtime organizations. Others build agents as deployable workflow components, tool-using scientific artifacts, or repositories intended for downstream reuse. This suggests that AgentBuilder is best understood as an umbrella term for systems that externalize and automate design decisions normally made by human developers: decomposition into roles, tool selection or synthesis, memory design, communication structure, validation strategy, and deployment packaging (Harper, 2024, Shin et al., 11 Jun 2026, Li et al., 18 Feb 2026).

A second boundary concerns what AgentBuilder is not. Several papers explicitly distinguish it from prompt-only agent authoring, from narrow AutoML search over fixed spaces, and from ordinary multi-agent runtimes that assume the topology, tools, and memory are still hand-designed (Shin et al., 11 Jun 2026, Zhang et al., 15 Apr 2026, Li et al., 18 Feb 2026). The common claim is not merely that agents can use tools, but that they can participate in constructing the systems, tools, or workflows they themselves depend on.

2. Recurrent architectural patterns

A striking regularity across the literature is decomposition into specialist roles. "AutoGenesisAgent" (Harper, 2024) enumerates ten core modules, including System Understanding Agent, System Design Agent, Agent Generator, Integration and Testing Agent, Optimization and Tuning Agent, Deployment Agent, Documentation and Training Agent, Feedback and Iteration Agent, LLM Prompt Design Agent, and Hierarchy Agent. "M$^3$Builder" (Feng et al., 27 Feb 2025) similarly adopts four role-specialized agents—Task Manager, Data Engineer, Module Architect, and Model Trainer—within a medical imaging ML workspace. "AIBuildAI" (Zhang et al., 15 Apr 2026) uses a manager agent coordinating designer, coder, and tuner sub-agents over multiple solution repositories. "CollabToolBuilder" (Xavier et al., 1 Dec 2025) extends the planner-builder-verifier pattern into Coach, Coder, Critic, and Capitalizer.

These systems differ in domain and implementation detail, but they converge on a shared systems principle: agent building is treated as a heterogeneous workload whose subtasks have different observables and different correctness criteria. Requirement elicitation, architecture synthesis, code generation, validation, tuning, documentation, and deployment are separated because each stage benefits from distinct prompts, state, and tools. This suggests that specialization is not an incidental interface choice but a structural response to long-horizon design complexity (Harper, 2024, Feng et al., 27 Feb 2025, Zhang et al., 15 Apr 2026).

Another recurrent pattern is hierarchy plus review. "AutoGenesisAgent" (Harper, 2024) introduces a Hierarchy Agent to ensure every work step has both an executing agent and an approving or rejecting agent. "AgentBuild" (Shin et al., 11 Jun 2026) implements a rubric-driven judge gating a meta-optimizer coding agent within an edit boundary. "Bel Esprit" (Kim et al., 2024) uses Builder and Inspector, with the Inspector performing both syntax inspection and semantic inspection before Matchmaker populates the graph with concrete models. "BUILD-AND-FIND" (Lin, 7 May 2026), though evaluative rather than generative, sharpens the same concern by treating generated repositories as communication artifacts whose design intent must be recoverable by downstream agents.

A further pattern is artifact-centered orchestration. In several systems, inter-agent communication is mediated less by persistent chat than by explicit artifacts: structured specifications, graph schemas, repositories, JSON part files, dataloaders, reports, or versioned packaged agents. "Bel Esprit" (Kim et al., 2024) formalizes pipeline generation as graph construction. "AIBuildAI" (Zhang et al., 15 Apr 2026) uses multiple repository-scoped workspaces, each containing a plan, code, config, and results. "El Agente Forjador" (Zhang et al., 16 Apr 2026) centers execution around a shared filesystem workspace containing question.md, tools/, tool-generation directories, scripts, logs, and report.md. The broad implication is that AgentBuilder systems often rely on explicit intermediate artifacts to stabilize long-horizon reasoning and make downstream validation tractable.

3. Lifecycle organization: from specification to deployable artifact

Many AgentBuilder systems can be read as lifecycle managers. "AutoGenesisAgent" (Harper, 2024) offers the clearest end-to-end formulation: Prompt -> Understanding -> Design -> Prompt/role specification -> Agent generation -> Hierarchical assignment -> Integration/testing -> Optimization -> Deployment -> Documentation -> Feedback/iteration. Its System Understanding Agent converts a user prompt into a structured specification; the System Design Agent decides the optimal number and type of agents, their roles, data flows, and interaction protocols; the Agent Generator produces code or configuration modules; the Integration and Testing Agent assembles and validates them; the Optimization and Tuning Agent improves them against predefined metrics; and the Deployment Agent operationalizes the result.

Other papers instantiate the same lifecycle pattern in narrower domains. "Science Consultant Agent" (K et al., 18 Dec 2025) uses a sequential pipeline from Questionnaire to Smart Fill to Research-Guided Recommendation to Prototype Builder, yielding a structured task profile, literature-grounded best solution and strong baseline, and executable prototype artifacts where supported. "Bel Esprit" (Kim et al., 2024) proceeds through clarification, specification extraction, abstract graph synthesis, syntax and semantic validation, repair loops, and model population. "AgentBuild" (Shin et al., 11 Jun 2026) begins from a scientist-authored contract—rubric, difficulty-graded curriculum, and producer external knowledge base—then iteratively mutates an Agent Under Development until the current curriculum passes under $P$-strict, after which it packages a versioned A2A agent with provenance.

The workflow stage perspective is particularly explicit in "AgentBuild" (Shin et al., 11 Jun 2026). Its six declared inputs are Curriculum, Rubric, Producer external knowledge base, Base model handle, MCP tool inventory, and Edit boundary. Its promotion gate is not a scalar score but a conjunctive acceptance rule: $$P\text{-strict} \iff \forall c,d,\; s(c,d)\ge \tau_d.$$ This means every active curriculum case must satisfy every rubric threshold simultaneously. The framework therefore distinguishes pattern-fitting success from contract-compliant trajectory success, especially in higher-stakes settings such as Rietveld refinement, where a plausible fit can still fail because the procedural trajectory violates the scientist-authored workflow envelope (Shin et al., 11 Jun 2026).

This lifecycle perspective suggests that AgentBuilder systems are often less about inventing a single agent algorithm than about turning agent construction into a staged, inspectable, iteratively repairable process.

4. Tools, memory, and execution substrates

AgentBuilder systems repeatedly treat tools and memory as design variables rather than fixed infrastructure. "OpenSage" (Li et al., 18 Feb 2026) makes this explicit by allowing models to automatically create agents with self-generated topology and toolsets while using hierarchical, graph-based memory. Its short-term memory is a graph of AgentRun, Event, and RawToolResponse nodes; its long-term memory is a typed Neo4j graph in which nodes store type, label, content, and embedding; and a memory agent mediates natural-language access to both stores. The system also supports agent creation as a runtime tool, a unified sub-agent pool, a filesystem-based hierarchical tool registry, and per-toolset Docker isolation with cached container states.

Related papers adopt more domain-specific but convergent strategies. "AutoGenesisAgent" (Harper, 2024) uses PostgreSQL to store detailed logs of agent interactions, system states, and feedback loops, so persistent memory is primarily operational logging and state storage rather than an explicit cognitive architecture. Communication is implemented through a custom message-passing framework. "AgentA/B" (Wang et al., 13 Apr 2025) externalizes persona, intention, action history, and webpage state in the orchestration layer rather than entrusting memory to a monolithic prompt; it parses live pages into structured JSON observations and constrains the action space to Search, Click Product, Click Filter Option, Purchase, and Stop. "AgentScope 1.0" (Gao et al., 22 Aug 2025) systematizes this design into four abstractions—message, model, memory, and tool—and adds asynchronous model calls, streaming, parallel tool execution, MCP integration, runtime sandboxing, and a deployment layer with A2A support.

Several systems go further and make tool construction itself part of the build loop. "El Agente Forjador" (Zhang et al., 16 Apr 2026) is particularly explicit: tool generation is a dedicated stage in which a Tool Analyzer identifies missing capabilities, a Tool Generator writes reusable Python tools plus tests, and a Tool Reviewer validates them before approved tools are copied into a persistent tools/ library. The generated tools must be reusable, avoid silent failure, and use Pydantic validation for type safety. "CollabToolBuilder" (Xavier et al., 1 Dec 2025) likewise centers the iterative creation of human-validated, reusable hybrid LLM+Python tools, with a Capitalizer that stores both validated tools and failed tools after retries. "OpenSage" (Li et al., 18 Feb 2026) also supports AI-written tools with metadata and runtime registration.

This body of work suggests that memory and tooling are no longer peripheral concerns. In AgentBuilder systems they are often first-class outputs of the design process, tightly coupled to topology generation and validation.

5. Evaluation regimes and application domains

AgentBuilder research spans a wide range of domains, and evaluation regimes vary accordingly. "AutoGenesisAgent" (Harper, 2024) presents four tested use cases—an educational content management system, a software development automation pipeline, an SME project management system, and a healthcare patient management system. Outcomes are case-study based rather than benchmarked: the educational content management system successfully automated the creation and curation of educational modules, the software development pipeline underperformed compared to a customized system but produced a pong-style terminal game, the SME system generated project plans and internal estimates, and the healthcare patient management attempt failed to make a functional system.

By contrast, several later papers provide benchmark-centric evidence. "M$^3$Builder" (Feng et al., 27 Feb 2025) introduces M$^3$Bench, spanning four task families on 14 medical imaging datasets. With Claude-3.7-Sonnet as the agent core, it reports a 94.29% success rate, where success means the model-building task completes and the trained model’s test performance falls within an acceptable range. "AIBuildAI" (Zhang et al., 15 Apr 2026) evaluates on MLE-Bench and reports a medal rate of 63.1%, ranking first on the March 18, 2026 MLE-Bench leaderboard, with 77.27% on low-complexity tasks, 61.40% on medium-complexity tasks, and 46.67% on high-complexity tasks. "El Agente Forjador" (Zhang et al., 16 Apr 2026) reports combined score improvements from direct solving to zero-shot tool generation to curriculum-built tool reuse, with chemistry averages of 76.7%, 81.5%, and 85.8%, and dynamics averages of 91.3%, 91.5%, and 93.2%, while tool reuse also reduced API cost by 33.2% to 64.2% in chemistry and 62.7% to 77.9% in dynamics.

Other papers show how AgentBuilder ideas migrate into specialized scientific, web, or infrastructure settings. "AgentBuild" (Shin et al., 11 Jun 2026) instantiates a contract-driven builder for Rietveld refinement through GSAS-II behind MCP and A2A, reaches the 4 hour LLZO scan as a frontier case, and reports that the iteration-9 AUD achieved $R_{wp}=5.78\%$, $R_p=4.54\%$, and $RF=9.86\%$, yet still failed strict acceptance due to a trajectory-scope issue rather than a pattern-fitting failure. "AgentA/B" (Wang et al., 13 Apr 2025) uses 1,000 autonomous LLM agents on Amazon.com in a between-subject A/B test, with 500 control and 500 treatment agents after sampling from 100,000 personas. "OpenSage" (Li et al., 18 Feb 2026) evaluates on CyberGym, Terminal-Bench 2.0, and SWE-Bench Pro, reporting 60.2%, $65.2 \pm 2.0$, and 59.0% respectively for SageAgent/OpenSage configurations.

The domain spread is itself informative. AgentBuilder methods have been applied to scientific agent construction, AI pipeline synthesis, medical AutoML, software repository creation and evaluation, web-behavior simulation, quantum tool forging, and dynamic CAD assembly generation. This suggests that AgentBuilder is less a domain-specific paradigm than a reusable systems pattern for automating the creation of agentic artifacts.

6. Limitations, controversies, and open problems

Despite rapid expansion, the literature is notably cautious about current limitations. "AutoGenesisAgent" (Harper, 2024) is explicit that the work is architectural and conceptual rather than mathematically formal, with no equations, benchmark scoring formulas, formal schemas, prompt examples, or reproducible deployment artifacts. It reports no standardized datasets, quantitative metrics tables, baseline comparisons with exact scores, ablations, cost measurements, or safety validation framework. "Science Consultant Agent" (K et al., 18 Dec 2025) likewise evaluates recommendation quality primarily through user feedback rather than objective superiority, and supports only a limited set of prototype tools. "AgentScope 1.0" (Gao et al., 22 Aug 2025) presents a comprehensive engineering framework but not benchmark comparisons, quantified latency gains, or formal safety guarantees for its runtime sandbox.

A second persistent concern is brittleness under real execution. "AutoGenesisAgent" (Harper, 2024) identifies conversational loops as a major failure mode and mitigates them with timeout mechanisms, loop detection, conversation resets, and even the heuristic of injecting an additional sentence at the end of the prompt prior to the loop. "AgentA/B" (Wang et al., 13 Apr 2025) reports fallback strategies such as retrying, scrolling into view, and re-parsing dynamic webpages. "AADvark" (Adler et al., 16 Apr 2026) shows that current LLMs are imperfect spatial reasoners and argues that success required modifications to FreeCAD and OndselSolver, including unique colors and textures for part faces and edges, text identifiers for part instances, quaternion-based orientation handling, improved determinism, and more informative error messages.

A third issue is the gap between plausible outputs and acceptable process. "AgentBuild" (Shin et al., 11 Jun 2026) makes this distinction especially clear through trajectory dimensions $T1$–$P$0 and the notion of trajectory scope. A plausible numerical fit is not enough if it violates the scientist-authored workflow protocol. "BUILD-AND-FIND" (Lin, 7 May 2026) sharpens a related distinction for code artifacts by separating behavioral correctness, artifact-side intent recoverability, and downstream modification success. Its target construct is artifact-evidenced recovery: whether a later agent can recover intended design choices from the repository artifact with low inspection effort. This suggests that future AgentBuilder evaluation may need to measure not only task success, but also the communicative quality of the produced artifact.

Finally, many papers indicate that the underlying optimization story remains underdeveloped. Several systems are strong on architecture and workflow, but weak on formal search objectives, learned decomposition policies, or theoretically grounded acceptance criteria beyond staged heuristics or rubric conjunctions. This suggests that the current state of AgentBuilder research is dominated by systems engineering rather than unified learning theory. A plausible implication is that the next stage of the field will require tighter integration of architecture generation, verifier design, artifact evaluation, and post-training or policy optimization, while preserving the inspectability that current contract-driven and graph-based approaches provide (Shin et al., 11 Jun 2026, Lin, 7 May 2026, Li et al., 18 Feb 2026).