EvoMaster: Evolving Agents and REST API Testing
- EvoMaster is a dual-purpose framework combining an evolving agent system for scientific discovery with a search-based tool for automated, robust REST API testing.
- The evolving agent framework features modular design, continuous self-evolution, and reproducible multi-agent workflows that yield significant benchmark improvements.
- The REST API testing tool utilizes search-based techniques and white-box instrumentation to achieve high code coverage and effective fault detection in diverse systems.
to=arxiv_search.search สำนักเลขานุการองค์กร тәшкиڭjson {"query":"EvoMaster arXiv", "max_results": 10} to=arxiv_search.search 北京赛车如何json to=arxiv_search.search 】【。】【”】【json {"query":"ti:EvoMaster", "max_results": 20} EvoMaster is the name used in recent arXiv literature for two distinct software systems. One is a foundational evolving agent framework engineered for Agentic Science at Scale, with continuous self-evolution, multi-agent collaboration, and a domain-agnostic base harness for scientific work (Zhu et al., 19 Apr 2026). The other is an open-source, search-based tool for automated system-level test generation of RESTful APIs, later extended across white-box and black-box fuzzing, industrial microservice testing, database-aware testing, security oracles, and flakiness mitigation (Arcuri, 2019).
| Usage in literature | Representative paper | Domain |
|---|---|---|
| Evolving agent framework | "EvoMaster: A Foundational Evolving Agent Framework for Agentic Science at Scale" (Zhu et al., 19 Apr 2026) | Agentic science |
| Search-based testing tool | "EvoMaster: Evolutionary Multi-context Automated System Test Generation" (Arcuri, 2019) | REST API and system-level testing |
1. Agentic-science EvoMaster
In the agentic-science line, EvoMaster is presented as a foundational evolving agent framework explicitly engineered for Agentic Science at Scale. The motivating claim is that the scientific method is inherently iterative, whereas many existing LLM-agent systems are fragmented, siloed, single-pass, and unable to accumulate experience across runs. EvoMaster addresses this by combining a domain-agnostic base harness with continuous self-evolution, so that agents can iteratively refine hypotheses, self-critique, and accumulate knowledge across experimental cycles (Zhu et al., 19 Apr 2026).
The framework is organized around two core tenets: a foundational harness reusable across scientific domains, and continuous self-evolution. These are instantiated in four design principles. First, modular composability is achieved through orthogonal layers, unified registries, MCP (Model Context Protocol) for tools, and Skills for reusable capability packages. Second, EvoMaster treats every agent run as a reproducible experiment through YAML configuration manifests and structured, thread-safe trajectory logging. Third, the central execution loop is iterative rather than single-shot: reason → invoke tools → observe → self-critique → refine → repeat. Fourth, the framework supports multi-agent collaborative evolution, with named roles such as solver, critic, rewriter, and selector (Zhu et al., 19 Apr 2026).
A notable engineering claim is that new scientific domains can be added in approximately 100 lines of code by writing a new Playground and plugging domain tools and skills into the shared core. This suggests that the framework is intended less as a domain-specific agent and more as reusable infrastructure for families of agents (Zhu et al., 19 Apr 2026).
2. Architecture and operation of the evolving-agent framework
The execution architecture separates Playground, Exp, and Agent. Playground is the orchestration layer for multi-agent workflows and domain-specific pipelines; Exp manages the lifecycle of a single run, including configuration loading and trajectory recording; Agent is the intelligence layer built around the BaseAgent abstraction and the multi-turn reasoning and tool-use loop. This separation is designed so that changes to context management or reasoning logic propagate across domains without rewriting domain playbooks (Zhu et al., 19 Apr 2026).
The Agent Engine uses a ContextManager with sliding windows and LLM-based summarization to maintain an evolving context across hundreds of turns. Old information is compressed while preserving salient findings, which is intended to prevent context-window overflow during long-horizon tasks such as web-scale browsing or full ML competitions. The capability layer includes an MCP-based tool system following an Action → Execution → Observation pattern, an Anthropic-style Skill system with lightweight metadata plus on-demand detailed instructions, and an LLM abstraction via LiteLLM for swapping among backend models (Zhu et al., 19 Apr 2026).
On top of this core, the paper describes the SciMaster ecosystem, including ML-Master / ML-Master 2.0, X-Master / X-Master 2.0, Browse-Master, PhysMaster, and EmboMaster. These systems reuse the same agent engine, capability layer, orchestration layer, and experiment harness, while specializing tools, skills, and workflow logic for particular domains (Zhu et al., 19 Apr 2026).
3. Benchmarks, performance, and limitations of the agentic-science framework
The reported evaluation uses GPT-5.4 as the backend model for both EvoMaster and the baseline OpenClaw, across four benchmarks directly tied to agentic-science capabilities: Humanity’s Last Exam, MLE-Bench Lite, BrowseComp, and FrontierScience. EvoMaster reports 41.1%, 75.8%, 73.3%, and 53.3%, respectively, with relative improvements over OpenClaw ranging from +159% to +316% (Zhu et al., 19 Apr 2026).
The benchmark-specific agents illuminate how evolution is operationalized. ML-Master 2.0 uses knowledge prefetch, drafting, iterative improvement for up to 20 rounds, and hierarchical cognitive caching on MLE-Bench Lite, reaching a 75.8% any-medal rate versus 18.2% for OpenClaw. Browse-Master uses a planner–executor loop for up to 10 rounds on BrowseComp, reaching 73.3% accuracy versus 28.3%. X-Master uses a four-phase solve–critique–rewrite–select pipeline on HLE, reaching 41.1% versus 13.6%. X-Master 2.0 applies tool-augmented literature search and synthesis on FrontierScience, reaching 53.3% averaged score versus 18.3% (Zhu et al., 19 Apr 2026).
The paper also states explicit limitations. EvoMaster is optimized for in silico work and does not yet natively support cloud labs or robotic experimental setups. Its gains are demonstrated on top of GPT-5.4, so weaker models may reduce performance. The framework has context management and caching, but no formal long-term memory persistence standard is detailed beyond caches, and safety considerations are not a central focus of the paper (Zhu et al., 19 Apr 2026).
4. REST-API testing EvoMaster: origin and technical core
In the older and larger API-testing literature, EvoMaster is an open-source tool for automated system-level test generation for RESTful web services running on the JVM. It applies search-based software testing to sequences of HTTP calls, using evolutionary algorithms such as MIO, WTS, and MOSA, plus white-box instrumentation to guide coverage and fault discovery (Arcuri, 2019). A closely related early technique paper describes the same line as a fully automated white-box approach to RESTful API testing, where tests are rewarded based on code coverage and fault-finding metrics, and reports 38 real bugs across two open-source systems and one industrial system (Arcuri, 2019).
Architecturally, this EvoMaster is split into a core process and a driver process. The core runs the search algorithm, maintains populations of candidate test suites, and generates output tests; the driver starts, stops, and resets the SUT, performs bytecode instrumentation, and exposes coverage and runtime data. The tool relies on Swagger/OpenAPI to enumerate endpoints, parameters, headers, and body schemas, and produces executable regression suites such as JUnit + RestAssured tests (Arcuri, 2019).
The core search representation is a sequence of one or more HTTP calls. Fitness is driven by structural coverage and fault signals, especially statement coverage, branch coverage, and HTTP 5xx responses. In the 2019 system paper, the summarized empirical outcome is 20–40% statement coverage together with 38 unique bugs, while also emphasizing the difficulty of database-heavy logic and the need for state reset and authentication support in practical deployments (Arcuri, 2019).
5. API-testing extensions and research ecosystem
Subsequent work turns the REST-testing EvoMaster into a research platform for new heuristics, search strategies, databases, and security analyses. "Advanced White-Box Heuristics for Search-Based Fuzzing of REST APIs" adds method replacements, taint analysis, support for under-specified REST schemas, JPA/SQL constraint alignment, bean validation targets, and timing-related mitigations, and reports clear improvements on 14 APIs from the EMB corpus, plus one industrial API (Arcuri et al., 2023). "Automated Test-Case Generation for REST APIs Using Model Inference Search Heuristic" augments EvoMaster with MISH, which learns a finite-state model from logs and reports especially strong gains on larger-endpoint systems such as ProxyPrint and OCVN (Cao et al., 2024). "LLM-assisted Mutation for Whitebox API Testing" describes MioHint as an extension of EvoMaster 3.2.0, improving average line coverage from 48.51% to 53.46%, hard-target coverage from 9.82% to 57.54%, and mutation hit rate from 0.35% to 22.17% on 16 real-world REST API services (Li et al., 8 Apr 2025).
Database-aware variants extend the same platform further. "Search-Based Fuzzing For RESTful APIs That Use MongoDB" adds driver-level interception of MongoCollection.find(), a NoSQL distance heuristic, and direct MongoDB insertions from test cases, reporting coverage gains of up to 18% on six MongoDB-backed APIs (Ghianni et al., 28 Jul 2025). "Enhancing REST API Fuzzing with Access Policy Violation Checks and Injection Attacks" adds automated security oracles for access-policy violations, SQL Injection, XSS, stack traces, and hidden endpoints, and evaluates them on 52 distinct APIs (Sahin et al., 1 Apr 2026). "Detecting and Mitigating Flakiness in REST API Fuzzing" studies flakiness in tests generated by EvoMaster across 36 REST APIs, analyzes nearly 3000 failing tests, and proposes FlakyCatch to detect and mitigate assertion-level instability (Zhang et al., 30 Mar 2026).
A standardization effort appears in "WFC/WFD: Web Fuzzing Commons, Dataset and Guidelines to Support Experimentation in REST API Fuzzing", which generalizes authentication configuration and fault schemas originally developed inside EvoMaster. In that study, EvoMaster in black-box mode attains 57.2% average 2xx endpoint coverage, 11.0 endpoints with at least one 500, and 45.5% average line coverage during fuzzing across 36 APIs, outperforming RESTler, Schemathesis, ARAT-RL, EmRest, and LLamaRestTest on the reported averages (Sahin et al., 1 Sep 2025).
6. Industrial use, benchmark studies, and nomenclature
Industrial studies show EvoMaster being used not only as a research prototype but also as testing infrastructure. A Meituan study reports integrating EvoMaster into testing processes over almost 2 years, involving 321,131 lines of code from five APIs and 27 industrial participants, and reports clear advantages in code coverage and fault detection together with persistent challenges in industrial microservice testing (Zhang et al., 2022). In a healthcare IoT study over 17 APIs with 120 endpoints and 14 releases, EvoMaster in black-box mode reaches about 84% REST API coverage and participates in the detection of 18 potential faults and 23 regressions, although the overall automation cost remains high (Sartaj et al., 2024).
A distinct healthcare line studies the Cancer Registry of Norway’s GURI rule engine. "Cost Reduction on Testing Evolving Cancer Registry System" introduces EvoClass, an EvoMaster extension with a Random Forest pre-filter that achieves about 31% cost reduction without reducing rule coverage (Isaku et al., 2023). "Testing Medical Rules Web Services in Practice" compares EvoMaster black-box and white-box variants with EvoGURI, finding that EvoGURI and EvoMaster's black-box mode produce test suites that cover the highest number of rules with Pass, Fail, and Warning results, while mutation testing favors EvoGURI and two EvoMaster white-box tools (Laaber et al., 2024). "Quantum Neural Network Classifier for Cancer Registry System Testing: A Feasibility Study" then replaces EvoClass’s classical model with a QNN-based classifier, reporting 0.921 accuracy for the best QNN configuration versus 0.946 for EvoClass (Wang et al., 2024).
A further black-box evaluation lens appears in "Assessing REST API Test Generation Strategies with Log Coverage", where EvoMaster v5.0.2 is compared with human-written Locust tests and LLM-generated tests on Light-OAuth2. EvoMaster yields 65 distinct log templates on average versus 88 for Locust and 113 for Claude Opus 4.6, but combining EvoMaster with Locust still increases total observed log coverage by 30.7% over Locust alone and 76.9% over EvoMaster alone, indicating substantial behavioral complementarity (Reinikainen et al., 8 Apr 2026).
Taken together, the citation record shows that EvoMaster is not a single codebase or research program. It designates, first, a 2026 evolving-agent framework for scientific discovery and, second, a long-running API-testing framework originating in 2019 and extended through heuristics, database awareness, security testing, benchmarking infrastructure, and flakiness mitigation. This shared name is therefore a nomenclature coincidence rather than a unified technical lineage (Zhu et al., 19 Apr 2026).