ToolPro: Agentic Tool Program Execution

• ToolPro is a system that transforms LLM-generated tool intents into executable programs with control flow, enabling complex agentic interactions with web services.
• It employs a four-stage pipeline—projection, compilation to Wasm, and bounded repair—to ensure safe, exactly-once execution of state-modifying operations.
• The architecture dynamically selects between program mode and stepwise execution based on runtime metrics, significantly reducing latency and client-side traffic.

ToolPro is a system for exposing an agent’s “tool intent” as an executable tool program rather than as a sequence of static API calls. In the formulation introduced in “Beyond Static Endpoints: Tool Programs as an Interface for Flexible Agentic Web Services” (Liu et al., 18 Jun 2026), an LLM-based agent synthesizes a short procedure whose only side effects are annotated calls into a web service; the ToolPro server then projects, compiles, repairs, and runs that program inside a WebAssembly sandbox, mediating every call to enforce exactly-once semantics for writes and falling back to the classic stepwise interface if consolidation is predicted to be unhelpful or if execution fails. The system is motivated by the observation that static endpoints poorly express long-horizon workflows with loops, conditionals, joins, and retries, even though such control flow is common in agentic web services.

1. Conceptual model

ToolPro defines a tool program $P$ as a small, Turing-equivalent procedure whose atomic operations are calls of the form $\mathrm{Call}(e,a)$ into a fixed endpoint set $\mathcal{E}$ of a target service (Liu et al., 18 Jun 2026). The program may contain loops, conditionals, variable binding, local computation, and explicit effect annotations on each call site. Formally, if $s$ denotes the service’s internal state, execution is written as

$\mathrm{exec}(P,s)\rightarrow (y,s')$

where execution proceeds in program order, emits zero or more endpoint invocations, returns a final result $y$, and may update the state to $s'$.

This representation is intended to compactly encode multi-step interactions that would otherwise require repeated client-agent-service rounds. Instead of a sequence such as $\langle (e_1,a_1),(e_2,a_2),\dots,(e_N,a_N)\rangle$, each with a separate LLM reasoning pass, ToolPro consolidates the workflow into one object. The paper’s illustrative example expresses a loop over user identifiers, performs a GetMemo read, branches on memo.visibility, and conditionally issues a SetVisibility write inside the same program. In that sense, ToolPro is not merely a serialization format for tool calls; it is an execution interface for structured service interaction.

A common misconception is that ToolPro simply batches calls. The source formulation is more specific: the consolidated object is executable code with control flow and annotated side effects, not a flat macro or static request bundle. This distinction matters because retries, branching, and data-dependent continuation are first-class within the program model.

2. Effect types and semantic guarantees

ToolPro assigns each endpoint $e\in\mathcal{E}$ an effect label

$\mathrm{eff}(e)\in\{\mathrm{READ},\mathrm{WRITE}\}$

to distinguish pure queries from state-modifying operations (Liu et al., 18 Jun 2026). A dynamic invocation is therefore written as $\mathrm{Call}(e,a)$0. The observed execution trace of a program is

$\mathrm{Call}(e,a)$1

where $\mathrm{Call}(e,a)$2 records each call and its response, and only WRITE calls may change service state.

The paper identifies two semantic properties as central. The first is observational equivalence: given the same low-level trace $\mathrm{Call}(e,a)$3, program mode returns the same outputs or errors as stepwise execution. The second is retry safety: across in-place repairs and re-executions, each logical WRITE is emitted at most once. These guarantees are narrower than full semantic equivalence over all possible executions, but they are directly aligned with the practical problem ToolPro addresses, namely safe consolidation of agentic workflows over services that may mutate state.

These semantics are significant because program-mode execution introduces a new failure surface—compilation errors, runtime crashes, and repair-induced re-execution—that does not arise in ordinary stepwise tool use. ToolPro’s effect system therefore serves not only as a descriptive typing discipline, but as the mechanism by which safe replay and write mediation become well-defined.

3. Program construction, projection, and bounded repair

ToolPro uses a four-stage, feedback-driven construction pipeline to transform LLM-generated code into a verifiable executable or else fail closed (Liu et al., 18 Jun 2026).

First, on the client side, the agent synthesizes a program and performs lightweight checks. These checks verify that required endpoints appear in the program, that the intended loop or conditional skeleton is present, and that outputs from READ sites feed into later calls. Second, on the server side, the projection operator $\mathrm{Call}(e,a)$4 rewrites each external invocation into a uniform host stub, $\mathrm{exec}(P,s)\rightarrow (y,s')$8 rejects disallowed constructs such as exceptions, dynamic linking, unsafe memory, and ad-hoc networking or filesystem access, and ensures that call-site effect annotations match the declared $\mathrm{Call}(e,a)$5.

Third, the projected program $\mathrm{Call}(e,a)$6 is compiled via Rustc to Wasm. If compilation errors or runtime crashes occur, ToolPro enters a bounded in-place repair loop. The system reads diagnostics or traces, applies minimal local edits such as fixing imports, adjusting JSON fields, or correcting an effect tag, then re-projects, re-compiles, and re-runs up to a fixed attempt budget. Fourth, if that budget is exceeded or a repair violates constraints, ToolPro reverts to stepwise calling and surfaces the collected diagnostics to the client.

This pipeline establishes that arbitrary LLM output is not executed directly. A plausible implication is that ToolPro’s practical novelty lies in combining agent synthesis with a constrained compilation target and a fail-closed recovery path, rather than assuming that generated code is trustworthy by default.

4. Replay discipline, exactly-once writes, and consolidation policy

The core of ToolPro’s retry safety is a log-and-replay algorithm over WRITE calls (Liu et al., 18 Jun 2026). For each intent instance, the system maintains an ordered history log $\mathrm{Call}(e,a)$7 of completed WRITE calls from prior executions and a working log $\mathrm{Call}(e,a)$8 for the current execution; each entry has a used-flag. On re-execution, $\mathrm{Call}(e,a)$9, $\mathcal{E}$0, and all entries in $\mathcal{E}$1 are marked unused. When handling a READ, ToolPro forwards the call and returns the real-time result. When handling a WRITE, it scans $\mathcal{E}$2 in order for an unused matching $\mathcal{E}$3 pair; if a match exists, it returns the cached outcome and marks that entry used, and only if no match exists does it emit the actual service call and append the result to $\mathcal{E}$4.

Under the paper’s replay discipline—repairs must not reorder or change already-committed WRITE prefixes—the resulting proposition is exactly-once forwarding: each logical WRITE is sent to the service at most once, and later executions replay the cached result. This is a strong operational property for stateful services, particularly when repair or retry is part of the normal execution model.

ToolPro does not always prefer consolidation. It estimates whether program mode is worthwhile using moving averages over recent tasks:

• $\mathcal{E}$5: average client-service round-trip time
• $\mathcal{E}$6: average per-step client LLM decision time
• $\mathcal{E}$7: average one-time build cost

If $\mathcal{E}$8 denotes the estimated number of dynamic Call sites, the paper gives

$\mathcal{E}$9

and

$s$0

so that predicted savings are

$s$1

If $s$2, ToolPro uses program mode; otherwise it remains stepwise. During cold start, the system bootstraps with 2–3 stepwise runs to warm up $s$3 and $s$4 and forbids program mode until the synthesized program clearly contains multi-step logic.

A frequent misunderstanding is that ToolPro is designed to replace stepwise agents universally. The described policy states the opposite: ToolPro chooses between stepwise and program execution, and the empirical evaluation explicitly studies when each regime dominates.

5. Architecture and execution over MCP-style services

ToolPro is instantiated over MCP-style services with a split client-server architecture (Liu et al., 18 Jun 2026). On the client side, an LLM, exemplified in the paper by Qwen3-Max, receives tool specifications and an intent and generates Rust code $s$5; stage-1 checks and the consolidation decision are then performed. On the server side, the system projects $s$6 to constrained Rust, compiles it to Wasm via Rustc, executes it under Wasmtime, exposes only one host import for mediated service invocation, runs the bounded repair loop on compile or runtime errors, applies effect-aware replay to every call, and falls back to stepwise execution on constraint or repair failure.

The paper gives four reasons for using WebAssembly: strong capability-based sandboxing with no ambient I/O, filesystem, threads, or dynamic linking; deterministic compilation targets for stable diagnostics; low overhead on the order of tens of milliseconds for short procedural workloads; and a host importer interface that allows central mediation of side effects. These design choices locate ToolPro at the boundary between LLM agent planning and systems-level service execution, rather than solely within prompting or retrieval.

A concrete example is provided for a Memos read-write workflow with $s$7. In stepwise mode, the agent repeatedly decides ListMemos, then GetMemo, then PatchMemo over three rounds. ToolPro instead synthesizes one Rust program:

$\mathrm{exec}(P,s)\rightarrow (y,s')$9

The server projects this program into a Wasm stub, compiles it, and runs it with one RTT to upload $s$8 and one RTT to execute the entire loop. The source text further states that exactly-once replay protects the three WRITE calls if the program must be repaired or re-executed.

6. Benchmarks, empirical results, and relation to adjacent tool-use research

ToolPro is evaluated on three open-source MCP services—Memos (Go/REST), Directus (Node/GraphQL), and MinIO (Go/S3-compatible)—each with read-only and read-write workflows parameterized by $s$9, plus four supplemental realistic workflows, cbench1 through cbench4, involving nondeterministic retrieval, branching, coordinated writes, and cross-service logic (Liu et al., 18 Jun 2026). The reported metrics are end-to-end latency, defined as wall-clock time from initial client intent to final result including LLM time, network RTTs, and server execution, and client-side traffic, defined as bytes transmitted by the client across client–LLM prompts and client–server requests.

Benchmark setting	Reported outcome	Interpretation
.r/.w workflows at $\mathrm{exec}(P,s)\rightarrow (y,s')$0	latency reduced by 25–50%; client traffic cut by 60–96.1%	gains on read-heavy and multi-service tasks
Increasing RTT from 1 ms to 2000 ms	full policy switches from stepwise to program mode and always matches or beats the better static strategy	policy tracks network regime
cbench1–cbench3	latency 30.16 s $\mathrm{exec}(P,s)\rightarrow (y,s')$1 17.91 s; accuracy 0.60 $\mathrm{exec}(P,s)\rightarrow (y,s')$2 0.93; LLM time cut by 52.8%	consolidation reduces repeated reasoning
cbench4	latency 52.68 s $\mathrm{exec}(P,s)\rightarrow (y,s')$3 24.54 s; traffic $\mathrm{exec}(P,s)\rightarrow (y,s')$4; accuracy $\mathrm{exec}(P,s)\rightarrow (y,s')$5	largest gain in high-RTT cross-service case

The paper also reports that, as workflow complexity $\mathrm{exec}(P,s)\rightarrow (y,s')$6 grows, stepwise latency scales linearly in $\mathrm{exec}(P,s)\rightarrow (y,s')$7, whereas program mode remains nearly flat as one build plus one run; the full ToolPro system selects the better strategy. The headline aggregate results are up to 53.4% reduction in end-to-end latency and up to 96.1% reduction in client-side traffic.

Within the broader literature on LLM tool use, ToolPro occupies a distinct systems layer. PTR formulates tool recommendation as selection of a precise tool set before execution, using tool bundle acquisition, functional coverage mapping, and multi-view re-ranking, and evaluates recommendation quality with TRACC (Gao et al., 2024). JTPRO addresses tool-calling reliability in large inventories by jointly optimizing a global instruction prompt and per-tool schema or argument descriptions to improve Tool Selection Accuracy, Slot Filling Accuracy, and Overall Success Rate (Ghoshal et al., 20 Apr 2026). ToolPro instead addresses the execution interface after tool intent has been identified, replacing repeated endpoint invocation with an executable, effect-typed program over services. This suggests a compositional view of the stack: recommendation methods such as PTR can reduce the candidate tool set, reflective optimization methods such as JTPRO can sharpen selection and slot semantics, and ToolPro can then consolidate the resulting multi-step interaction into a sandboxed program.