Tool-Augmented Demonstration Construction

• Tool-augmented demonstration construction is a framework where external tools actively participate in creating, expanding, and validating operational demonstrations rather than relying on static examples.
• Its methodology spans robotics, language agents, and geospatial reasoning by combining embodied capture, simulation-based verification, and tool-augmented dialogue or code synthesis.
• Empirical studies show that careful validation, controlled abstraction, and iterative expansion of demonstrations lead to significant improvements in downstream performance and system reliability.

Tool-augmented demonstration construction denotes a family of methods in which external tools participate not only in downstream inference or control, but in the creation, expansion, validation, compression, or long-term reuse of demonstrations themselves. Across recent work, the constructed artifact may be a robot trajectory, a tool-use dialogue, a code-interpreter trace, a geospatial reasoning sequence, a rehearsed browser demo, or an executable tutorial. What unifies these settings is that demonstrations are treated as operational objects grounded in tool interfaces, environment state, and validation procedures rather than as static input-output examples alone (Yang et al., 2024, Shabbir et al., 19 Feb 2026).

1. Scope and historical broadening of the concept

In imitation learning for robotics, demonstration construction begins from physical or embodied interaction. ARCADE uses augmented reality to capture one human-performed seed demonstration with a Microsoft HoloLens 2, then algorithmically expands it into a much larger dataset with user review and automatic filtering (Yang et al., 2024). SaLfD for LEGO construction likewise observes human assembly, infers symbolic brick operations, and uses a Gazebo digital twin to verify and correct the inferred plan before robot execution (Liu et al., 2023). In these systems, “demonstration construction” is literal dataset construction from embodied behavior.

In tool-using language agents, the emphasis shifts from physical traces to executable reasoning trajectories. TALM represents a demonstration as a text sequence containing task input, tool call, tool result, and final output, then grows a tool-use corpus through iterative self-play from few seed demonstrations (Parisi et al., 2022). Self-Demos constructs query-aware in-context examples to bridge out-of-demonstration queries to the available tool specification and seed demos (He et al., 2024). Customer-Agent generates multi-turn code-interpreter traces and retains only those whose tool calls execute successfully and whose final answers are correct (Liu et al., 6 Jun 2026). HyperTool goes further by changing the demonstration unit itself from atomic tool calls to an executable block that can locally compose existing tools (Du et al., 11 Jun 2026).

A third strand treats demonstrations as reusable infrastructure for presentation, instruction, or evaluation. ALD and HALD embed executable logic tools directly into web documents, turning tutorials into active, tool-backed demonstrations (Ferreiro et al., 2023). Rhetor automatically constructs a rehearsed live product demonstration from a running web application and its repository, including scripted browser actions, synchronized narration, and real-time QA support (Khedar et al., 29 Jun 2026). TRACE synthesizes multi-turn tool-augmented conversations across positive and negative situations, effectively supplying a scenario library for dialogue-level demonstration design (Hou et al., 22 Oct 2025). This breadth suggests that the topic now spans both learning-time supervision and author-time construction of executable demonstrative artifacts.

2. Demonstration as trajectory, artifact graph, or executable block

A recurrent representational pattern is the shift from plain labels to stateful trajectories. In ARCADE, the seed demonstration is recorded exactly as

$\tau^{AR} \coloneqq \{(p_i, j_i, g_i)\}_{i=1}^{N},$

where $p_i$ is end-effector pose, $j_i$ robot joints, and $g_i$ binary gripper state (Yang et al., 2024). In OpenEarthAgent, the formal trajectory is

$\Gamma_i = \{(s_t, r_t)\}_{t=1}^{T_i},$

with $s_t$ a predicted tool invocation and $r_t$ the structured observation returned by the execution environment (Shabbir et al., 19 Feb 2026). In Customer-Agent, the interaction is serialized as a hybrid trajectory

$$\tau = \left[g_1 \oplus c_1 \oplus f_1 \oplus g_2 \oplus c_2 \oplus f_2 \oplus \cdots \oplus o \right],$$

where reasoning text, code blocks, interpreter feedback, and final answer are interleaved (Liu et al., 6 Jun 2026). TALM uses the same broad idea in text-only form: task input, tool input, tool output, and final task output are concatenated into a single supervision sequence (Parisi et al., 2022).

The same shift appears at the level of memory and state. ML-Tool-Bench formalizes each task as an MDP whose state contains “all AI, Human, and Tool messages together with artifacts such as dataframes and ML models,” and it introduces in-memory named object management so that demonstrations encode not only tool calls but also bindings of intermediate objects by name (Chittepu et al., 29 Nov 2025). OpenEarthAgent similarly relies on a central orchestrator with standardized JSON-based tool contracts and a cache of derived vector layers, raster subsets, index maps, and computed geometries (Shabbir et al., 19 Feb 2026). ToolMATH exposes only tool name, natural-language description, and typed input schema to the model, making demonstrations depend on disciplined use of tool metadata and observations rather than hidden implementations (Choi et al., 24 Feb 2026).

HyperTool makes the granularity issue explicit. Standard MCP-style interaction uses an atomic action

$a_t = \mathrm{Call}(\tau_t, x_t),$

whereas HyperTool changes the model-visible action to

$a_t = \mathrm{Block}(S_t),$

with $p_i$0 a bounded executable program over primitive tools (Du et al., 11 Jun 2026). This suggests that demonstration construction is not only about collecting more traces; it is also about deciding the right unit of supervision, from waypoint sequence to dialogue turn to code-interpreter block.

3. Construction operators: capture, expansion, synthesis, and distillation

One line of work constructs large demonstration sets from a small embodied seed. ARCADE first captures one AR demonstration, then samples random waypoint subsets from the pose trajectory, preserves grasp/release and slow sharp-turn key poses, plans motions through the resulting waypoint set with MoveIt! and IK, and submits a small candidate set to the user for approval. The retained set is typically 10–15 demonstrations, after which the system continues generating and automatically accepting new candidates by Dynamic Time Warping similarity to the approved subset; the reported experiments use 100 generated demonstrations for BC training (Yang et al., 2024). SaLfD follows a related proposal-and-check pattern: a task extractor infers candidate LEGO brick operations from consecutive RGB-D keyframes, and simulation verifies the resulting state against the observed human-demonstrated state, accepting a candidate if similarity $p_i$1 and otherwise trying lower-ranked candidates (Liu et al., 2023).

A second line uses LLMs to synthesize demonstrations or pseudo-demonstrations conditioned on the target problem. Self-Demos first performs query understanding,

$p_i$2

then generates query-aware demos

$p_i$3

selects the best subset

$p_i$4

and answers with the seed and selected generated demos jointly in context

$p_i$5

On OOD-Toolset, the best configuration generates $p_i$6 demos and keeps $p_i$7 (He et al., 2024). TALM uses a different expansion operator: given a small tool-use set $p_i$8, it samples tool queries $p_i$9, executes them, generates answers $j_i$0, and adds successful trajectories back into $j_i$1 across self-play rounds (Parisi et al., 2022).

A third line synthesizes fully executable tool-grounded corpora. OpenEarthAgent aggregates candidate regions, annotations, and metadata from geospatial sources, harmonizes them into a unified JSON schema, synthesizes natural-language queries and reasoning trajectories with prompt templates and one-shot exemplars, and then replays every accepted trajectory through the tool controller before inclusion in training (Shabbir et al., 19 Feb 2026). Customer-Agent starts from retrieval-verifiable ShopTrajQA items, prompts a strong model to answer through code-interpreter interaction, and keeps only the resulting traces whose tool calls execute and whose final answers are correct (Liu et al., 6 Jun 2026). UCT distills inference-time experience one step further: when the agent detects a missing capability, it emits a build ticket summarizing the subproblem, synthesizes executable tool code plus tests in an isolated build loop, verifies the tool through sandbox execution and critique, packages it with invocation instructions and dependencies, and later consolidates it into long-term memory (Shen et al., 2 Feb 2026). This suggests a continuum from trace expansion to trace synthesis to trace compilation into reusable tools.

4. Validation, curation, and the transformation of traces into reliable assets

Validation is the decisive operation that separates a mere generated trace from a usable demonstration. In ARCADE, quality control is intentionally two-stage: human validation in AR catches unnatural IK motions, hazardous motions, and preference violations on a small initial set, after which automatic validation accepts a new trajectory $j_i$2 only if its DTW-based similarity statistic satisfies

$j_i$3

with $j_i$4 given as an example of the coverage-homogeneity tradeoff (Yang et al., 2024). SaLfD likewise uses simulator feedback as a validator: if no candidate operation crosses the acceptance threshold, it falls back to the candidate maximizing similarity with the observed end state (Liu et al., 2023).

Tool-grounded language systems tend to prefer deterministic replay or execution filtering. OpenEarthAgent requires programmatic execution of candidate GIS queries during construction, then deterministic replay of each full trajectory to verify argument formatting, coordinate integrity, geometric validity, spatial consistency, and full-chain executability (Shabbir et al., 19 Feb 2026). Customer-Agent filters cold-start SFT traces so that all tool calls execute successfully and the final answer is correct, while RLVR later rewards both final-answer correctness and executable intermediate tool use (Liu et al., 6 Jun 2026). HyperTool removes traces with malformed block syntax, missing tool invocations, environment-level tool errors, unresolved execution failures, and other anomalies, then subjects the survivors to evidence-consistency verification by an LLM judge with majority vote; removing execution filtering drops average accuracy to 18.06%, removing evidence filtering to 21.05%, versus 33.33% for the full system (Du et al., 11 Jun 2026).

Benchmark work turns these validators into explicit failure taxonomies. ToolMATH validates tools first against test cases and then question-wise through successful traces, while its manual analysis identifies plan error, tool selection error, wrong parameter value, formatting error, thought error, observation omission, repeated call, and incomplete execution as distinct failure modes (Choi et al., 24 Feb 2026). TRACE shows that dialogue-level validation must separate tool execution correctness, agent performance, user satisfaction, and overall conversation success, because a user can appear satisfied in conversations that are objectively bad (Hou et al., 22 Oct 2025). This suggests that demonstration construction increasingly depends on multi-layer curation criteria: execution validity, semantic correctness, trajectory coherence, and severity of latent failure.

5. Empirical support across robotics, agent planning, geospatial reasoning, and dialogue

In robotics, demonstration construction is directly tied to downstream control quality. ARCADE reports that the full system trained with 100 generated demonstrations achieved the best performance on all three evaluated Fetch tasks, with two-way ANOVA showing significant main effects of both demonstration collection type and demonstration set size at $j_i$5 for all three tasks; on the household Pouring-Water task it achieved an 80% success rate, 8/10 trials (Yang et al., 2024). SaLfD reports consistent gains from simulation-aided correction over plain LfD across custom LEGO objects: for example, success on “AI” rises from 73% to 100%, “Spiral” from 71.0% to 98.0%, and “Temple” from 63.5% to 95.2% (Liu et al., 2023). These results support the proposition that validation and augmentation can substitute for large-scale manual collection.

In tool-using ML and multimodal agents, the same pattern appears at larger workflow scale. ML-Tool-Bench defines a benchmark with 61 specialized tools and 15 Kaggle-style challenges, and reports that on GPT-4o, Hierarchical MCTS improves over ReAct by 16.52 percentile positions in median leaderboard percentile across competitions (Chittepu et al., 29 Nov 2025). SpaceTools constructs an 8k-teaching dataset from two teachers—6k trajectories from a universal teacher and 2k from an IRL-trained teacher—and shows that removing either source lowers mean performance, while direct IRL over the full tool space collapses to 19.79 mean versus 52.48 for the full system (Chen et al., 3 Dec 2025). HyperTool trains on 10,422 verified trajectories and improves average accuracy from 20.92% to 33.33% on Qwen3-8B and from 24.18% to 35.29% on Qwen3-32B relative to ReAct-SFT on the same synthesized tasks and filtering pipeline (Du et al., 11 Jun 2026). Customer-Agent shows the same lesson in long-context retrieval: on ShopTrajQA, Qwen3-4B in naïve tool mode scores 13.8 at 32k and 9.3 at 64k, while the SFT-RLVR agent reaches 62.5 and 60.1, indicating that tool access alone is much weaker than tool-augmented demonstration construction plus verifiable training (Liu et al., 6 Jun 2026).

OpenEarthAgent demonstrates corpus-scale construction of verified tool-grounded reasoning in the geospatial setting. Its released corpus contains 14,538 training instances and 1,169 evaluation instances, with 100,656 reasoning steps in training and 7,064 in evaluation, spanning urban, environmental, disaster, infrastructure, and related domains (Shabbir et al., 19 Feb 2026). ToolMATH, while primarily an evaluation benchmark, supplies the same type of evidence for long-horizon multi-tool trajectories: after validation it contains 7,699 questions and 12,369 tools, plus a 329-question hard split, and it finds that the central failure factor is not merely tool-list redundancy but the inability to reason across long trajectories without accumulating intermediate-result errors (Choi et al., 24 Feb 2026). TRACE contributes analogous evidence for dialogue: its 516 synthesized conversations over 26 situations show that hard negatives, especially those where the user appears satisfied, require a richer notion of demonstration quality than surface conversational success (Hou et al., 22 Oct 2025).

Beyond learning datasets, the same construction logic appears in executable documents and live product demos. ALD and HALD turn documentation into active, browser-local demonstrations with embedded runnable logic and tool-filtered outputs, including verification tutorials for CiaoPP (Ferreiro et al., 2023). Rhetor constructs a rehearsed live demonstration artifact with scenes, grounded actions, locator strategies, rehearsal logs, and synchronized narration; across six sessions on four applications, the rehearser’s internal locator-firing rate $j_i$6 spans 0.31–1.00 over 147 scripted actions, with Excalidraw reaching $j_i$7 at iteration 2 after locator repair (Khedar et al., 29 Jun 2026). This broadens the empirical basis of the field: tool-augmented demonstration construction is not confined to training corpora, but extends to executable pedagogical and presentational artifacts.

6. Persistent design tensions and open problems

A first tension concerns the unit of construction. HyperTool argues that standard tool supervision suffers from an “execution-granularity mismatch,” because deterministic local workflows are unfolded into many model-visible micro-decisions (Du et al., 11 Jun 2026). ToolMATH, by contrast, shows that gains in realistic long-horizon settings come less from local action selection than from long-range plan coherence and disciplined use of observations (Choi et al., 24 Feb 2026). ML-Tool-Bench adds that decomposition alone is insufficient unless the relevant tool subset is also restricted; removing tool masking in Hierarchical MCTS causes median consistency to drop from 0.8 to 0.3 and median leaderboard percentile from 21.10 to 0 on the selected GPT-4.1-mini study (Chittepu et al., 29 Nov 2025). Together these results suggest that demonstration construction must choose its abstraction level carefully: too atomic, and traces become noisy bookkeeping; too compressed, and planning or interpretability can degrade.

A second tension concerns diversity versus validity. ARCADE makes the tradeoff explicit through $j_i$8: stricter DTW filtering yields more homogeneous demonstrations but less state-space coverage, while looser filtering increases diversity and the risk of bad trajectories (Yang et al., 2024). OpenEarthAgent accepts only clean, replayable trajectories, but this also means that accepted training traces appear to contain little explicit failure recovery or branch correction (Shabbir et al., 19 Feb 2026). TRACE shows that some failure types are ambiguous without additional latent annotations such as world status, particularly when visible dialogue and tool output do not reveal that an action silently failed (Hou et al., 22 Oct 2025). Customer-Agent reveals a related phenomenon from the opposite direction: naïve tool access can degrade performance relative to plain text models, so demonstration construction must teach not just tool calling but also when and how to rely on tool outputs (Liu et al., 6 Jun 2026).

A third tension is maintenance of the constructed artifact. UCT explicitly introduces offline memory consolidation because an unconstrained library of created tools would accumulate duplicates, rarely used items, and high-failure-rate assets; its long-term library organizes 207 tools into 7 major categories and 64 sub-categories, with high reported reuse rates (Shen et al., 2 Feb 2026). Rhetor likewise depends on rehearsal, locator repair, and graceful degradation to narration-only segments because UI drift makes unrehearsed scripted demos fragile (Khedar et al., 29 Jun 2026). ALD and HALD address the same maintenance problem at documentation scale by regenerating filtered tool outputs from the current toolchain rather than freezing stale screenshots or pasted traces (Ferreiro et al., 2023). A plausible implication is that future systems will treat demonstration construction less as a one-time data generation step than as a continual process of regeneration, consolidation, and audited reuse.

Across these strands, the field is converging on a common view: demonstrations are increasingly built as executable, stateful, validated objects. Whether the final artifact is a robot trajectory set, a multi-tool reasoning trace, a reusable tool package, a benchmark conversation, or a rehearsed product demo, construction quality depends on tool-grounded state representation, explicit validation, and careful control of abstraction, memory, and failure handling.