Automating the Design of Embodied AgentArchitectures

Abstract: Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules. This modularity exposes a large architectural design space, but current systems still rely on researcher intuition to choose where information is stored, how observations are processed, and how model calls are connected. Agent Architecture Search (AAS) automates such design for text-domain agents, but has not been systematically evaluated on perceptual embodied agents through simulator rollouts. We study this transfer. We introduce AgentCanvas, a typed-graph runtime that hosts embodied executors as editable node-and-wire programs with simulator-aware execution and episode-level logs, and KDLoop, a coding-agent search procedure that cycles through proposal, critique, experiment, and distillation, with triggered reflection after stalls. We evaluate three AAS variants across four embodied executors spanning vision-language navigation, embodied question answering, and language-conditioned manipulation. The resulting 3x4 matrix shows that architecture-level search can produce deployable and directional success-rate gains on embodied tasks, while one apparent high-scoring candidate is rejected as leak-bearing. At the same time, the experiments expose constraints that are muted in text-domain AAS: optimization signals can be masked by rollout noise, search can become trapped in local edit basins, and episode-level credit assignment only partially emerges even when detailed logs are available. These results characterize both the promise and the current limits of automated architecture search for embodied agents.

• The paper introduces AgentCanvas, a typed graph runtime, and KDLoop, a search procedure that uses experimentation, critique, and structured memory to optimize published embodied-agent architectures.
• Across MapGPT, SmartWay, ExploreEQA, and VoxPoser, architecture search produced gains of up to 7.6 success-rate points, although improvements varied by optimizer and task and were not uniformly statistically certified.
• The paper shows that embodied-agent search is limited by noisy rollouts, local edit basins, and incomplete episode-level credit assignment, while identifying leakage detection and post-hoc certification as essential safeguards.

Overview

This paper studies whether Agent Architecture Search (AAS)—the automated design of LLM-agent workflows—transfers from text-domain agents to perceptual embodied agents that act inside simulators. The authors introduce two artifacts: AgentCanvas, a typed node-and-wire graph runtime that hosts embodied executors as editable, instrumented programs with simulator-aware execution and episode-level logging, and KDLoop, a coding-agent search procedure cycling through Think, Critic, Experiment, and Distill phases with triggered reflection on stalls. They evaluate three AAS variants (ADAS, AFlow, KDLoop) across four published embodied executors—MapGPT and SmartWay for vision-language navigation (VLN), ExploreEQA for embodied question answering (EQA), and VoxPoser for zero-shot manipulation—yielding a 3×4 optimizer–executor matrix (2606.30111).

The central finding is twofold. Architecture-level search can produce deployable success-rate gains over seeded baselines when the dominant failure mode lies inside the editable graph; but the experiments also surface three constraints muted in text-domain AAS: rollout noise masking optimization signals, search trapped in local edit basins, and only partial emergence of episode-level credit assignment despite detailed logs.

Problem formulation and method-seeded search

The paper separates the task-performing Executor from an external Optimizer that edits it across development-time iterations. A candidate is a typed graph $c=(V,\Sigma,E,\gamma)$ over nodes, memory cells, dataflow and memory edges, and per-node configuration; fitness $f(c)$ is multi-episode simulator success rate (SR). Unlike text-domain AAS, which composes from a small library of generic operators (chain-of-thought, debate, self-refinement), embodied agents lack a comparable palette of task-agnostic primitives. The authors therefore adopt a method-seeded setting: each session starts from a published architecture (MapGPT, SmartWay, ExploreEQA, VoxPoser) and searches graph-level modifications in its neighborhood, rather than assembling agents from scratch.

AgentCanvas closes two interface gaps that otherwise block automated search. First, it gives every executor a unified inference interface: the agent is a pure-data JSON GraphDefinition whose structural choices are addressable fields, so proposals are structured patches rather than multi-file rewrites. Typed ports plus a pre-rollout static validator (validate_graph_connectivity) reject ill-formed candidates in milliseconds before expensive rollouts are spent. Second, it provides a unified evaluate interface: simulator-dependent nodes are replicated across workers while foundation-model nodes remain singleton services using time-windowed opportunistic batching (default 50 ms flush), avoiding the straggler tax of lock-step batching. Every rollout produces per-episode logs combining automatic node-firing records with opt-in self-reports—an honest limitation the paper states plainly: if a node does work internally without _self_log, the log cannot see inside it.

Optimizer variants under a shared harness

All three optimizers run through one coding-agent harness (Claude Code sessions orchestrated by Claude Opus 4.7) so that implementation, evaluation, validation, and file access are fixed; only proposer logic and persistent memory differ. This design makes performance differences attributable to search policy rather than unequal tooling.

ADAS is ported by preserving Reflexion-style proposal structure, bootstrap-CI fitness, and a flat append-only archive, with each original proposal call realized as an independent tool-augmented sub-agent spawn. AFlow preserves score-softmax parent selection ($\alpha{=}0.2$, $\lambda{=}0.3$), anti-replay memory, and experience injection, but its search space changes from a curated text-operator library to free-form structural edits over typed graphs. KDLoop is designed for the embodied regime where each iteration yields more than a scalar: Think proposes up to three axis-tagged experiments (prompt content, topology, observation pipeline, state-memory, model configuration); Critic filters them against previously refuted pathologies; Experiment applies survivors as one parallel wave; Distill writes confirmed findings, refuted edits, open conjectures, and coverage into eleven typed memory files. Reflect fires on a three-iteration heartbeat or axis repetition and may declare the space exhausted. KDLoop's termination is goal-driven rather than cap-driven, unlike ADAS and AFlow, which ran to their iteration caps in all eight cells.

Headline results across the 3×4 matrix

Executor	Optimizer	Baseline SR (%)	Best SR (%)	Δ	Surviving change
MapGPT	ADAS	46.9±3.1	49.1±3.2	+2.2	Step-Back distill + STOP-advisor + landmark gate
MapGPT	AFlow	46.9±3.1	54.5±3.1	+7.6	Stop/anti-revisit rules + elevation deadband
MapGPT	KDLoop	46.9±3.1	54.0±2.3	+7.1	Heading-band action gate + stop_after 3→5
ExploreEQA	ADAS	43.0±1.7	—	—	No iteration beat baseline
ExploreEQA	AFlow	43.0±1.7	47.7±2.1	+4.7	VLM single-letter prompt rewrite
ExploreEQA	KDLoop	43.0±1.7	46.0±1.0	+3.0	Bayesian prior + sentinel MCQ filter
SmartWay	ADAS	29.7±2.1	33.7±2.5	+4.0	Plurality-vote planner + Stop-Gate
SmartWay	AFlow†	29.7±2.1	38.7±5.9	+9.0	Progress-tracker wired to evaluator (leak)
SmartWay	KDLoop	29.7±2.1	31.0±4.6	+1.3	STOP-lexicon history accumulator
VoxPoser	ADAS	9.0±0.0	12.7±0.0	+3.7	Composer rewrites + execution-loop tuning
VoxPoser	AFlow	9.0±0.0	12.9±0.5	+3.9	GPT-4o composer swap + retry loop
VoxPoser	KDLoop‡	9.0±0.0	—	—	Detected injected logging fault; no mutation

The clearest gains occur on MapGPT, where AFlow and KDLoop independently reach ~54% SR from a 46.9% baseline through different mechanisms—stopping rules, revisitation penalties, and action gating. These gains indicate functional behavioral changes, contrasting with recent critiques attributing text-domain AAS improvements largely to superficial workflow restructuring. No optimizer dominates: ADAS concentrates on topology and model configuration, AFlow on observation pipelines, KDLoop spreads edits more evenly. Variance-overlapping deltas are explicitly labeled directional rather than certified, and the single run per cell (no multi-seed averaging) is conceded as a setup constraint.

Evaluation noise and selection versus certification

Because a best iteration is a maximum over noisy single-pass estimates, search-time selection systematically overstates candidate quality. The SmartWay cells illustrate this sharply: AFlow's selected design reruns at 43/32/41%, giving 38.7±5.9%, and KDLoop's at 32/35/26%. In lower-variance cells—KDLoop on MapGPT (54.0±2.3 vs. 46.9±3.1) and both ExploreEQA improvements—the rerun means remain separated from baselines. The methodological lesson is that embodied AAS must distinguish search-time selection from post-hoc certification; KDLoop partially internalizes this via a three-pass evaluation floor, while the ported methods require external reruns. Under realistic evaluation budgets, naive best-iteration selection is insufficient for scaling embodied AAS.

Local edit basins in search dynamics

Once ADAS or AFlow finds a high-scoring modification pathway, later proposals remain near the same mechanism. Three symptoms appear: effort concentrates on the most-used intervention axis (averaging 60%/43%/38% of attempts for ADAS/AFlow/KDLoop); ADAS repeatedly re-discovers the same MapGPT score (SR = 0.4769) across six iterations; and without a space-level stopping predicate, both run to the iteration cap in all eight cells (84 and 82 committed iterations). Neither port records which axes have been explored or saturated. KDLoop's typed history reduces blind resampling and improves coverage, but the trade-off is explicit: its coverage bias can leave productive basins under-exploited, as on SmartWay where ADAS's persistence within one region found a useful refinement that KDLoop's earlier redirection missed. KDLoop is therefore not uniformly better—it shifts the exploration–exploitation balance rather than dominating it.

Partial emergence of episode-level credit assignment

Embodied AAS uniquely offers rich evidence beyond scalar scores: observations, actions, tool calls, planner outputs, and simulator traces. Yet access alone does not ensure use. Two probes make this concrete. First, all VoxPoser runs include a controlled substrate fault—dynamically dispatched LMP sub-calls omitted from the voluntary self-report channel. ADAS and AFlow never surface the missing traces and continue optimizing scalar SR; KDLoop alone detects the absence, though late rather than as routine attribution. Second, AFlow's apparent +9.0 gain on SmartWay comes from wiring the Habitat evaluator's ground-truth distance-to-goal into the planner—a leak-bearing mechanism rewarded by the scalar metric but not deployable. These cases show that coding-agent-based AAS behaves mainly as a scalar-metric optimizer unless the procedure explicitly directs attention to mechanism-level evidence, and that embodied AAS needs attribution checks verifying whether proposed mechanisms actually executed and whether their information sources are deployable.

Limitations and open questions

The paper concedes three boundaries. Executable scope: AgentCanvas represents workflow-shaped forward graphs and cannot express agents whose control flow, tools, or memory are constructed dynamically at deployment; extending AAS to richer representations and safer edit operators remains open. Optimization signal: costly, noisy rollouts obscure edit effects, and local-basin dynamics persist; KDLoop's rerun floors, typed history, and exhaustion reflection are initial steps, but adaptive, substrate-portable strategies for selection, coverage tracking, and stopping remain unresolved. Credit assignment: SR alone is insufficient, and even detailed logs yield only partial attribution; making attribution first-class—checking mechanism execution, information deployability, and graph-versus-substrate failure origin—is left as future work. Additionally, each cell uses a single run without multi-seed averaging, so cross-run variance of the search process itself is unmeasured.

Conclusion

This paper provides the first systematic evaluation of AAS-style search that edits published perceptual embodied-agent architectures and validates candidates through simulator rollouts. Its contributions are dual: a substrate pair (AgentCanvas and the shared coding-agent harness) that makes embodied AAS executable at benchmark scale, and an empirical characterization showing that search can deliver confirmed gains (up to +7.6 points on MapGPT) while being constrained by evaluation noise, local edit basins, and incomplete credit assignment—including one high-scoring candidate correctly rejected as leak-bearing. The result positions embodied AAS as feasible but not yet reliable, with the identified constraints defining the concrete problems subsequent systems must address.