EmboCoach-Bench: Autonomous Robotic Policy Benchmark

• EmboCoach-Bench is a benchmark that evaluates LLM agents in automating the engineering of embodied robot policies using iterative, code-based closed-loop workflows.
• It integrates 32 tasks from imitation and reinforcement learning across high-fidelity simulators like ManiSkill, RoboTwin, RoboMimic, and MetaWorld to measure success rates.
• The framework employs a Draft–Debug–Improve paradigm with Monte Carlo Tree Search to optimize policy architectures and reward tuning, achieving significant performance gains.

EmboCoach-Bench is a benchmark designed to systematically evaluate the capability of LLM agents to autonomously engineer embodied robot policies. Established in response to scaling constraints imposed by labor-intensive manual engineering in embodied AI—spanning reward shaping, hyperparameter tuning, and cross-backend adaptation—EmboCoach-Bench targets the automated development and optimization of robotics solutions through dynamic, code-based closed-loop workflows. Its core contributions include a unified framework spanning 32 tasks in both Imitation Learning (IL) and Reinforcement Learning (RL), executable code as the universal agent interface, and robust assessment protocols for agent-driven synthesis and improvement cycles (Lei et al., 29 Jan 2026).

1. Benchmark Composition and Task Structure

EmboCoach-Bench is constructed atop four industry-standard, high-fidelity simulation platforms: ManiSkill, RoboTwin, RoboMimic, and MetaWorld. These environments were chosen for their strong physics fidelity and robust reproducibility under containerized infrastructure (Docker, Kubernetes).

Thirty-two tasks are curated to sample the full spectrum of embodied robotics challenges:

• Rigid-body manipulation (e.g., pick-cube, push-cube),
• Fine motor assembly (e.g., peg-insertion-side, tool-hang),
• Articulated interactions (e.g., drawer-open, door-open).

Each task is formalized as a tuple: $$\mathcal{T} = (\mathcal{D}_{prd}, \mathcal{P}_{sys}, \mathcal{C}_{env})$$ where:

• $\mathcal{D}_{prd}$: “Semantic Specification” (Product Requirements Document) encompassing task objectives, operational constraints (e.g., wall-clock limits), and domain scaffolding (architecture hints, hyperparameter ranges).
• $\mathcal{P}_{sys}$: “Operational Interface” – deterministic API schema and tool protocols—FileEditor, Terminal, TaskTracker—provided as context to the LLM agent.
• $\mathcal{C}_{env}$: “Development Substrate” – full codebase and simulator stack, supplied as Docker images and job templates.

RL task success is measured via a binary per-episode “success” indicator, aggregated as success rate (SR): $$\mathrm{SR} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}[\mathrm{success}_i]$$

2. Agentic Workflow and Policy Engineering

The agent workflow is a closed-loop “Draft–Debug–Improve” paradigm, wherein the LLM agent:

1. Drafts code modifications, guided by $\mathcal{D}_{prd}$.
2. Initiates a 10-episode dry-run in an Interactive Debug Pod via TerminalTool.
3. Analyzes Python tracebacks or preliminary SR outcomes.
4. Iteratively refines code—using FileEditorTool and TaskTrackerTool—until convergence or computational budget is depleted.

This iterative process is embedded within a Monte Carlo Tree Search (MCTS) framework. In this structure:

• Nodes correspond to specific codebase versions.
• Edges represent code patch proposals.
• Rollouts are conducted as debug-test cycles.

A single closed-loop iteration (simplified) is: $$\begin{aligned} s_0 &\leftarrow \text{initial code state}, \ \text{for }k=1,\dots,K:& \ \quad & \pi_k = \mathrm{LLM.generate}(s_{k-1}, \mathcal{D}_{prd}, \mathcal{P}_{sys}), \ \quad & s'_k = \mathrm{applyPatch}(s_{k-1}, \pi_k), \ \quad & e_k = \mathrm{debugTest}(s'_k), \ \quad & s_k = \mathrm{LLM.debug}(s'_k, e_k), \ \text{end} \end{aligned}$$

Agents optimize or synthesize policy architectures, including:

• Diffusion Policies (flow-matching U-Net with timestep embeddings, e.g., $\texttt{diffusion\_step\_embed\_dim}=256$, $\texttt{down\_dims}=[384,768,1536]$),
• Action-Chunking Transformers (ACT) for IL,
• Vision-Language-Action (VLA) models with cross-modal encoders,
• RNN and VAE-based policies (RoboMimic),
• MLP-based policies for RL tasks (ManiSkill, MetaWorld).

3. Reward Formulation and Automated Tuning

Reward design in EmboCoach-Bench is not fixed per task; LLM agents are evaluated on the ability to construct physics-informed dense rewards, generically expressed as: $$r_t = w_{d}\,\|\mathrm{pos}_{t} - \mathrm{pos}_{\mathrm{target}}\|_2 + w_{o}\,\mathrm{orient\_alignment}_{t} + w_{\mathrm{contact}}\,\mathbb{I}[\mathrm{stable\_contact}_t] - w_{\mathrm{collision}}\,\mathbb{I}[\mathrm{collision}_t]$$ with regularization: $-\lambda\|a_{t} - a_{t-1}\|$ Agents employ MCTS to tune key hyperparameters under hard resource constraints (e.g., 4-hour training limits, 1000 demonstration budget), adapting parameters such as:

• Learning rate ($5\times10^{-5}$),
• Max grad norm ($\max\_grad\_norm=1.0$),
• Horizon length (8→16),
• EMA power (0.75→0.85).

Agents demonstrate accelerated reward and hyperparameter optimization compared to human experts, automating procedures that traditionally rely on manual trial-and-error.

4. Evaluation Protocol and Empirical Results

Performance is benchmarked using per-task and mean aggregated success rates: $$\overline{\mathrm{SR}} = \frac{1}{32} \sum_{t=1}^{32} \mathrm{SR}_t$$

Two experimental conditions are defined:

• Improving setting (21 tasks): Initialization from a human baseline codebase.
• From-scratch setting (11 tasks): End-to-end pipeline synthesis.

Table excerpts:

Embodied Task	Human-SR	LLM-w/oAgentic	LLM-Agentic
Avg (21, improve)	0.47	0.59	0.80
Avg (11, scratch)	0.85	0.88	0.99

Agentic iteration provides an absolute +0.33 gain (improve) and closes the gap to near-perfect performance (from-scratch). State-of-the-art LLMs (Gemini 3.0 Pro, GPT-5.2, Claude Opus 4.5) exhibit substantial SR improvements, rising from ~40% (non-agentic) to ~80% (agentic MCTS). In “pathological recovery” scenarios—cases with near-total baseline failures—agents achieve “resurrection”: peg-insertion-side, baseline 0.00 → agentic 0.94; coffee-pull-st, baseline 0.15 → agentic 0.95.

5. Key Insights and Implications

Three principal findings are established:

1. Autonomous agents surpass expert-tuned baselines by 26.5% in average SR, demonstrating superhuman engineering capability under the formulated benchmarks.
2. The closed-loop “Draft–Debug–Improve” workflow, reinforced by real-time simulation feedback, is critical for robust policy development. This agentic mechanism reliably converts high-risk structural edits into consistent policy improvements and diminishes the disparity between open-source and proprietary LLM models.
3. LLM agents exhibit robust self-diagnosis and iterative repair; they can “resurrect” failed pipelines through repeated debug cycles, overcoming pathological engineering failures.

A plausible implication is the acceleration of the transition from artisanal, labor-intensive embodied system engineering to scalable, industrialized, and eventually self-evolving embodied intelligence. EmboCoach-Bench establishes a foundation for fully autonomous reward shaping, policy architecture search, and hyperparameter tuning in robotics, reducing reliance on manual expert intervention and advancing the automation frontier for embodied AI (Lei et al., 29 Jan 2026).