---
title: 'FetchMan-Bench: Humanoid Loco-Manipulation Benchmark'
url: https://www.emergentmind.com/topics/fetchman-bench
type: topic
---

# FetchMan-Bench: Humanoid Loco-Manipulation Benchmark

FetchMan-Bench is a reproducible simulation benchmark for visual humanoid loco-manipulation introduced in “FetchMan: Learning Visual Humanoid Loco-Manipulation Policies from Simulated Experiences” [2608.17027]. It evaluates policies that must navigate toward a target, coordinate whole-body reaching, grasp and lift an object, and transition between locomotion and manipulation. The benchmark uses a Unitree G1 humanoid with Dex1-1 grippers, procedurally generated indoor scenes from MolmoSpaces, visual-proprioceptive observations, a 15-dimensional action interface, and binary success-rate metrics evaluated on 100 held-out initializations.

## 1. Purpose and task scope

FetchMan-Bench addresses the gap between conventional humanoid locomotion benchmarks and fixed-base manipulation benchmarks. Locomotion benchmarks generally emphasize low-dimensional observations and foot-floor interaction, whereas fixed-base manipulation benchmarks do not require walking, balance, or coordination between navigation and grasping. FetchMan-Bench instead evaluates the integrated sequence of locomotion, approach, whole-body reaching, grasping, lifting, and transition from navigation to manipulation [2608.17027].

The benchmark supports two related task settings:

1. **Manipulation-only**, also called pick-only: the humanoid is initialized at a grasp standoff from the target and must execute the terminal sequence

   $$
   \text{reach} \rightarrow \text{descend} \rightarrow \text{close gripper} \rightarrow \text{lift}.
   $$

   This setting isolates visual reaching, torso and arm control, grasping, and lifting.

2. **Full loco-manipulation**, also called fetch: the humanoid starts away from the target, navigates through the scene, approaches the object, stops at an appropriate grasp standoff, reaches, grasps, and lifts it.

The full task tests the locomotion-to-manipulation handoff. The paper identifies this transition as a central difficulty because the scripted demonstrator uses a privileged discrete phase variable that is unavailable to the learned policy. The benchmark therefore evaluates not only terminal manipulation but also the ability to infer and execute the appropriate behavioral phase from visual-proprioceptive observations.

The policy-training mixture contains 80% fetch episodes and 20% pick-only episodes. Fetch episodes exercise navigation and grasping; pick-only episodes expose the policy to varied initial upper-body configurations and gripper apertures.

## 2. Procedural scenes, objects, and evaluation splits

### Scene generation

FetchMan-Bench uses MolmoSpaces, an ecosystem of procedurally generated indoor environments with physically simulated objects. Scene sources are curated from ProcTHOR-10k, Holodeck, and ProcTHOR-Objaverse. A task is represented as

$$
(s,m,x_0),
$$

where $s$ is the scene, $m$ is the target object, and $x_0$ is the humanoid’s initial pose.

The overall data-generation pipeline spans approximately 150,000 distinct procedurally generated scenes. This figure refers to the training and data-generation experience rather than to the number of benchmark test episodes. Benchmark evaluation uses a fixed set of 100 held-out initializations.

Scene variation includes:

- new procedural scene layouts;
- randomized object placements;
- randomized support-surface heights;
- lighting and texture changes;
- camera perturbations;
- action noise;
- novel initial upper-body states in pick-only episodes.

The benchmark’s procedural construction is intended to evaluate generalization beyond fixed, hand-authored scenes.

### Object and grasp generation

The principal single-object experiments use bowl targets and approximately 150,000 bowl-pick demonstrations. For the multi-object extension, the sampler draws from all pickable object categories marked by MolmoSpaces and conditions the model on the target object’s name through a dino.txt-based vision-language backbone.

Each pickable object has a precomputed grasp set $\mathcal{G}_m$. Grasps are generated using antipodal sampling and filtered for:

- Dex1-1 gripper compatibility;
- inverse-kinematic feasibility;
- self-collision avoidance.

Object pose and support-surface height are randomized per episode. The target’s planar pose is jittered, and support-surface changes are propagated to resting objects through contact-graph traversal.

### Humanoid initialization

For fetch episodes, candidate base positions are sampled from free cells of a two-dimensional occupancy grid $\mathcal{O}_s$ within an annulus around the target:

$$
r_{\min} \leq \|p-p_m\|_2 \leq r_{\max}.
$$

The initial yaw is approximately aligned with the target bearing, with random offset

$$
\Delta \psi \sim \mathcal{U}(-0.5,0.5)\ \text{rad}.
$$

A candidate is accepted only when:

- the full humanoid pose is contact-free;
- the target is visible in the head fisheye camera;
- the simulator validates the pose as feasible.

Validation is performed in the full three-dimensional simulator rather than solely through a conservative two-dimensional footprint test. This permits upper-body reach over tables and other surfaces while rejecting invalid base contacts.

### Held-out evaluation

Training data are generated from approximately 150,000 distinct procedurally generated scenes. Model selection uses the best validation checkpoint, and benchmark evaluation uses 100 held-out initializations. The paper does not specify a conventional named train/validation/test partition with separate numerical scene counts; the 100 held-out initializations are described as the validation/evaluation set.

The scaling experiment uses 5,000, 50,000, and 150,000 demonstrations. Behavior-cloning performance increases from 40% to 67% loco-manipulation success between 5,000 and 50,000 demonstrations, then remains at 67% with 150,000 demonstrations.

## 3. Embodiment, observations, and control

### Robot and control stack

The benchmark uses a Unitree G1 humanoid with Dex1-1 grippers. The policy does not directly output joint torques. Instead, it produces a 15-dimensional action at 10 Hz:

$$
a_t=(v_t,\bar h_t,\bar q^{\text{waist}}_t,\bar q^{\text{arm}}_t,\bar g_t),
$$

where:

- $v_t=(v_x,v_y,\omega)\in\mathbb{R}^3$ is the commanded planar base velocity and yaw velocity;
- $\bar h_t\in\mathbb{R}$ is the base-height target;
- $\bar q^{\text{waist}}_t\in\mathbb{R}^3$ is the waist target;
- $\bar q^{\text{arm}}_t\in\mathbb{R}^7$ is the right-arm joint target;
- $\bar g_t\in\mathbb{R}$ is the right-gripper target.

The lower-body command $(v_t,\bar h_t)$ is tracked by the pretrained SONIC controller at 50 Hz. SONIC contains decoupled balance and walking behaviors selected by a hard threshold on commanded base velocity. Waist, arm, and gripper targets are tracked through PD control. Resulting torques are applied at a 200-Hz physics rate. The same control stack is used in simulation and on the physical G1.

This hierarchical interface reduces the learned control problem from approximately 29 whole-body torque dimensions to 15 command dimensions. However, the lower-body controller is fixed: the policy cannot learn new gaits, brace against heavy objects, widen its stance for additional reach, or recover from disturbances outside SONIC’s operating envelope.

### Visual-proprioceptive observation

The policy receives a single observation with no temporal history:

$$
o_t=\left(I_t^{\text{head}},I_t^{\text{wrist}},o_t^{\text{prop}}\right).
$$

The head and wrist images have dimensions

$$
I_t^{\text{head}},I_t^{\text{wrist}}\in\mathbb{R}^{224\times384\times3}.
$$

They consist of:

- a head-mounted fisheye RGB image;
- a wrist-mounted RGB image.

The proprioceptive state is

$$
o_t^{\text{prop}}=(h_t,r_t,p_t,q_t^{\text{upper}},g_t),
$$

where $h_t$ is base height, $r_t$ is base roll, $p_t$ is base pitch, $q_t^{\text{upper}}$ contains upper-body joint positions, and $g_t$ is gripper aperture.

The policy does not receive:

- the simulator pose $T_m$ of the object;
- the selected grasp $T_g$;
- the occupancy grid $\mathcal{O}_s$;
- privileged navigation state;
- the demonstrator’s phase index $k_t$.

The reference architecture uses a frozen DINOv3 ViT-B/16 visual encoder. Image patch tokens from both cameras are combined with a projected proprioceptive state token, and a flow-matching DiT action head predicts an action chunk.

### Action chunking

The policy predicts an action chunk

$$
a_{t:t+H},
$$

with $H=16$. At deployment, only the first eight actions are executed before a new chunk is generated from the latest observation.

The 11 dimensions corresponding to absolute joint or pose targets—base height, waist, and arm targets—are represented as deltas relative to an observation-dependent per-chunk anchor. Base velocity and gripper commands remain absolute. This delta-action parameterization improves transfer compared with absolute action targets.

## 4. Demonstrations, reinforcement learning, and baselines

### Scripted demonstrations

The privileged demonstrator uses the discrete phase variable

$$
k_t\in\{\text{reach},\text{descend},\text{close},\text{lift}\}.
$$

During navigation, it uses A* planning over an inflated occupancy grid, reduces the path to straight-line waypoints, and commands the base while facing the target. At the grasp standoff, it progresses through manipulation phases when the end effector reaches the current phase target.

Approximately 150,000 bowl-pick demonstrations correspond to roughly 650 hours of robot experience. Data generation runs at approximately 100 episodes per minute on a single NVIDIA L40S GPU and requires roughly 40 GPU-hours. The demonstrations are generated from approximately 150,000 distinct procedurally generated scenes.

The primary behavior-cloning model is FetchMan-BC. Its plateau at 67% full-task simulation success, despite increasing the dataset from 50,000 to 150,000 demonstrations, is interpreted as an imitation ceiling associated with the demonstrator’s privileged phase information.

### Flow-GRPO refinement

The final FetchMan policy begins from the 150,000-demonstration behavior-cloning checkpoint and is refined with Flow-GRPO using only sparse terminal returns:

$$
R_i\in\{0,1\}.
$$

For each task, $M=8$ rollouts are generated from the same simulator state. Within a group, the standardized relative advantage is

$$
A_i=\frac{R_i-\mu}{\sigma},
$$

where $\mu$ and $\sigma$ are the group mean and standard deviation. Groups consisting entirely of successful or entirely unsuccessful rollouts are discarded because they provide no within-group relative signal.

The flow policy is made stochastic by replacing deterministic Euler integration with

$$
x_{\tau+\Delta\tau}\sim\mathcal{N}\left(
x_\tau+\hat v_\theta(x_\tau,\tau,o_t)\Delta\tau,\,
\sigma_\tau^2|\Delta\tau|
\right),
$$

where

$$
\sigma_\tau=\eta\sqrt{\frac{1-\tau}{\tau}}.
$$

This produces a closed-form Gaussian transition likelihood for importance-ratio optimization over flow-integration steps. The actor uses a PPO-style clipped objective with dual clipping for large negative-advantage samples and an optional KL penalty to the frozen behavior-cloning policy. The return-weighted actor loss is length-normalized:

$$
\mathcal{L}_i\leftarrow\frac{\bar L}{L_i}\mathcal{L}_i,
$$

where $L_i$ is the episode length and $\bar L$ is the batch mean episode length.

For the single-object policy, the batch configuration is $M=8$ and $G=32$, giving 256 episodes per update, with KL coefficient $\beta=0$. For the multi-object policy, $M=8$ and $G=64$, giving 512 episodes per update, with $\beta=0.01$.

Flow-GRPO applied to the 5,000-demonstration checkpoint increases full-task success only from approximately 40% to 42% and oscillates without a clear upward trend. This behavior is attributed to mostly all-failure groups being discarded during mixed-group filtering.

### Evaluated baseline families

FetchMan-Bench supports comparison among:

- behavior cloning from scripted demonstrations;
- Flow-GRPO refinement;
- alternative visual encoders;
- absolute-action and delta-action parameterizations;
- multi-object, text-conditioned policies.

The principal reported models are FetchMan-BC and FetchMan (BC+RL). The paper also evaluates a multi-object behavior-cloning model and its Flow-GRPO refinement.

## 5. Metrics and benchmark protocol

### Binary success metrics

FetchMan-Bench uses sparse terminal success rather than partial-credit metrics. For episode $i$,

$$
R_i=
\begin{cases}
1,&\text{if the target is successfully fetched and lifted},\\
0,&\text{otherwise}.
\end{cases}
$$

Manipulation success rate is

$$
\mathrm{SR}_{\mathrm{manip}}
=
\frac{1}{N}
\sum_{i=1}^{N}
\mathbf{1}\{\text{successful grasp and lift in manipulation-only task }i\}.
$$

Loco-manipulation success rate is

$$
\mathrm{SR}_{\mathrm{loco-manip}}
=
\frac{1}{N}
\sum_{i=1}^{N}
\mathbf{1}\{\text{successful navigation, grasp, and lift in task }i\}.
$$

Simulation results are evaluated on $N=100$ held-out initializations and reported with binomial standard errors. The benchmark does not report path length, energy, time to grasp, collision count, or grasp-pose error as official metrics.

### Quantitative results

The main simulation results are:

| Method | Manipulation SR | Loco-manipulation SR |
|---|---:|---:|
| FetchMan-BC | $75.0\pm4.3\%$ | $67.0\pm4.7\%$ |
| FetchMan (BC+RL) | $\mathbf{79.0\pm4.1\%}$ | $\mathbf{83.0\pm3.8\%}$ |

Flow-GRPO produces the largest improvement in the complete task, increasing loco-manipulation success from 67% to 83%. Manipulation-only success increases from 75% to 79%. These results support the interpretation that reinforcement learning primarily repairs the locomotion-to-manipulation handoff rather than the terminal grasp behavior.

The architecture ablations are:

| Variant | Sim manipulation | Sim loco-manipulation | Real manipulation | Real loco-manipulation |
|---|---:|---:|---:|---:|
| DINOv3 + delta actions | $75.0\pm4.3\%$ | $67.0\pm4.7\%$ | $72.7\pm9.5\%$ | $56.7\pm9.0\%$ |
| SigLIP | $58.0\pm4.9\%$ | $42.0\pm4.9\%$ | $8.3\pm8.0\%$ | $0.0\%$ |
| Absolute actions | $62.0\pm4.9\%$ | $45.0\pm5.0\%$ | $16.7\pm10.8\%$ | $0.0\%$ |

The ablations indicate that frozen DINOv3 visual features and delta action targets are important for sim-to-real transfer.

### Multi-object extension

The multi-object extension regenerates data across all pickable object categories and uses approximately 350,000 demonstrations. The object name is encoded using a frozen dino.txt vision-language backbone. Its reported simulation performance is approximately 40% loco-manipulation success for behavior cloning and approximately 62% after BC+Flow-GRPO refinement.

These results are below the single-object 83% result because the object distribution is harder and the multi-object behavior-cloning prior is weaker. The paper reports qualitative real-robot multi-object fetches but no numerical real-world success rate for that extension.

## 6. Findings, limitations, and physical transfer

### Principal findings

FetchMan-Bench shows that visual humanoid fetching remains difficult even with large-scale synthetic data. Behavior cloning improves with additional demonstrations but reaches a performance plateau. Flow-GRPO improves the full loco-manipulation task substantially when initialized from a large behavior-cloning dataset.

The benchmark’s principal failure mode is the transition from locomotion to manipulation. The learned policy must infer when to stop walking, adjust torso height, reach, close the gripper, and lift without access to the demonstrator’s phase variable or privileged object and grasp poses.

The benchmark also exposes the importance of action parameterization and visual representation. Replacing DINOv3 with SigLIP or replacing delta targets with absolute targets substantially degrades both simulation and physical performance.

### Limitations

The benchmark has several stated limitations:

- **Stateless observations**: the policy receives a single observation without temporal history and must infer phase from instantaneous visual-proprioceptive information.
- **Fixed lower-body controller**: SONIC is not adapted or jointly learned.
- **Narrow task family**: the benchmark focuses primarily on reach-and-pick behavior and does not yet include multi-object sequential tasks, articulated-object manipulation, door opening, placement, long-horizon household activities, or general object rearrangement.
- **Perfect segmentation assumptions**: the benchmark assumes perfect target and robot segmentation.
- **Simulation-to-real gap**: simulated contact physics and object dynamics may not accurately predict real performance.
- **Limited embodiment**: experiments use a single Franka-like upper-body configuration on the Unitree G1 humanoid with Dex1-1 grippers.
- **Filtered feasibility**: invalid paths, infeasible grasps, and phase-timeout demonstrations may be discarded during generation.
- **Incomplete metric scope**: no official metrics are reported for energy, collision count, grasp-pose error, recovery behavior, or time to grasp.

MolmoSpaces supports articulated-object manipulation, and the authors identify articulated objects and longer-horizon compositions as future directions.

### Real-robot transfer

FetchMan-Bench simulation results are distinct from physical evaluation. The best simulation checkpoint is deployed zero-shot on a real Unitree G1 without real-world training data, per-scene tuning, or real-world reinforcement learning.

| Method | Manipulation SR | Loco-manipulation SR |
|---|---:|---:|
| FetchMan-BC | $72.7\pm9.5\%$ | $56.7\pm9.0\%$ |
| FetchMan (BC+RL) | $\mathbf{77.2\pm8.9\%}$ | $\mathbf{73.3\pm8.1\%}$ |

The physical evaluation uses 22 manipulation trials and 30 loco-manipulation trials. FetchMan (BC+RL) achieves 73.3% zero-shot loco-manipulation success on the Unitree G1. This result is separate from the FetchMan-Bench simulation score.

## 7. Release and research significance

FetchMan-Bench is released as a reproducible simulation benchmark with fixed held-out scenes and released scoring. The project website given in the paper is:

```text
https://www.orayyan.com/fetchman
```

The paper does not provide a complete inventory of repository files, exact download URLs, license terms, or a file-by-file specification of the released assets. It also does not explicitly confirm that all raw demonstrations, full simulator assets, pretrained checkpoints, or the complete training pipeline are publicly downloadable.

FetchMan-Bench differs conceptually from benchmarks focused on isolated grasp selection, fixed-base manipulation, or motion planning alone. “FetchBench: A Simulation Benchmark for Robot Fetching” [2406.11793] evaluates complete fetching in procedural scenes, including approaching, grasping, and retrieving objects from clutter; FetchMan-Bench instead targets visual humanoid loco-manipulation with walking, balance, whole-body reaching, and a locomotion-to-manipulation transition. MotionBenchMaker [2112.06402] provides a related procedural dataset-generation and planner-benchmarking framework that explicitly supports the Fetch robot, but it evaluates single start-to-goal motion-planning requests rather than the Unitree G1’s visual humanoid fetching task.

The principal methodological contribution of FetchMan-Bench is the explicit evaluation of locomotion and manipulation as a coupled visual-control problem. Its benchmark design combines procedurally generated indoor scenes, held-out initializations, unprivileged visual-proprioceptive observations, sparse terminal rewards, hierarchical humanoid control, and separate manipulation and loco-manipulation success rates. The reported results indicate that large-scale imitation is useful but insufficient, while reinforcement-learning refinement can overcome part of the performance ceiling associated with privileged scripted demonstrations.

Source: https://www.emergentmind.com/topics/fetchman-bench