MoMaGen: Mobile Manipulation Data Generation

• MoMaGen is a demonstration synthesis framework that automates the production of diverse, high-quality datasets for multi-step bimanual mobile manipulation.
• It formulates trajectory generation as a constrained optimization problem by enforcing hard constraints (e.g., kinematics, collision avoidance) alongside soft constraints (e.g., visibility and smoothness).
• Experimental evaluations show that MoMaGen achieves superior data diversity and sim-to-real policy success compared to prior methods, enabling robust imitation learning with minimal real-world fine-tuning.

MoMaGen (Mobile Manipulation Data Generation) is a demonstration synthesis framework designed to automate the production of high-diversity, high-quality datasets for imitation learning in multi-step, bimanual, mobile manipulation tasks. Unlike prior systems limited to static or single-arm settings, MoMaGen formulates robot trajectory generation as a constrained optimization problem, enforcing both hard constraints—such as reachability and collision avoidance—and soft constraints—such as view-centric visibility and trajectory smoothness—across full robot kinematics, including mobile base, torso, and dual arms. The system is demonstrated to generate procedurally diversified demonstrations from a single human seed, supporting robust and generalizable imitation learning policies and sim-to-real transfer with minimal real-world fine-tuning (Li et al., 21 Oct 2025).

1. Problem Formulation and Constrained Optimization Framework

MoMaGen treats demonstration generation as a whole-body constrained optimization trajectory synthesis for robots equipped with a mobile base, torso, and two arms. The configuration at each timestep includes:

• Robot base pose $\mathbf{T}^{\mathrm{base}}_t \in SE(2)$
• Torso joint angles $$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$
• Left/right arm joint angles $$\mathbf{q}^{\mathrm{armL}}_t,\,\mathbf{q}^{\mathrm{armR}}_t \in \mathbb{R}^7$$
• Camera head pose $\mathbf{T}^{\mathrm{cam}}_t \in SE(3)$
• Contact-rich end-effector trajectories for each subtask, $\tau_i = \{\mathbf{T}_W^{E_k}\}_{k=0}^{K_i}$, fixed via object-centric transforms

MoMaGen enforces several hard constraints:

1. System dynamics: $s_{t+1} = f(s_t, a_t)$
2. Kinematic feasibility: $\mathcal{G}_{\mathrm{kin}}(\cdot) \le 0$ (joint limits, reachability)
3. Collision avoidance: $\mathcal{G}_{\mathrm{coll}}(\cdot) \ge 0$
4. Manipulation reachability via inverse kinematics: existence of joint configuration for all end-effector poses
5. Task-specific success predicates achieved within the trajectory
6. Head camera visibility: object within field-of-view and minimally occluded during manipulation

Soft constraints supplement the planning objective via weighted penalties:

• Visibility during navigation (minimize angular deviation between camera axis and target)
• Trajectory brevity and smoothness (penalize length and jerk)
• Retraction to a “tucked” pose after manipulation

The overall trajectory synthesis is posed as:

$$\min_{\{a_t\}_{t=0}^{T-1}} \mathcal{L} = \sum_j w_j \ell_j \ \text{subject to hard constraints above.}$$

2. Trajectory Synthesis Methodology

MoMaGen employs a subtask-wise, hybrid planning procedure rather than full nonlinear programming. Key steps:

• Seed is one annotated demonstration, segmented into navigation and manipulation subtasks.
• For free-space subtasks, it samples candidate base and camera poses near randomized object position, verifies hard constraints (IK feasibility, collision, visibility), then plans using GPU-accelerated cuRobo with soft-cost augmentation for visibility.
• For contact-rich subtasks, end-effector trajectories are transformed from the source via object pose, with reachability and visibility checks; control is performed in task space.
• After manipulations, arms and torso are retracted via short joint-space plans.

Key runtime parameters:

• Base-pose sampling radius is task-adaptive ($\pm 0.5$ m to $$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$0 m).
• Up to 50 base poses × 10 camera poses per subtask.
• Per-demo generation time is 0.1–1.3 GPU hours depending on task complexity.
• Hard constraint violations are handled by outright rejection of candidates; soft constraint terms shape the planner’s selection.

3. Dataset Generation and Diversity Metrics

Starting from a single (1–3 min) annotated human demonstration, the system supports progressive scene perturbation:

• D0: Small randomization around source object ($$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$1 cm, $$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$2)
• D1: Arbitrary placement on the same furniture (full 2D placement, rotation)
• D2: D1 plus random distractor/obstacle addition

Object, base, and camera poses are sampled to maximize diversity while retaining feasibility. Diversity is quantified by:

• Object-pose spread (convex hull volume in pose space)
• Base-pose variance in $$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$3 dimensions
• End-effector workspace coverage
• Joint-configuration diversity (PCA of full-trajectory joint angle space)

This structure allows systematic scaling of dataset diversity for robust learning.

4. Experimental Evaluation

Experiments utilize four multi-step bimanual mobile manipulation tasks in simulation (OmniGibson):

• Pick Cup (navigation + single-arm manipulation)
• Tidy Table (extended navigation and manipulation)
• Put Dishes Away (bimanual, stacking)
• Clean Frying Pan (bimanual, contact-rich)

Comparative baselines: SkillMimicGen (single-arm, base replay) and DexMimicGen (bimanual, base replay).

Key findings:

Metric	MoMaGen (D0)	Baselines (D0)	MoMaGen (D1)	Baselines (D1)
Data Gen. Success (Pick Cup)	0.86	1.00	0.60	0.00
Data Gen. Success (Tidy Tbl)	0.80	0.69/0.72	0.64	0.00
Visibility Ratio	75–95%	30–60% (ablate)	>50%	n/a
Policy Success (D1)	0.25 (WB-VIMA, Pick Cup)	0.00	0.60 ($$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$4)	0.00

Notably, under high-randomization domains (D1, D2), only MoMaGen achieves generation and imitation learning policy success. Ablative removal of visibility constraints incurs up to 70% relative drops in policy performance.

Scaling policies (Vision–Language–Action model $$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$5) with more synthetic demos (500/1,000/2,000) yields monotonic performance improvements, especially in complex domains.

Sim-to-real transfer experiments on the Galaxea R1 robot show that synthetic pretraining via MoMaGen enables considerable real-world policy success (60% for $$\mathbf{q}^{\mathrm{torso}}_t \in \mathbb{R}^{d_{\mathrm{torso}}}$$6 with synthetic pretrain vs. 0% for real-only), even with just 40 real demos for fine-tuning.

5. Advantages Over Prior Approaches

MoMaGen subsumes prior X-Gen demonstration expansion frameworks by providing:

• A unified constrained-optimization formalism integrating base, arm, torso, and camera variables
• Explicit handling of reachability hard constraints for mobile base placement and manipulation
• Hard and soft visibility constraints for generating useful visuomotor data
• GPU-parallelized planning generators for tractable and diverse whole-body trajectory synthesis
• Demonstrated capability to support robust, generalizable imitation learning and effective sim-to-real policy transfer

Compared to SkillMimicGen and DexMimicGen, MoMaGen exhibits superior diversity in data distributions and achieves policy generalization across greater workspace and environmental variations (Li et al., 21 Oct 2025).

6. Limitations and Future Directions

MoMaGen’s deployment requires full scene state knowledge, which is practical in simulation but difficult in the real world without robust object pose estimation. It is optimized for sequential navigation and manipulation phases; truly continuous whole-body, contact-rich interaction (e.g., doors, articulated objects) remains an open area. Cross-embodiment transfer is demonstrated in proof-of-concept but is challenged by simulation-to-reality gaps in confined spaces and collisions.

Computational cost is nontrivial (up to 1.3 GPU hours per complex demo); efficiency improvements and the use of precomputed subtrajectories are suggested future directions. Extension to more general embodied tasks and integration with strong vision models (e.g., segmentation models for scene parsing) is anticipated to broaden applicability and realism.

7. Broader Implications

MoMaGen enables scalable, principled demonstration synthesis for complex, high-degree-of-freedom robotic platforms. The generality of its constrained-optimization viewpoint provides a modular template for further advances in automated data generation, robot learning, and policy transfer. A plausible implication is that, as scene parsing technologies mature, MoMaGen-like frameworks could be directly used in physical environments for autonomous data collection, further accelerating progress in real-world robot learning (Li et al., 21 Oct 2025).