GenMimic: Scalable Robotic Imitation

• GenMimic is a suite of automated systems for generating demonstration data and training policies to achieve high-fidelity robotic imitation from minimal human input.
• It employs object-centric segmentation, subtask retargeting via SE(3) mapping and DMPs, and dynamic adaptation for versatile applications across diverse robotic platforms.
• GenMimic frameworks demonstrate robust sim-to-real transfer with high success rates in tasks including dexterous manipulation, bimanual coordination, and video-to-robot imitation.

GenMimic refers to a suite of automated data generation and policy learning systems for robot imitation learning, designed to scale high-fidelity, physically plausible robotic behavior from minimal human supervision. The GenMimic paradigm underpins multiple instantiations—including MimicGen, DexMimicGen, and DynaMimicGen—each embodying common principles of object-centric temporal segmentation, context-aware trajectory transformation, and robust policy learning, with extensions for dexterous bimanual manipulation and adaptation to dynamic environments. The term is also used in work on humanoid motion imitation from generated video, where physics-aware policies enable zero-shot tracking of 4D human signals in robotic joint space. GenMimic intersects with reinforcement learning, classical control, pose estimation, and domain adaptation, and has demonstrated efficacy in both simulation and direct sim-to-real transfer across a variety of manipulation and mobile/humanoid platforms (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025, Ni et al., 4 Dec 2025).

1. Underlying Principles and Formalism

GenMimic is grounded in the automatic synthesis of large demonstration datasets from a handful of reference human demonstrations, supporting policy learning algorithms such as behavioral cloning and diffusion models. The approach relies on:

• Object-centric decomposition: Each demonstration is parsed into contiguous subtasks, $S_i(o_{S_i})$, indexed by object $o_{S_i}$, reflecting the agent's interaction with specific objects within a scene. The segmentation leverages event signals (e.g., "mug grasped") or manual annotation, enabling fine-grained behavioral units (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).
• Subtask retargeting: For each $S_i(o_{S_i})$, a transformation or model (e.g., rigid SE(3) mapping, Dynamic Movement Primitives) enables the replay or adaptation of action/pose sequences in novel contexts—including different object placements, robot morphologies, or perturbed scenes (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).
• Automated curation and filtering: Only successful synthetic demonstrations, as determined by task-completion predicates, are absorbed into the dataset for downstream training (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).

The fundamental aim is to amplify coverage over context and state space, yielding statistical diversity and robustness in learned policies with vastly reduced human overhead compared to direct demonstration collection.

2. Core Algorithmic Workflow

The canonical GenMimic pipeline consists of the following stages:

1. Human Demonstration Acquisition: A small set of seed trajectories $\mathcal{D}_{\rm src} = \{\tau^j\}_{j=1}^N$ is collected via teleoperation or kinesthetic teaching. Each trajectory comprises pose/action pairs $(s^j_t, a^j_t)$ over time (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024).
2. Subtask Segmentation: Each trajectory is decomposed into $M$ object-centric segments $(\tau_1, ..., \tau_M)$ using transitions inferred from the task structure or event flags (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).
3. Trajectory Transformation/Adaptation: For each segment, a context-dependent mapping is defined:
   • In rigid retargeting (e.g., MimicGen), each pose is mapped via $T^{C'_t}_W = T^{O'_0}_W (T^{O_0}_W)^{-1} T^{C_t}_W$, aligning the original segment to the new object's initial pose (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024).
   • In dynamic settings (e.g., DynaMimicGen), Dynamic Movement Primitives (DMPs) are trained per segment, enabling the system to adapt motion in real time to moving objects by reparameterizing the start pose $y_0$ and goal $g$ as external conditions evolve (Pomponi et al., 20 Nov 2025).
   • For bimanual/coordination tasks (DexMimicGen), parallel, sequential, and coordinated subtasks are processed with arm-specific or shared transforms and synchronization constraints (Jiang et al., 31 Oct 2024).
4. Execution and Validation: The retargeted segment(s) are sequentially executed in simulation (or on hardware if feasible), using either the transformed waypoints or on-the-fly DMP rollout (including gripper/finger commands as appropriate). Only runs passing a task predicate are retained; unsuccessful attempts are discarded (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).
5. Dataset Finalization: After repeating the process across varied initial states/scene configurations, the resulting demonstration dataset $\mathcal{D}_{\rm gen}$ is normalized (e.g., to $[-1,1]$) and used to train policies via imitation learning (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).
6. Policy Training: Standard algorithms include behavior cloning (with RNNs for visual or low-dimensional channels), diffusion-based sequence denoisers, or Gaussian Mixture Model (GMM) heads for multimodal action spaces (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).

This general architecture is tailored in each instantiation (see below) to address the requirements of static, dynamic, bimanual, or video-to-robot imitation.

3. Notable Instantiations

System	Key Innovations	Target Platforms
MimicGen	SE(3)-frame retargeting, delta-pose, action noise, multi-arm & object transfer	Arm manipulation, mobile robots (Mandlekar et al., 2023)
DexMimicGen	Bimanual segmentation, coordination/synchronization, fingered hands	Humanoids, dual-arm, dexterous hands (Jiang et al., 31 Oct 2024)
DynaMimicGen	DMP-based dynamic adaptation, online goal retargeting	Visual RL, dynamic scenes (Pomponi et al., 20 Nov 2025)
GenMimic (video)	4D human-lifting, keypoint-weighted tracking, symmetry regularization	Full humanoids, video tracking (Ni et al., 4 Dec 2025)

MimicGen (Mandlekar et al., 2023) first formalized the core pipeline, creating over 50K demonstrations for 18 tasks from ~200 human seeds. Datasets covered basic stacking, fine assembly, mobile manipulator scenarios, and robotic transfer between arm types. Key features include context-adaptive SE(3) retargeting, noise injection for action diversity, and success-based filtering.

DexMimicGen (Jiang et al., 31 Oct 2024) extends GenMimic to bimanual and dexterous platforms, decomposing tasks into parallel, coordinated, and sequential subtasks. Per-arm and coordination subtasks use different synchrony (asynchronous queues, synchronization barriers, ordering constraints), allowing generation of 21K demonstrations across nine MuJoCo environments with only 60 initial demos. The pipeline was validated on real humanoid can-sorting with sim-in-the-loop transfer.

DynaMimicGen (Pomponi et al., 20 Nov 2025) introduces generalization in dynamic, non-static environments using DMPs. Instead of rigid frame mapping, each segment is represented as a second-order dynamic system with nonlinear forcing, trained via locally weighted regression and integrated online with continuous reparameterization of start/goal according to observed object pose. This enables adaptation mid-trajectory to environment changes, yielding data more representative of real-world complexity and improving downstream policy robustness.

GenMimic for Video-to-Robot (Ni et al., 4 Dec 2025) addresses the challenge of tracking noisy, morphologically inconsistent human motions from video generation models. The pipeline lifts generated RGB video to SMPL-based 4D skeletons, retargets these via kinematic mapping (PHC) to humanoid robot coordinates, and trains physics-aware reinforcement learning policies; key features include keypoint-weighted tracking rewards and symmetry regularization. GenMimicBench was proposed for benchmark evaluation. The system achieves zero-shot transfer from noisy video to real humanoid (Unitree G1) trajectories without fine-tuning.

4. Empirical Performance and Evaluation

GenMimic-based systems have been systematically evaluated on manipulation, bimanual, dynamic, and humanoid imitation tasks:

• Data Generation Rate (DGR) measures the fraction of generated attempts producing fully valid trajectories, often exceeding static baselines with fewer human seeds (Pomponi et al., 20 Nov 2025).
• Imitation Success Rate quantifies downstream policy success (e.g., achieving task completion in random new scenes, or visually matching reference clips). Empirical highlights include:
  • DynaMimicGen: (image-based DP/BC) Stack: 83.3%/76.0%; Square: 86.7%/97.3%; MugCleanup: 90.0%/79.3% versus MimicGen at 74.0%/72.7%; 75.3%/88.7%; 61.3%/65.3%, respectively (static scenes) (Pomponi et al., 20 Nov 2025).
  • Significant drop (20–40 pp) in success without dynamic adaptation, confirming the value of on-the-fly retargeting.
  • DexMimicGen increases real-world can-sorting success to 90% using 40 synthetic demos versus 0% from 4 raw demos (Jiang et al., 31 Oct 2024).
  • GenMimic (video) student policies attain 29.8% visual success rate on physically tracking real-world humanoid motions from generated videos, outperforming strong teacher-student baselines; in simulation, teacher policy reaches up to 86.8% SR (Ni et al., 4 Dec 2025).

Repeated findings indicate that synthetic GenMimic datasets match or outperform the impact of additional direct human demonstration collection, and policy generalization is robust to expanded initial state distributions, scene layouts, and perturbation (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).

5. Extensions and Limitations

GenMimic architectures are extensible to:

• Sim-to-real transfer: Via domain randomization, object variation, and careful control parameter tuning, policies learned on synthetic data generalize with minimal (or zero) real-world retraining; e.g., DynaMimicGen and DexMimicGen both demonstrate robust sim-real deployment in manipulation and can-sorting (Pomponi et al., 20 Nov 2025, Jiang et al., 31 Oct 2024, Ni et al., 4 Dec 2025).
• Transformation models: The breadth of trajectory adaptation spans from rigid SE(3) transforms (suitable for static/structured tasks) to DMP-based dynamic motion generation, and incorporates asynchrony/synchrony and coordination cues for bimanual or sequential dependencies (Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025).
• Input modalities and robustness: Video-based GenMimic uses keypoint/policy design tailored for noisy, visually ambiguous contexts, with weighted tracking and reflection losses to mitigate instability and artifact propagation from synthetic percepts (Ni et al., 4 Dec 2025).

Current limitations reside in handling low-quality or semantically inconsistent generated data (especially in video-based approaches), absence of contact or force modeling in keypoint-based tracking, and the need for richer 4D reconstruction domains to broaden policy skills to include dynamic, athletic, or manipulation-centric humanoid motions (Ni et al., 4 Dec 2025). Coordination transforms in highly constrained bimanual handover present additional challenges, often requiring special-case handling or relaxed constraints (Jiang et al., 31 Oct 2024).

6. Relationship to Prior Art, Implications, and Future Directions

GenMimic consolidates, generalizes, and surpasses prior data-generation and augmentation methods for robot learning by providing a unified framework for context-adaptive, object-centric, and scalable demonstration synthesis. A plausible implication is a shift in robot learning toward heavily data-driven, simulation-first pipelines with reduced human-in-the-loop effort, enabling rapid expansion to new tasks, platforms, and sensor modalities (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025, Ni et al., 4 Dec 2025).

Open directions include:

• Joint adaption of perception (e.g., 4D human reconstruction networks) to simulated/generated video domains (Ni et al., 4 Dec 2025).
• Enriching policy representations with contact, affordance, or latent trajectory information for higher-fidelity manipulation and locomotion.
• Integrating large-scale language-conditioned scene variation for further data diversity.

In summary, GenMimic (“Editor’s term”: the family of context-adaptive, automated demonstration generation and imitation-learning pipelines) constitutes a scalable, domain-transferrable approach to robot skill acquisition, drawing from advances in pose retargeting, dynamic motion primitive encoding, coordination control, and robust policy optimization. It establishes a foundation for future research in autonomous, data-centric robotics (Mandlekar et al., 2023, Jiang et al., 31 Oct 2024, Pomponi et al., 20 Nov 2025, Ni et al., 4 Dec 2025).