EasyMimic: Low-Cost Robot Imitation

• EasyMimic is a low-cost robot imitation learning framework that learns manipulation policies from ordinary human RGB videos combined with limited robot teleoperation data.
• It overcomes data cost and human-to-robot embodiment gaps by using 3D hand reconstruction, geometric retargeting, and rendering-based visual augmentation.
• Balanced co-training with independent action heads enables robust policy learning, significantly improving task performance in low-cost home-use robotic settings.

Searching arXiv for the primary EasyMimic paper and closely related imitation-learning context. EasyMimic is a low-cost robot imitation learning framework that learns manipulation policies from ordinary human RGB videos together with a small amount of robot teleoperation data, with the stated goal of making home-use robots easier and cheaper to train (Zhang et al., 12 Feb 2026). Its central design premise is that robot imitation learning is constrained by two coupled bottlenecks: the high cost of collecting real robot demonstrations and the human-to-robot embodiment gap. EasyMimic addresses those bottlenecks by extracting 3D hand trajectories from monocular video, retargeting them into robot end-effector actions, reducing visual embodiment mismatch through hand visual augmentation, and co-training a vision-language-action policy on processed human data and limited robot data.

1. Problem formulation and scope

EasyMimic is positioned around two explicit difficulties in robot imitation learning. The first is a data cost bottleneck: collecting real robot demonstrations is expensive and slow. The second is a human-to-robot embodiment gap, decomposed into a visual gap, because human hands look unlike robot grippers, and an action gap, because human arm and hand motion does not directly match robot kinematics or gripper control (Zhang et al., 12 Feb 2026).

The framework therefore bridges the gap in two spaces. In action space alignment, human hand motion is retargeted into robot end-effector poses and gripper commands. In visual space alignment, human hand images are augmented so that the policy is driven less by human-specific appearance and more by task-relevant geometry. The training regime is then not purely robot-only and not purely human-only: EasyMimic uses co-training on processed human demonstrations and a small robot dataset, rather than a sequential pretrain-then-finetune pipeline (Zhang et al., 12 Feb 2026).

A common misconception is that EasyMimic is a fully robot-free learning system. The reported design does not support that interpretation. It still uses a small robot teleoperation dataset, and the reported gains come from combining that dataset with a larger, cheaper human-video corpus. Another misconception is that the method merely performs visual imitation from raw video. The reported pipeline is more structured: it explicitly reconstructs 3D hand representations, performs geometric retargeting, applies rendering-based augmentation, and learns with embodiment-specific action heads.

2. Hand reconstruction and action alignment

EasyMimic begins from monocular human RGB video. For each frame, it applies HaMeR to recover a 3D hand representation in the camera frame, producing 21 hand keypoints $X_t^\mathcal{C}$ and 778 hand mesh vertices $V_t^\mathcal{C}$ (Zhang et al., 12 Feb 2026). These two outputs are used differently: the keypoints support action alignment, while the mesh supports visual augmentation.

The action module maps human motion into robot gripper control as a pose sequence

$P_t = (p_t, q_t, g_t),$

where $p_t$ is 3D position, $q_t$ is orientation, and $g_t$ is gripper state. The human hand trajectory is denoted

$\mathcal{X}_h = \{X_t\}_{t=1}^T,$

and the corresponding robot pose sequence is

$\mathcal{P}_h = \{P_t\}_{t=1}^T.$

For position alignment, EasyMimic does not use the wrist or fingertip midpoint. It instead uses the center of the thenar eminence, defined as the midpoint between the thumb PIP joint and the index finger MCP joint. The reported rationale is stability during grasping and a better approximation to the interaction center with the object (Zhang et al., 12 Feb 2026).

For orientation alignment, the method fits a plane through five keypoints: the four joints of the index finger and the thumb PIP joint. The plane normal defines the $Z$-axis, the vector from index MCP to index PIP defines the $X$-axis, and the third axis is obtained by cross product, yielding a rotation matrix $V_t^\mathcal{C}$0, which is then converted into the robot orientation representation $V_t^\mathcal{C}$1. For gripper alignment, the control signal is derived from the thumb-tip to index-fingertip distance,

$V_t^\mathcal{C}$2

which is normalized and clipped into a continuous gripper command,

$V_t^\mathcal{C}$3

The use of a continuous $V_t^\mathcal{C}$4, rather than a binary open/close label, is reported as enabling gentle or partial grasps (Zhang et al., 12 Feb 2026).

After the hand pose is defined in the camera frame, a pre-calibrated hand-eye transform

$V_t^\mathcal{C}$5

maps it to the robot base frame. The framework then adopts action chunking: the action at time $V_t^\mathcal{C}$6 is defined as the next-step state,

$V_t^\mathcal{C}$7

and a chunk is

$V_t^\mathcal{C}$8

This suggests that EasyMimic treats short-horizon future prediction as a stabilizing prior for imitation, rather than relying on single-step reactive supervision alone.

3. Visual alignment and co-training architecture

The visual alignment module is intentionally lightweight. For each frame, EasyMimic takes the reconstructed 3D hand mesh, renders it over the original image, randomizes the hand color during rendering, and produces an augmented image $V_t^\mathcal{C}$9 (Zhang et al., 12 Feb 2026). The reported mechanism is domain randomization: by varying color and standardizing hand rendering, the policy is encouraged to attend to pose, shape, and object-relative geometry rather than to skin tone, texture, or other human-specific appearance cues.

The reported ablations indicate that this is not a minor auxiliary component. Removing visual alignment causes a large performance drop, and masking only the thumb-index region is substantially worse than augmenting the entire hand. This directly contradicts the intuitive but incorrect assumption that only the grasping fingertips are visually important. The reported result instead indicates that the whole hand contributes to the human-robot visual gap (Zhang et al., 12 Feb 2026).

The policy is built on the pre-trained VLA backbone GR00T N1.5-3B. The architecture contains a shared frozen vision-language encoder, a shared DiT module for policy learning, and independent action encoders and decoders / heads for human and robot data. The independent heads are not a cosmetic design choice: the reported interpretation is that human and robot action statistics differ significantly, and a single shared head introduces interference (Zhang et al., 12 Feb 2026).

Training uses a mixed dataset comprising processed human demonstrations $P_t = (p_t, q_t, g_t),$0 and robot teleoperation data $P_t = (p_t, q_t, g_t),$1, combined into $P_t = (p_t, q_t, g_t),$2. To keep the larger human dataset from dominating optimization, EasyMimic uses balanced sampling, so each minibatch includes both human and robot samples. Human videos do not provide the robot wrist-camera view, so the missing wrist-view slot is zero-padded to keep the input interface consistent (Zhang et al., 12 Feb 2026).

The paper reports that co-training is superior to Pretrain-Finetune and also faster; the sequential alternative is reported as about 1.6× slower. A plausible implication is that joint optimization is functioning as a regularizer on the shared representation: human data injects broad task priors, while robot data constrains those priors to the target embodiment.

4. Platform, tasks, and data regime

EasyMimic is evaluated on the LeRobot SO100 platform, upgraded from 5-DoF to 6-DoF by adding one joint. The reported reason for the upgrade is that the original 5-DoF arm had orientation limitations and singularity issues, while the 6-DoF version expands the workspace and improves pose control (Zhang et al., 12 Feb 2026). The full setup includes a Nintendo Joy-Con for teleoperation, a fixed first-person/global RGB camera, and a wrist-mounted RGB camera. The entire system is reported to cost under \$300.

The evaluation covers four tabletop manipulation tasks:

Task	Description	EasyMimic score
Pick and Place (Pick)	pick toy duck, place it in a bowl	1.00
Pull and Push (Pull)	pull open drawer, push it closed	0.90
Stacking (Stack)	stack cube on block, stack pyramid on cube	0.70
Language Conditioned (LC)	execute instructions such as “pick up the pink duck and place it into the yellow bowl”	0.90

For each task, the reported data budget is 100 human video demonstrations and 20 robot teleoperation trajectories (Zhang et al., 12 Feb 2026). The framework emphasizes the asymmetry in collection cost. The paper gives two explicit examples: for Pick, 100 human demos in 8 min versus 20 robot demos in 10 min; for Stack, 100 human demos in 15 min versus 20 robot demos in 40 min. The significance is not merely that human data is cheaper, but that it scales much faster even on the same task family.

Training uses GR00T N1.5-3B, absolute actions, 5000 training steps, a single RTX 4090, AdamW, learning rate 1e-4, and batch size 32 (Zhang et al., 12 Feb 2026). These details place the method within a low-cost robotics setting rather than a large-scale industrial data regime.

5. Empirical performance and ablations

The main reported comparison is between robot-only training, sequential pretrain-finetune, and EasyMimic co-training. Average performance across tasks is reported as 0.26 for Robot-Only (10 traj.), 0.51 for Robot-Only (20 traj.), 0.75 for Pretrain-Finetune, and 0.88 for EasyMimic (Zhang et al., 12 Feb 2026). The associated interpretation is explicit: small robot-only datasets are insufficient, human data substantially improves performance, and co-training with aligned human videos performs best.

The Language Conditioned task highlights this effect sharply. The reported score rises from 0.40 with 20 robot trajectories alone to 0.90 with EasyMimic. This suggests that the framework is not only transferring geometric manipulation priors but also supporting language-conditioned policy learning under severe robot-data constraints (Zhang et al., 12 Feb 2026).

The ablation study localizes the main sources of gain. Removing action alignment reduces average performance from 0.87 to 0.60. Removing visual alignment reduces it from 0.87 to 0.40. Using VA-Partial yields 0.27, much worse than full visual alignment. Replacing independent action heads with a shared head reduces average performance from 0.87 to 0.47. Without pretraining, scores fall for both robot-only and EasyMimic settings, but EasyMimic still outperforms the robot-only baseline, indicating that pretraining and human data are complementary rather than interchangeable (Zhang et al., 12 Feb 2026).

The paper also reports data-scaling behavior. More human data improves performance, but gains diminish after about 50 demos. Robot data improves performance most from 5 → 10 trajectories, and performance saturates quickly after about 10–20 robot trajectories (Zhang et al., 12 Feb 2026). This is consistent with a regime in which human data supplies broad behavioral coverage while robot data primarily calibrates embodiment-specific execution.

On generalization to unseen objects in Pick and Place, the average score is reported as 0.35 for Robot-Only and 0.65 for EasyMimic. The stated failure cases include premature gripper release, imprecise handle grasping, collisions during stacking, and unstable placement near edges (Zhang et al., 12 Feb 2026). These failure modes indicate that the remaining error is concentrated in contact timing, grasp precision, and geometric clearance, rather than in gross trajectory generation.

6. Relation to adjacent imitation and mimicry frameworks

EasyMimic belongs to a broader research line in which imitation becomes practical by changing either the data interface, the embodiment mapping, or the formalization of mimicry. In robotics, “Visual Imitation Made Easy” uses a commercial reacher-grabber assistive tool as both the human demonstration interface and the robot end-effector, recovering actions from video with COLMAP and a finger detection network, and then training with standard behavior cloning (Young et al., 2020). “Dexterous Imitation Made Easy” collects Allegro-hand demonstrations from a single RGB camera using MediaPipe Hands, then learns dexterous manipulation policies through either RL with demonstrations or non-parametric nearest-neighbor imitation (Arunachalam et al., 2022). “Multiple Interactions Made Easy (MIME)” provides 8,260 demonstrations over 20 tasks with paired third-person human videos and kinesthetic Baxter trajectories, explicitly framing video-to-robot mapping as a visual imitation benchmark (Sharma et al., 2018).

Relative to those systems, EasyMimic is distinctive in the specific way it decomposes the transfer problem. It does not merely collect paired data, and it does not rely only on behavior cloning from visually extracted actions. Instead, it explicitly combines monocular 3D hand reconstruction, geometric pose-and-gripper retargeting, rendering-based visual alignment, and VLA co-training with embodiment-specific action heads (Zhang et al., 12 Feb 2026). This suggests a shift from demonstration-interface engineering alone toward representation-level human-to-robot alignment.

The term “mimic” also has a formal usage outside robotics. “Mimicking Behaviors in Separated Domains” defines the MBSD problem, where one domain $P_t = (p_t, q_t, g_t),$3 generates behavior and another domain $P_t = (p_t, q_t, g_t),$4 must produce a matching behavior step by step under an $P_t = (p_t, q_t, g_t),$5 mapping specification (Giacomo et al., 2022). That work studies game-based synthesis and complexity for point-wise, target, and general $P_t = (p_t, q_t, g_t),$6 mappings. EasyMimic is not formulated in that automata-theoretic language, but both frameworks share an underlying concern: how to map behavior observed in one domain into admissible behavior in another under explicit constraints.

Across these neighboring systems, a recurring theme is that “easy” imitation is not achieved by simplifying the target task; it is achieved by simplifying the interface between source behavior and target execution. In EasyMimic, that interface is the combination of 3D hand trajectory extraction, retargeted action chunks, full-hand visual augmentation, and balanced co-training.

7. Limitations, interpretation, and significance

The reported limitations are concrete. EasyMimic depends on accurate monocular 3D hand reconstruction; errors in HaMeR propagate downstream. It still requires some robot data and is therefore not a pure observation-only transfer method. The experiments focus on tabletop manipulation with a low-cost arm and gripper, and the evidence does not yet extend to broader mobile or long-horizon settings (Zhang et al., 12 Feb 2026).

These limitations are important for interpreting the contribution correctly. EasyMimic should not be understood as solving general human-video-to-robot transfer in the abstract. Rather, it demonstrates that, on a low-cost home-robot platform, a carefully designed hybrid data regime can sharply reduce dependence on expensive robot teleoperation while preserving strong task performance. The framework’s main significance is thus methodological: it identifies a viable operating point between purely robot-supervised imitation and purely human-video supervision.

A plausible implication is that EasyMimic is best viewed as an embodiment-alignment framework rather than only a data-collection shortcut. Its reported gains are distributed across action alignment, visual alignment, independent action heads, and co-training. That distribution matters because it indicates that low-cost imitation is not determined by one module alone; it emerges from coordinated reductions in both the visual and action-space mismatch between human demonstrations and robot execution (Zhang et al., 12 Feb 2026).