Imitation-Guided Bimanual Planning

• Imitation-guided bimanual planning is a set of robotic frameworks that convert demonstrations into planning-relevant structures like keyposes and affordance maps.
• These frameworks use representations such as movement primitives, screw actions, and semantic scene graphs to manage dual-arm coordination, collision avoidance, and contact dynamics.
• The approach enhances stability and precision in complex, long-horizon tasks, with empirical gains demonstrated in both simulation and real-world experiments.

Imitation-guided bimanual planning denotes a family of robotic frameworks that use demonstrations to generate, sequence, or constrain coordinated two-arm behavior for tasks in which dual-arm synchronization, object-relative geometry, contact dynamics, and long-horizon stage structure are all consequential. Across recent work, the shared objective is not merely to imitate arm trajectories, but to convert demonstrations into planning-relevant structure: movement primitives, keyposes, object-centric transforms, affordance maps, scene-graph predicates, symmetry operators, or latent future representations. These structures are then coupled to generative policies, task-and-motion planners, consistency or diffusion models, graph neural networks, or compliant controllers to produce stable execution under coordination, reachability, collision, and contact constraints (Xie et al., 2020, Bahety et al., 2024, Yu et al., 2024, Chen et al., 23 Jun 2025, Xu et al., 17 Jan 2026, Chen et al., 7 Mar 2026).

1. Problem setting and design rationale

Bimanual manipulation is repeatedly formulated as a setting in which two arms must coordinate continuously while remaining compatible with the manipulated object, the environment, and the task phase. The difficulty is attributed to several recurring factors: high-dimensional action spaces, multi-modal dynamics, strict spatial and temporal synchronization, contact-rich transitions, and long-horizon error accumulation. In the graph-based hierarchical formulation of "Deep Imitation Learning for Bimanual Robotic Manipulation," these issues appear as multi-modal dynamics, relational variability, and generalization across changed object locations (Xie et al., 2020). In "BiKC+: Bimanual Hierarchical Imitation with Keypose-Conditioned Coordination-Aware Consistency Policies," the same setting is framed as a combination of dual-arm coordination and long-horizon, multi-stage structure, with the additional observation that failures or delays at one stage propagate to later stages (Xu et al., 17 Jan 2026).

Several systems make the object explicit in the planning problem. "RoTri-Diff: A Spatial Robot-Object Triadic Interaction-Guided Diffusion Model for Bimanual Manipulation" states that generating two-arm trajectories requires maintaining spatial consistency with the manipulated object over time, avoiding inter-arm collisions, and maintaining stable contacts (Chen et al., 7 Mar 2026). "BiPreManip: Learning Affordance-Based Bimanual Preparatory Manipulation through Anticipatory Collaboration" focuses on asymmetric sequential tasks in which one arm must first reconfigure an object so that the other arm can perform the goal-directed action, such as pushing an iPad to the table edge before grasping it or lifting a pen body so that the other hand can remove its cap (Shen et al., 23 Mar 2026).

Other formulations emphasize execution conditions beyond nominal kinematics. "A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies" treats a bracket insertion task as a closed-chain, contact-rich process in which the policy must exploit contact rather than avoid it, and reports that force/torque data is crucial for phase estimation and robustness (Stepputtis et al., 2022). "Imitation-Guided Bimanual Planning for Stable Manipulation under Changing External Forces" instead centers the problem around changing external wrenches, grasp transitions between uni-manual and bi-manual support, and motion-performance optimization under stability constraints (Cai et al., 23 Sep 2025). In mobile settings, "MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation" extends the planning problem to include mobile base placement and active camera placement, with hard reachability, collision, visibility, and temporal constraints (Li et al., 21 Oct 2025).

A plausible implication is that imitation-guided bimanual planning has evolved from direct behavioral cloning of joint trajectories toward structured formulations in which demonstrations are used to define intermediate planning objects. Those intermediate objects differ across papers, but they all function as coordination scaffolds.

2. Representational foundations

A central distinction among frameworks concerns what demonstrations are converted into before planning or action generation begins.

Framework	Core representation	Representative role
HDR-IL	Elemental movement primitives with recurrent graph neural network dynamics	Decomposes multi-modal trajectories into primitive-specific relational dynamics
ScrewMimic	Screw action $\sigma = (g_l, g_r, S, \tau_l)$	Projects human bimanual motion into a compact serial-linkage manifold
BiKC / BiKC+	Joint-space bimanual keyposes, with $m_j \in \{0,1\}$ in BiKC+	Marks sub-stage completion and coordination mode
RoTri-Diff	21D RoTri vector $R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$	Encodes robot-object triadic geometry densely over time
SViP	Semantic scene graphs and object-centric motion primitives	Segments demonstrations into symbolic-contact structure for TAMP
EquiBim	Bilateral reflection group action on observations and actions	Enforces left-right equivariance during imitation learning

In relational formulations, the goal is to encode how arms and objects constrain one another. HDR-IL uses a fully connected graph over state features, with Graph Attention message passing inside a GRU encoder-decoder, and explicitly reports that absolute pose features plus learned interactions outperform hand-crafted relational coordinates (Xie et al., 2020). RoTri-Diff makes this coupling geometric: with end-effectors $A,B$ and object $O$, world-frame poses $T_A,T_B,T_O \in SE(3)$ induce relative transforms such as $T_{AO}=T_A^{-1}T_O$, $T_{BO}=T_B^{-1}T_O$, and $T_{AB}=T_A^{-1}T_B$, together with the triangular consistency condition

$T_{AO}T_{OB}T_{BA}=I.$

The associated loop-consistency loss,

$m_j \in \{0,1\}$0

penalizes drift among pairwise relations (Chen et al., 7 Mar 2026).

Keypose-centered frameworks represent demonstrations as milestone states rather than dense trajectories. In BiKC, a keypose is a joint-space waypoint that is common across demonstrations and marks a sub-stage boundary; the low-level policy then generates action chunks toward that waypoint (Yu et al., 2024). BiKC+ extends this by attaching a binary coordination indicator $m_j \in \{0,1\}$1, where $m_j \in \{0,1\}$2 requires both arms to reach the keypose simultaneously and $m_j \in \{0,1\}$3 allows either arm to reach independently, with coordination ranges derived from VLM-inferred contact graphs (Xu et al., 17 Jan 2026).

Other papers encode coordination in more task-specific abstractions. ScrewMimic models the two hands as a serial kinematic linkage with a virtual 1-DoF screw joint, and represents a bimanual behavior by the screw action $m_j \in \{0,1\}$4, where $m_j \in \{0,1\}$5 is the screw axis and $m_j \in \{0,1\}$6 are grasp contacts (Bahety et al., 2024). BiPreManip predicts dense goal and preparatory affordance maps over a point cloud, together with object transformation $m_j \in \{0,1\}$7, so that one arm can reconfigure the object into a configuration better suited for the other arm’s final action (Shen et al., 23 Mar 2026). SViP instead monitors demonstrations through semantic scene graphs $m_j \in \{0,1\}$8 and turns key graph transitions into symbolic preconditions and effects for object-centric primitives such as Pick, Place, Transfer, and Align (Chen et al., 23 Jun 2025).

Symmetry-based work treats coordination as an inductive bias over observation-action structure. EquiBim defines a bilateral reflection group $m_j \in \{0,1\}$9 acting on images, point clouds, proprioception, and actions, and imposes the equivariance condition

$R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$0

with an additional regularizer $R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$1 added to the behavior cloning objective (Zhang et al., 9 Mar 2026).

3. Hierarchical policy architectures and generative models

Most imitation-guided bimanual planners are hierarchical, but the level at which hierarchy is introduced differs markedly. HDR-IL uses an explicit three-layer decomposition: a high-level planner predicts the next primitive $R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$2, a primitive-specific recurrent GNN forecasts a short horizon of future states, and an IK controller turns those states into joint commands (Xie et al., 2020). BiKC and BiKC+ also split the problem into high- and low-level components, but replace primitive labels with predicted keyposes. In BiKC+, the factorization is written as

$R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$3

so that stage planning and trajectory generation remain distinct but tightly coupled (Xu et al., 17 Jan 2026).

Generative modeling has become a dominant mechanism for the low-level component. BiKC formulates the trajectory generator as a consistency model trained from scratch, which produces an action sequence in one inference step rather than by iterative denoising (Yu et al., 2024). BiKC+ retains this consistency-model formulation and uses it jointly with a CM-based keypose predictor, with reported real-world latencies such as 34.9 ms on Screwdriver Packing and 27.8 ms on Conveyor (Xu et al., 17 Jan 2026). The emphasis on one-step generation is shared by Bi-HIL, although Bi-HIL uses a different hierarchy: a high-level transformer predicts a subtask command and a scalar subtask-level progress rate, while a low-level transformer-based CVAE generates force-aware action chunks conditioned on that progress estimate and keyframe memory (Buamanee et al., 4 Mar 2026).

Diffusion-based hierarchical policies are more explicit about multi-scale decomposition. RoTri-Diff separates prediction into three stages: object pointflow and RoTri segment prediction, keypose generation, and dense continuous trajectory generation conditioned on the full RoTri segment (Chen et al., 7 Mar 2026). Its diffusion formulation uses a forward noising process

$R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$4

and a reverse denoising process with conditioning on scene tokens, language, proprioception, object initialization, and RoTri state. Diffusion-Based Imaginative Coordination trains a shared diffusion decoder for both actions and future visual latents, but imposes a unidirectional attention mask so that video prediction is conditioned on actions while action prediction remains independent of video tokens; the resulting action-only inference path is then used at test time (Xu et al., 15 Jul 2025).

Transformer-based chunking architectures push hierarchy into the temporal interface between policy and control. "Learning Bimanual Manipulation via Action Chunking and Inter-Arm Coordination with Transformers" separates per-arm encoders and inserts an Inter-Arm Coordinated transformer Encoder (IACE) before decoding action chunks, with decoder choice depending on whether the task is synchronous or asynchronous (Motoda et al., 18 Mar 2025). InterACT similarly predicts chunked action sequences, but uses a Hierarchical Attention Encoder with segment-wise and cross-segment attention, followed by a Multi-arm Decoder with a Synchronization Block that lets each arm’s decoder consume the other arm’s intermediate outputs before producing final actions (Lee et al., 2024).

A related but distinct trend is to insert learned policies into explicit planning systems. SViP does not rely on a single end-to-end low-level policy for an entire task; instead, it partitions demonstrations into contact-rich bimanual segments and uses learned visuomotor policies only for those segments, while scripted object-centric primitives and PDDLStream-based TAMP handle the surrounding transitions (Chen et al., 23 Jun 2025). This suggests a broader interpretation of imitation-guided planning: the demonstrations need not directly specify continuous control everywhere, so long as they define where learned control is necessary and how the planner should enter and exit those regions.

4. Planning integration, control realization, and constraints

Execution layers in imitation-guided bimanual planning are typically classical control systems augmented by learned targets, constraints, or safety monitors. IK remains a standard bridge from learned end-effector or keypose targets to joint-space execution. HDR-IL uses stacked Jacobians for both arms and damped least-squares inverse kinematics, with

$R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$5

for coordinated velocity control (Xie et al., 2020). RoTri-Diff describes both IK and differential IK deployment, including the damped least-squares update

$R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$6

together with joint-limit, velocity, acceleration, and SDF-based collision penalties (Chen et al., 7 Mar 2026).

Constraint handling ranges from soft penalties to full constrained optimization. MoMaGen casts data generation itself as a constrained optimization problem with hard constraints for dynamics, kinematics, collision avoidance, visibility, preserved end-effector–object transforms during contact-rich segments, and task success. Its formulation jointly reasons over mobile base pose, active camera pose, dual-arm trajectories, timing, and subtask synchronization (Li et al., 21 Oct 2025). SViP uses PDDLStream with optimistic adaptive planning, and wraps learned modules as black-box samplers and validators: the switching-condition generator provides candidate continuous parameters, and a feasibility validator predicts whether a bimanual learned skill can be initiated safely under the current object arrangement (Chen et al., 23 Jun 2025).

Contact-rich systems often replace pure trajectory tracking with compliant control. In the multipoint insertion system of (Stepputtis et al., 2022), the low-level controller is Cartesian admittance:

$R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$7

with the learned Bayesian Interaction Primitive estimating phase and future references from multimodal observations that include force/torque. SIMPLe instead deploys Cartesian impedance policies learned from kinesthetic demonstrations, saturates attractor displacement and stiffness to enforce explicit velocity and force limits, and modulates stiffness by epistemic uncertainty so that the robot can be safely reshaped during active kinesthetic correction (Franzese et al., 2022). Bi-HIL uses bilateral control during data collection and then conditions its low-level policy on torque, joint angle, angular velocity, language, and progress-rate information, with leader–follower servo constraints $R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$8 and $R_t = [p_{(A\to B)}, p_{(A\to O)}, p_{(B\to O)}]$9 shaping the demonstrations themselves (Buamanee et al., 4 Mar 2026).

Stability under task-specific physical requirements has generated additional planning layers. The grasp-transition framework of (Cai et al., 23 Sep 2025) models uni-manual and bi-manual grasps as grasp manifolds $A,B$0 and $A,B$1, seeks stable intersection states $A,B$2, plans the minimal-regrasp sequence over a graph of stable configurations, and then executes motions with a hierarchical architecture comprising ProMP-based global path generation and a QP-driven local planner with manipulability optimization and obstacle avoidance. ScrewMimic also constrains execution geometrically, but through a different mechanism: once a screw action has been estimated from human video, the two hands are synchronized by construction because the relative motion satisfies $A,B$3 and the waypoints are tracked by a stack-of-tasks IK-based whole-body controller (Bahety et al., 2024).

A plausible synthesis is that imitation-guided bimanual planning has not displaced classical control; rather, it increasingly supplies the intermediate variables that make constrained control tractable.

5. Empirical findings across simulation and real robots

Reported performance gains are substantial but heterogeneous, reflecting differences in task families, observation modalities, and evaluation criteria.

Framework	Reported outcome
RoTri-Diff	80.9% average success on 11 RLBench2 tasks; outperforms baselines by 10.2%; real-world tasks: 5/5, 3/5, 4/5, 4/5 (Chen et al., 7 Mar 2026)
HDR-IL	Table lift: 100% success; peg-in-hole: 29% success (Xie et al., 2020)
InterACT	Transfer Cube: 82% vs ACT 50% on Transfer; Peg Insertion: 44% vs ACT 20% on Insert (Lee et al., 2024)
BiKC+	Screwdriver Packing: 59.9%; Pants Hanging: 85.0%; Conveyor: 100.0% (Xu et al., 17 Jan 2026)
EquiBim	RoboTwin Image+Joint: 34.1 → 43.6; mirrored Banana Handover: 0/10 → 5/10 under shift (Zhang et al., 9 Mar 2026)
Ag2x2	73.5% success across 13 diverse tasks (Xiong et al., 26 Jul 2025)

The evidence repeatedly supports the value of explicit coordination structure. HDR-IL reports that graph attention, residual target-object features, and primitive specialization are all consequential: on Table lift, HDR-IL achieved 100% success, while GRU-GRU reached 13%, and on Peg-in-hole, HDR-IL achieved 29% versus 1% for GRU-GRU (Xie et al., 2020). InterACT reports systematic gains over ACT on both simulated and real tasks, such as Peg Insertion Insert 44% versus 20%, Insert Plug 42% versus 30%, and Sweep 52% versus 42%, with ablations showing that removing the Synchronization Block or cross-segment encoder degrades the coordination-heavy stages most strongly (Lee et al., 2024). The related IACE-based transformer reports an 8–9% average success-rate improvement over ACT, with split decoders favored on asynchronous tasks and a single decoder favored on synchronous tasks (Motoda et al., 18 Mar 2025).

Keypose-guided hierarchical models report benefits in both reliability and operational speed. BiKC shows that keypose conditioning improves sensitive stages such as contact and insertion, and that a one-step consistency policy can outperform diffusion baselines on dynamic tasks because diffusion latency causes missed pick opportunities (Yu et al., 2024). BiKC+ extends this claim to more complex real-world tasks: 59.9% success on Screwdriver Packing, 85.0% on Pants Hanging, and 100.0% on Conveyor, while also reporting lower total duration than ACT on Screwdriver Packing despite ACT’s comparable per-step latency (Xu et al., 17 Jan 2026). Bi-HIL reports 80% success on both 6-Cup Downstack and 4-Peg-in-Hole, versus 60% and 20% for Bi-ACT and lower numbers for ablations without keyframe memory or subtask-level progress rate (Buamanee et al., 4 Mar 2026).

Diffusion and latent imagination have produced notable gains when the visual future is used during training but not at inference. Diffusion-Based Imaginative Coordination reports a 24.9% increase on ALOHA, an 11.1% increase on RoboTwin, and a 32.5% increase in real-world experiments over ACT, while preserving higher control frequency than standard diffusion inference by omitting video prediction at test time (Xu et al., 15 Jul 2025). RoTri-Diff reports especially large gains on precision-heavy RLBench2 tasks, including Pick plate 40.7% versus PPI 0.0% and AnyBimanual 8.0%, and Put item into drawer 87.0% versus 3DDA 52.7% (Chen et al., 7 Mar 2026).

Planning-integrated systems show a different kind of improvement: better recovery under OOD layouts or explicit task constraints. SViP reports 100% success on ID and XY-OOD peg-in-hole settings and 88% on XYH-OOD, while ACT falls to 2% in XYH-OOD; in real-world Object Handoff, Screwdriver Packing, and Cup-sleeve Insertion, SViP consistently exceeds diffusion-policy baselines under OOD and tilted setups using only 20 demonstrations (Chen et al., 23 Jun 2025). MoMaGen reports that synthetic demonstration generation under hard reachability and visibility constraints can support policies trained from a single source demonstration and later fine-tuned with as few as 40 real-world demonstrations, with visibility constraints materially affecting both generated-data quality and downstream policy performance (Li et al., 21 Oct 2025).

Contact- and force-aware systems likewise show clear empirical patterns. The contact-rich insertion system of (Stepputtis et al., 2022) reports up to 90% success in the real bimanual task and 100% robustness under disturbances in simulation, with an approximately 20 percentage-point improvement when force/torque data is available under varied starts. The external-force planning framework of (Cai et al., 23 Sep 2025) reports shorter end-effector path lengths, higher minimum singular values, larger joint-limit margins, and lower execution times than minimal-regrasp, greedy, and random-sampling baselines in cutting and drilling scenarios.

6. Limitations, trade-offs, and future directions

The surveyed literature reports several recurring limitations. RoTri-Diff explicitly depends on rigid-body assumptions and accurate 6D object pose estimation, and notes degradation under calibration error or poor tracking, especially for tight-clearance or deformable-object tasks (Chen et al., 7 Mar 2026). ScrewMimic similarly notes that pose-estimation noise and calibration errors can perturb the recovered screw axis and that tasks not well described by a single 1-DoF constraint require richer action sequences or mixtures of primitives (Bahety et al., 2024). BiPreManip identifies instability near small components, single-camera occlusion, and one-shot pushing failures, and argues that multi-step closed-loop pushing and multi-view perception would improve robustness (Shen et al., 23 Mar 2026).

Several frameworks depend on manual or heuristic task structure. HDR-IL uses manually labeled primitives and fixed-length call-and-return execution, and explicitly identifies automatic primitive discovery and termination learning as future directions (Xie et al., 2020). BiKC and BiKC+ rely on heuristic keypose extraction and, in BiKC+, VLM-based coordination-range detection, which the paper notes may be viewpoint-sensitive and may benefit from wrist force/torque signals (Yu et al., 2024, Xu et al., 17 Jan 2026). Bi-HIL requires manual subtask annotation and visual keyframe supervision, and it reports no explicit collision-avoidance module, so safe operation remains heavily dependent on demonstration quality and bilateral compliance (Buamanee et al., 4 Mar 2026).

A second recurring trade-off concerns how explicitly coordination should be constrained. BiKC+ states that there is no separate coordination regularizer and that coordination is enforced structurally through synchronized keyposes, shared encoders, and runtime gating (Xu et al., 17 Jan 2026). RoTri-Diff, by contrast, recommends loop-consistency, collision, alignment, antipodal, kinematic, and smoothness penalties for deployment (Chen et al., 7 Mar 2026). This suggests a design spectrum rather than a single consensus: some systems encode coordination in representation and inference logic, while others regularize it directly in geometry or control space.

A third trade-off concerns inductive bias versus task asymmetry. EquiBim shows that bilateral equivariance improves robustness under mirrored distribution shifts, but also reports that tasks with functional asymmetry can show smaller gains or mild drops when symmetry is enforced too strongly (Zhang et al., 9 Mar 2026). More broadly, scene-agnostic planners such as the visual-affordance reasoning framework of (Lee et al., 10 Dec 2025) gain flexibility by deferring to rule-based constraints and foundation models, but then inherit possible LLM hallucination, role asymmetry errors, and dependence on accurate grounding.

Future work is correspondingly diverse. Reported directions include multi-object triadic sets, differentiable contact models, friction cones, tactile feedback, and embodiment-conditioned policies in RoTri-Diff (Chen et al., 7 Mar 2026); whole-body loco-manipulation and improved planner efficiency in MoMaGen (Li et al., 21 Oct 2025); automatic success metrics and grasp optimization in ScrewMimic (Bahety et al., 2024); richer object-centric sub-goals and open-scene planning in scene-agnostic affordance planners (Lee et al., 10 Dec 2025); and stronger integration of latent representations, demonstrations, and zero-shot skill transfer in Ag2x2, which shows that agent-agnostic visual representations can still be leveraged for imitation learning (Xiong et al., 26 Jul 2025). A plausible implication is that the next generation of imitation-guided bimanual planning frameworks will be less defined by any one policy class than by how effectively they couple demonstrations to geometry, contact, and structured task decomposition.