Bi-HIL: Hierarchical Imitation Learning

• The paper introduces Bi-HIL, a novel framework that integrates bilateral control-based imitation learning with explicit subtask progression via keyframe memory and progress rates.
• It employs a two-level policy architecture where a high-level module generates subtask commands and phase information, and a low-level transformer-based controller produces force-aware continuous actions.
• Empirical results show Bi-HIL achieves up to 80% success in complex tasks by effectively managing subtask transitions in long-horizon, contact-rich manipulation scenarios.

Bi-HIL is a hierarchical imitation learning framework for long-horizon, contact-rich robotic manipulation that combines bilateral control-based force-aware demonstration collection with multimodal hierarchical policy learning. In the formulation introduced in "Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning via Subtask-Level Progress Rate and Keyframe Memory for Long-Horizon Contact-Rich Robotic Manipulation," the framework is designed for settings characterized by partial observability, ambiguous subtask completion, unstable subtask transitions, and repeated contact events such as slips, misalignment, and insertion. Its defining components are a two-level policy architecture, a subtask-level progress rate used as an explicit phase variable, and a keyframe memory used to retain representative frames from important task moments (Buamanee et al., 4 Mar 2026).

1. Conceptual definition and problem domain

Bi-HIL targets long-horizon manipulation tasks that are not single motions but sequences of subtasks. The motivating claim is that failure to identify the correct transition point between subtasks causes errors to accumulate until the task fails. This difficulty is amplified in contact-rich environments, where force interaction changes task state in ways that vision alone may not reliably capture (Buamanee et al., 4 Mar 2026).

The framework is presented as an overview of two previously separate lines of work. Hierarchical methods improve temporal reasoning but often lack force awareness. Bilateral control-based imitation learning captures force information, but the prior methods emphasized in the paper are flat policies that do not explicitly model long-horizon structure. Bi-HIL is therefore defined by the conjunction of hierarchical planning over subtasks and bilateral control-based force-aware imitation learning rather than by either ingredient in isolation (Buamanee et al., 4 Mar 2026).

A central term in the framework is the subtask-level progress rate. The paper defines this quantity for an active subtask as a normalized phase indicator based on the current timestep and the start and end times of that subtask. It is reset to zero whenever a new subtask begins. In operational terms, this makes it a local phase clock rather than a global task-completion signal. The paper’s interpretation is that the progress rate stabilizes the notion of where execution lies within the active subtask and reduces premature or repeated transitions (Buamanee et al., 4 Mar 2026).

A second defining term is keyframe memory. The high-level policy predicts keyframe scores over a recent observation window, and frames whose predicted keyframe probability exceeds a threshold $O_{th}$ are treated as candidate keyframes. Their indices are accumulated over time, clustered by 1D single-linkage using a fixed temporal distance, and represented by the median index in each cluster. The resulting keyframes are stored in memory $K_t$ up to a maximum size $K_{\max}$. In the paper’s formulation, these frames serve as anchors for completed or important subtask boundaries (Buamanee et al., 4 Mar 2026).

2. Hierarchical architecture and information flow

Bi-HIL is a two-level hierarchical policy comprising a high-level policy and a low-level policy. The high-level policy reasons over subtasks, memory, and progression; the low-level policy produces force-aware continuous control actions. The division of labor is explicit: the high-level policy provides task context and phase information, while the low-level policy executes contact-aware behavior conditioned on that context (Buamanee et al., 4 Mar 2026).

At each timestep, the high-level policy consumes a recent observation window $R_t = I_{t-N:t}$ and previously selected keyframes $K_t \subseteq I_{0:t-1}$, with $|K_t| \le K_{\max}$. Images are encoded by a frozen CLIP encoder and processed by a transformer encoder. The high-level outputs are the current subtask command $C_t$, the subtask-level progress rate $p_t \in [0,1]$, and keyframe scores for each frame in the current observation window (Buamanee et al., 4 Mar 2026).

The low-level policy is a transformer-based CVAE. It receives the subtask-level progress rate, the SigLIP-embedded subtask instruction predicted by the high-level policy, current RGB images, and follower robot joint states comprising angle, angular velocity, and torque. It predicts the next leader robot joint states: angle, angular velocity, and torque. The language embedding and RGB features are processed by a FiLM-conditioned EfficientNet, and the progress rate is discretized into 10 uniform levels for robustness before the features are passed to a transformer encoder (Buamanee et al., 4 Mar 2026).

This organization makes Bi-HIL structurally different from flat bilateral imitation learning. The paper’s claim is not merely that hierarchy is present, but that hierarchical coordination is stabilized by conditioning both levels on explicit within-subtask phase information and by furnishing the high-level policy with persistent visual memory. A common misconception is that this reduces to action chunking with additional observations. The paper’s architecture indicates otherwise: the high-level policy predicts commands, progression, and keyframe scores, while the low-level policy is explicitly conditioned on the resulting phase and instruction variables rather than inferring all temporal structure from raw history alone (Buamanee et al., 4 Mar 2026).

3. Bilateral control, multimodality, and demonstration acquisition

Bi-HIL uses four-channel bilateral control to collect demonstrations. A human directly operates the leader robot, while the follower robot interacts with the environment and feeds force information back to the leader. The control objectives are

$\theta_l - \theta_f = 0$

and

$\tau_l + \tau_f = 0,$

where $K_t$0 are leader and follower joint angles and $K_t$1 are leader and follower torques. The first equation enforces position synchronization and the second force consistency via action-reaction. The paper also states that disturbance torques are estimated with a DOB and reaction torques are inferred with an RFOB (Buamanee et al., 4 Mar 2026).

This bilateral setup defines the multimodal character of the method. The low-level controller is not trained only from visual observations and kinematic trajectories; it is trained from demonstrations in which contact-force information is embedded in the teleoperation process. The framework therefore addresses contact-rich manipulation by coupling visual context, language-like subtask conditioning, and force-rich actuation targets within a single imitation learning pipeline (Buamanee et al., 4 Mar 2026).

The paper evaluates this approach on both unimanual and bimanual real-robot tasks. The unimanual platform is OpenMANIPULATOR-X in a leader/follower setup with a 4 DOF arm plus gripper, totaling 5 actuated joints, running bilateral control at 1000 Hz with two RGB cameras at 100 Hz. The bimanual platform is ALPHA-α, using four robots in total—two leaders and two followers—with each robot having 6 DOF plus gripper, totaling 7 motors, bilateral control at 1000 Hz, and four RGB cameras (Buamanee et al., 4 Mar 2026).

4. Learning objectives, conditioning mechanisms, and internal representations

The high-level training objective is a weighted sum of three losses: cross-entropy for subtask command prediction, MAE for progress-rate prediction, and weighted BCE for keyframe prediction. The paper denotes these as $K_t$2, $K_t$3, and $K_t$4, combined through balancing weights $K_t$5. This design makes the high-level module jointly responsible for symbolic subtask selection, continuous phase estimation, and memory management (Buamanee et al., 4 Mar 2026).

The low-level objective combines an action imitation term and a latent regularization term. Concretely, the paper specifies an L1 error between predicted and target action chunks together with a KL divergence term weighted by $K_t$6. The low-level policy is therefore framed as a stochastic sequence model that imitates demonstrated motion while retaining the capacity to represent uncertainty (Buamanee et al., 4 Mar 2026).

The paper’s explanatory argument for why the combination works is precise. Keyframe memory supplies persistent anchors for completed subtasks. The subtask-level progress rate informs both policies where execution lies within the current subtask. Bilateral control supplies force-rich demonstrations and low-level execution cues. The low-level policy then benefits from explicit phase guidance instead of inferring all coordination from raw observations. This is presented as the mechanism by which ambiguity at subtask boundaries is reduced and robustness in contact-rich tasks is improved (Buamanee et al., 4 Mar 2026).

The role of ablations is important in interpreting the method. The unimanual study includes Bi-HIL (w/o KF+SPR), Bi-HIL (w/o SPR), Bi-HIL (w/o KF), and the full Bi-HIL (Proposed). The paper states that without keyframe memory transitions become less stable, without progress rate the model lacks phase guidance, and without either hierarchical coordination is much worse. This supports the claim that Bi-HIL is defined by the joint use of keyframe memory and subtask-level progress rate rather than by hierarchy alone (Buamanee et al., 4 Mar 2026).

5. Real-robot tasks, baselines, and empirical findings

The unimanual task is Put-Three-Balls-in-Drawer, in which green, red, and white balls must be placed into matching drawers. The task contains 12 subtasks, lasts about 59.2 s, and includes visually similar subtasks and ambiguous initial configurations. Two configurations are used, Left and Right, with ordering determined by initial ball placement. Training uses 6 bilateral demonstrations with manually annotated subtask boundaries, augmented with DABI to 60 demonstrations total (Buamanee et al., 4 Mar 2026).

The bimanual tasks are 6-Cup Downstack and 4-Peg-in-Hole. Both are described as contact-rich and requiring repeated coordinated pick-and-insert behaviors. For each task, 5 demonstrations are collected and augmented to 50 demonstrations with DABI, and the same policy settings as in the unimanual experiments are used (Buamanee et al., 4 Mar 2026).

The baselines are Bi-ACT and, for bimanual experiments, Bi-ACT (w/o Force). The reported success rates are as follows.

Task	Baseline results	Bi-HIL result
Put-Three-Balls-in-Drawer	Bi-ACT: Left 60%, Right 100%	Left 100%, Right 100%
6-Cup Downstack	Bi-ACT (w/o Force): 20%; Bi-ACT: 60%	80%
4-Peg-in-Hole	Bi-ACT (w/o Force): 0%; Bi-ACT: 20%	80%

The paper interprets these results in task-specific terms. In the unimanual task, the Left configuration is described as harder because the order of picking balls changes, demanding stronger temporal reasoning. Bi-HIL is reported as the only method that succeeds on both configurations. In 6-Cup Downstack, force information improves local execution, but a flat policy still struggles in later stages where multiple coordinated subtasks must be chained; Bi-HIL improves performance particularly in the later merging stages. In 4-Peg-in-Hole, a vision-only policy fails entirely, force-aware flat control helps, and hierarchical coordination plus progress-aware phase modeling yields the strongest robustness (Buamanee et al., 4 Mar 2026).

The result visualizations are described as showing smoother transitions in predicted subtasks and progress rate over time, with keyframes concentrated near subtask boundaries. This suggests that the method’s explicit phase and memory variables are not only architectural additions but also learned latent organizers of task progression (Buamanee et al., 4 Mar 2026).

6. Significance, limitations, and terminological scope

Within robotic imitation learning, Bi-HIL is significant because it frames long-horizon contact-rich manipulation as a coordination problem across subtasks rather than solely as a low-level control problem. Its contribution is not a generic hierarchical decomposition, but a specific combination of force-aware bilateral demonstrations, keyframe-based memory, and explicit subtask progression. The empirical findings support the paper’s stated claim that hierarchical reasoning alone is not enough and force-aware control alone is not enough (Buamanee et al., 4 Mar 2026).

The paper does not provide an extensive limitations section. This suggests potential constraints already implicit in the experimental setup: dependence on manual subtask boundary annotations, reliance on demonstration augmentation, evaluation on a limited set of real-robot tasks, and the absence of an explicit study of fully automatic subtask discovery or large-scale generalization beyond the tested manipulation tasks. The same discussion suggests natural future directions such as automatic subtask segmentation, broader task and morphology generalization, stronger memory retrieval policies, and integration with larger multimodal foundation models (Buamanee et al., 4 Mar 2026).

The term Bi-HIL is also not globally unique across arXiv usage. In "Selective Deployment of Bidirectional Hollow-Core Fibers in Hybrid SMF/HCF Optical Networks," the term refers instead to a hybrid optical network in which only a selected subset of links is upgraded from legacy single-mode fiber to bidirectional hollow-core fiber, specifically the hybrid Uni-SMF/BiDi-HCF scenario (Ibrahimi et al., 29 Jun 2026). In the robotics literature discussed here, however, Bi-HIL denotes Bilateral Control-Based Multimodal Hierarchical Imitation Learning via Subtask-Level Progress Rate and Keyframe Memory for Long-Horizon Contact-Rich Robotic Manipulation. The acronym therefore requires domain-sensitive interpretation rather than a single universal expansion.