---
title: Hybrid Internal Model for Locomotion
url: https://www.emergentmind.com/topics/hybrid-internal-model-him
type: topic
---

# Hybrid Internal Model for Locomotion

Searching arXiv for recent and relevant papers on "Hybrid Internal Model" and related locomotion/control formulations.
Hybrid Internal Model (HIM) denotes a family of learned internal-model architectures that use the robot’s own response to compensate for partial observability in control. In contemporary legged locomotion, HIM is formulated as a proprioception-driven module that maps a short observation history to two outputs: an explicit velocity estimate and an implicit latent representation of stability or successor-state information, which are then appended to the policy input. Closely related variants extend the same principle with perceptual terrain inputs for humanoid locomotion and with dual temporal horizons for low-gravity quadrupedal jumping [2312.11460], [2411.14386], [2603.07999]. An earlier use of the same term in a different control context referred instead to a supervised forward model plus inverse model within an arbitrated planning–habit architecture for a simulated robotic arm [1712.02441].

## 1. Terminology and research lineages

The label “Hybrid Internal Model” has been used in at least two technically distinct ways. In Fard and Trappenberg’s 2017 work on arbitration between planning and habitual control, the HIM is the supervised-learning subsystem of the larger Arbitrated Predictive Actor–Critic architecture, consisting of a forward model and an inverse model trained to capture plant kinematics [1712.02441]. In legged locomotion work from 2023 onward, HIM instead denotes a learned response model that extracts an explicit velocity prediction together with an implicit latent embedding from proprioceptive histories [2312.11460].

| Paper | HIM meaning | Primary role |
|---|---|---|
| “A Novel Model for Arbitration between Planning and Habitual Control Systems” [1712.02441] | Forward model + inverse model | Deliberative planning controller |
| “Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response” [2312.11460] | Velocity prediction + latent response embedding | Policy augmentation for quadrupedal locomotion |
| “Learning Humanoid Locomotion with Perceptive Internal Model” [2411.14386] | Same HIM backbone used inside PIM | Humanoid locomotion with terrain-aware extension |
| “Dual-Horizon Hybrid Internal Model for Low-Gravity Quadrupedal Jumping with Hardware-in-the-Loop Validation” [2603.07999] | Two-branch temporal HIM | Continuous jumping under lunar gravity |

In the 2023–2026 locomotion line, the “hybrid” designation does not refer to discrete switching or piecewise dynamics. In the humanoid formulation, the term refers to combining explicit prediction, such as \(\hat v\), with an implicit latent \(l\), both produced by a single neural network; the paper explicitly states that there are no discrete or piecewise switches inside \(f_\phi\) [2411.14386]. By contrast, the 2017 usage is hybrid in the sense of combining planning and habitual control through an arbitrator [1712.02441].

## 2. Core formulation in legged locomotion

The modern HIM formulation is grounded in classical Internal Model Control. The 2023 quadrupedal paper treats unobserved environmental properties such as terrain height, friction, and restitution as disturbances, and estimates the robot’s response rather than regressing raw terrain parameters [2312.11460]. From a history \(\mathbf{o}^a_{t-H:t}\), with \(H=5\) in practice, the extractor computes
$$
\bigl(\hat{\mathbf v}_t,\hat{\boldsymbol\ell}_t\bigr)=\Phi(\mathbf{o}^a_{t-H:t}),
\qquad
\hat{\mathbf v}_t\in\mathbb R^3,\;
\hat{\boldsymbol\ell}_t\in\mathbb R^{16}.
$$
Here \(\hat{\mathbf v}_t\) is an explicit velocity prediction for the base \([v_x,v_y,\omega_{\mathrm{yaw}}]\), and \(\hat{\boldsymbol\ell}_t\) is an implicit stability or response latent vector [2312.11460].

The 2024 humanoid formulation presents the same basic pattern with slightly different indexing and observation notation. HIM is written as
$$
f_\phi:\{o_t,\dots,o_{t-h}\}\to (\hat v_{t+1}, l_t),
$$
where the proprioceptive observation is
$$
o_t^n=[\theta_t,\dot\theta_t,\omega_t,g_t,a_{t-1}],
$$
the history \(o_{t-h:t}^n=\{o_{t-h}^n,\dots,o_t^n\}\) is fed into \(f_\phi\), \(\hat v_{t+1}\in\mathbb R^3\) is the predicted Cartesian linear velocity at \(t+1\), and \(l_t\in\mathbb R^d\) is a latent embedding intended to capture the next-step proprioceptive successor state [2411.14386]. The paper also states that there is no explicit action inside HIM beyond the last action \(a_{t-1}\) included in \(o_t^n\).

A central design choice is that HIM operates on proprioceptive histories only. In the 2023 quadruped system, those proprioceptions are joint encoders, IMU signals, and last action; in the 2024 humanoid system, the observation includes joint positions, joint velocities, base angular velocity, gravity direction, and previous action [2312.11460], [2411.14386]. This makes HIM a deployable internal model rather than a privileged teacher.

## 3. Losses, representation learning, and optimization

HIM is trained through a combination of explicit regression and contrastive representation learning. In the 2023 formulation, the velocity regression loss is
$$
\mathcal L_{\mathrm{vel}}=\|\hat{\mathbf v}_t-\mathbf v_t\|_2^2,
$$
where \(\mathbf v_t\) is the ground-truth base velocity available in simulation. The latent is trained with a SwAV-style prototypical contrastive loss that aligns \(\hat{\boldsymbol\ell}_t\) with the successor full-observation embedding of \(\mathbf{o}^a_{t+1}\), using source and target encoders, \(K\) prototypes, soft cluster assignments, and Sinkhorn-Knopp target assignments. The total Hybrid Internal Optimization loss is
$$
\mathcal L_{\mathrm{HIO}}
=
\lambda_{\mathrm{vel}}\mathcal L_{\mathrm{vel}}
+
\lambda_{\mathrm{ctr}}\mathcal J_{\mathrm{SwAV}},
$$
with \(\lambda_{\mathrm{vel}}=\lambda_{\mathrm{ctr}}=1.0\) [2312.11460].

The 2024 humanoid formulation preserves the same two-part structure. Its regression term is
$$
L_{\mathrm{vel}}=\|\hat v_{t+1}-v^{\mathrm{gt}}_{t+1}\|_2^2,
$$
and its contrastive term treats \((o^n_{t-h:t},o^n_{t+1})\) from the same trajectory as a positive pair and all other combinations as negatives, implemented as
$$
L_{\mathrm{contrastive}}
=
-
\sum_{(i,j)\in\mathrm{Pos}}
p(c|o^{n,(i)})\cdot \log q(c|o^{n,(j)}).
$$
The total loss is
$$
L_{\mathrm{HIM}}=\alpha L_{\mathrm{vel}}+\beta L_{\mathrm{contrastive}}.
$$
No further \(L_2\) or weight-decay regularizers are mentioned for HIM in that paper [2411.14386].

Optimization is similarly direct. The 2023 quadruped system alternates between Hybrid Internal Optimization and PPO-based policy learning [2312.11460]. The 2024 humanoid system states that HIM is trained separately, off-policy, from the policy, using Adam on \(L_{\mathrm{HIM}}\), with batch-level SwAV updates for the contrastive component [2411.14386]. In both cases, the contrastive term uses batch-level information, so other trajectories in the batch act as negatives.

## 4. Policy integration, simulation–reality consistency, and perceptual extension

In the 2023 quadruped architecture, the actor input is the concatenation \([\mathbf o^a_t,\hat{\mathbf v}_t,\hat{\boldsymbol\ell}_t]\), while the critic at train time additionally receives privileged inputs \(\mathbf f_t,\mathbf h_t\). The actor and critic are each 4-layer MLPs with hidden dimensions \([512,512,256,128]\), and the output is a joint-position bias \(\mathbf a_t\in\mathbb R^{12}\) [2312.11460]. Because the deployable policy uses only proprioceptions together with HIM outputs, there is no teacher-student or mimic-loss stage. The same paper emphasizes that HIM maintains consistent observations between simulation reference and reality, thereby avoiding information loss in mimicking learning [2312.11460].

The 2024 humanoid work introduces the Perceptive Internal Model (PIM) as an extension of exactly the same architecture. Pure HIM uses only proprioception:
$$
(\hat v_{t+1},l_t)=f_\phi(o^n_{t-h:t}),
$$
whereas PIM replaces the history by the concatenated sequence \((o^n_{t-h:t},p_t)\), where \(p_t\) is the sampled elevation heights:
$$
(\hat v_{t+1},l_t)=f_\phi(o^n_{t-h:t}\parallel p_t).
$$
The paper states that HIM is first pre-trained, or alternately co-trained with the policy, and that in PIM the same network \(f_\phi\) is frozen when updating the policy. After each PPO policy update, trajectories are used to fine-tune \(f_\phi\) on the HIM losses with perceptive inputs included. During inference, both HIM and PIM are disabled as trainable modules and simply produce \(\hat v_{t+1}\) and \(l_t\) to feed the policy network alongside the raw observation \(o_t\) [2411.14386].

The perceptual augmentation is not based on raw depth maps or point clouds. Instead, PIM relies on onboard, continuously updated elevation maps centered around the robot and samples terrain heights from that map at inference [2411.14386]. A plausible implication is that the HIM backbone is intended to remain unchanged while the sensory front end is modified to suit the locomotion regime.

## 5. Empirical behavior across terrains, disturbances, and morphologies

The 2023 quadruped paper reports a data-efficient training schedule in NVIDIA Isaac Gym with 4,096 parallel environments, 100-step rollouts, and 1,000 PPO rollouts for initial convergence, amounting to approximately 1 hour on an RTX 4090 and about 200 M simulator steps [2312.11460]. The same source states that prior work required 600–1,280 M steps. Real-world benchmarks include stairs, unseen compositional terrain, deformable slope, and disturbances such as a 10 kg dragging obstacle, an 8 kg pendulum lateral hit, an 8 kg payload, and missing steps with a 42 cm drop. On long-range stairs, HIM achieves \(176.5\pm 7.8\) steps versus \(75.3\pm 20.0\) for RMA, and simulation tracking on Aliengo yields linear or angular error \(0.071\)–\(0.147\) m/s for HIM versus \(0.158\)–\(0.377\) m/s for RMA [2312.11460].

The humanoid PIM paper reports a comparison between HIM and the terrain-aware extension using estimation loss and traversable terrain level. In the plotted training curves, HIM’s estimation loss plateaus at around \(0.15\) m/s and enables traversal of terrain level approximately \(2\), corresponding to small steps, whereas PIM’s loss drops to \(0.08\) m/s and supports terrain level approximately \(4\), described as continuous stairs up to \(15\) cm [2411.14386]. The same paper states that depth map rendering is not required in simulation and that the policy can train in 3 hours on an RTX 4090 GPU.

Ablation results in the 2023 work clarify the role of each HIM component. Removing the velocity input or velocity loss causes an approximately \(20\)–\(50\%\) drop in success rates on stairs and unseen terrains. Removing the latent input or latent loss causes catastrophic collapse, including stairs success falling to \(50\%\). Replacing the contrastive objective with regression produces less separable latent t-SNE clusters and approximately \(30\%\) worse tracking [2312.11460]. These results are consistent with the architectural claim that HIM is hybrid because both explicit velocity and implicit latent response are required.

## 6. Dual-horizon extension and low-gravity jumping

The 2026 low-gravity quadrupedal system introduces a Dual-Horizon Hybrid Internal Model that uses proprioceptive sensing only and separates fast and slow jump-cycle dynamics into two temporal encoders [2603.07999]. The short-horizon branch processes 6 recent frames, approximately \(0.12\) s at \(50\) Hz, explicitly regresses vertical velocity \(\hat v_z\), and emits a latent vector \(h_s\). The long-horizon branch processes 15 subsampled frames over approximately \(0.9\) s, explicitly regresses horizontal velocity \((\hat v_x,\hat v_y)\) and center-of-mass height \(\hat h\), and emits a latent vector \(h_l\). These are fused as
$$
z_t=[\hat v_x,\hat v_y,\hat v_z,\hat h,h_s,h_l],
$$
which is appended to the current observation \(o_t\) before policy inference [2603.07999].

This variant keeps the HIM pattern of explicit regression plus latent predictive representation, but distributes it over two horizons. Its regression loss sums squared error over \(v_x,v_y,v_z,h\), and each branch has an InfoNCE contrastive objective with offsets \(\delta_s=1\) and \(\delta_l=6\). The combined loss is
$$
\mathcal L_{\mathrm{DH}}
=
\lambda_{\mathrm{reg}}\mathcal L_{\mathrm{reg}}
+
\lambda_s\mathcal L_s
+
\lambda_l\mathcal L_l.
$$
The same paper adds phase-adaptive gating with binary indicators based on CoM height and vertical velocity, but explicitly states that no contact sensor is used [2603.07999].

Reported results show that Dual-Horizon outperforms Short-only and Long-only models in both estimation and jumping performance. In Table I, Dual-Horizon achieves \(v_z\) MSE \(0.031\), \(v_x\) MSE \(0.010\), \(v_y\) MSE \(0.009\), and \(h\) MSE \(0.012\), compared with \(0.041,0.013,0.013,0.028\) for Short-only and \(0.067,0.012,0.011,0.014\) for Long-only. The same table reports survival \(18.9\) s, landing success rate \(86.7\%\), and height error \(0.026\) m for Dual-Horizon, versus \(13.6\) s, \(71.2\%\), \(0.054\) m for Short-only, \(14.8\) s, \(74.5\%\), \(0.041\) m for Long-only, and \(9.7\) s, \(58.4\%\), \(0.073\) m without the phase reward [2603.07999]. Hardware-in-the-loop validation is conducted on the MATRIX platform, which couples a pulley-counterweight gravity offloading system with a digital-twin-driven motion platform and treadmill.

## 7. Conceptual interpretation and common misunderstandings

Several recurrent misunderstandings are resolved directly by the source texts. First, HIM in the locomotion literature is not a terrain-parameter regressor. The 2023 paper states that it estimates the robot’s response rather than raw terrain parameters, treating external states such as terrain frictions and elevation maps as disturbances [2312.11460]. Second, “hybrid” in the 2024 humanoid formulation does not mean a switched or piecewise model; all predictions are produced by a single neural network, and the hybrid aspect lies in combining explicit velocity prediction with an implicit latent successor-state embedding [2411.14386]. Third, the term does not denote a single canonical architecture across the literature: the 2017 robotic-arm HIM is a forward-plus-inverse supervised planning module, whereas the 2023–2026 legged-locomotion HIM is a learned response representation appended to policy observations [1712.02441], [2603.07999].

Within the legged-locomotion lineage, however, a stable pattern is visible. HIM repeatedly appears as an internal-model backbone that extracts deployable state estimates from proprioceptive histories, and then augments the control policy with both explicit and implicit predictive variables. This suggests that the main contribution of HIM is architectural rather than tied to a single embodiment: quadrupedal rough-terrain traversal, humanoid stair climbing with elevation maps, and low-gravity pronking all retain the same basic coupling of explicit kinematic regression and implicit predictive embedding [2312.11460], [2411.14386], [2603.07999].

Source: https://www.emergentmind.com/topics/hybrid-internal-model-him