---
title: Executable Motion Prior (EMP)
url: https://www.emergentmind.com/topics/executable-motion-prior-emp
type: topic
---

# Executable Motion Prior (EMP)

Executable Motion Prior (EMP) is a robotics and motion-modeling concept in which a motion representation is constructed so that it is not merely descriptive, generative, or predictive, but directly usable in execution. In the robotics usage, EMP denotes a motion prior that can be issued as control-relevant commands, embedded into a physics solver, edited online under scene constraints, or translated into an embodiment-specific action interface. Across recent work, EMP appears as a stable dynamical system learned from a single demonstration, as a sampled transition manifold coupled to projective dynamics, as a state-conditioned motion-correction module for humanoid standing imitation, and, in closely related formulations, as an executable intermediary between imagined tool motion and robot action or as a compositional motion program executed through zero-shot policy inference [2503.08029; 2309.13742; 2507.15649; 2601.18323; 2604.18064].

## 1. Conceptual scope and defining properties

The common structure of EMP is the conjunction of a **prior** and an **execution interface**. The prior captures reusable motion structure: for example, a learned SE(3) dynamical system, a sampled manifold of plausible next states, a corrected goal trajectory, a tool trajectory recovered from generated video, or a symbolic motion program. The execution interface makes that structure operative: direct end-effector velocity commands, a projective-dynamics energy term, a pre-controller goal filter, a rigid-body kinematic transform, or a reward-conditioned policy inference mechanism.

| Work | Prior representation | Executability mechanism |
|---|---|---|
| "Elastic Motion Policy" [2503.08029] | SE(3) LPV-DS from a single demonstration | Direct velocity and angular-velocity policy with Lyapunov constraints |
| "DROP" [2309.13742] | Sampled transition manifold from a pretrained motion prior | Motion prior embedded as \(E_{kin}\) in projective dynamics |
| "EMP: Executable Motion Prior for Humanoid Robot Standing Upper-body Motion Imitation" [2507.15649] | Desired upper-body target motion | State-conditioned target correction before RL tracking |
| "TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion" [2601.18323] | Imagined tool trajectory from a world model | Metric 3D tool motion translated into 6-DoF TCP actions |
| "Understanding Human Actions through the Lens of Executable Models" [2604.18064] | ExAct motion programs | Reward-generating functions for zero-shot policy inference |

A recurring implication is that EMP is better understood as a design principle than as a single standardized architecture. In all of these formulations, executability requires a representation that is already close enough to control, geometry, or physics that the remaining gap to action can be closed analytically, convexly, or with a lightweight downstream module rather than a monolithic end-to-end policy.

## 2. Formal patterns of executability

Recent EMP formulations instantiate executability in three principal ways. The first is **stability-constrained policy representation**. In Elastic Motion Policy, translation is modeled by an LPV-DS,
$$
\dot{x}=\sum_{k=1}^{K}\gamma_k(x)\mathbf{A}_k(x-x^*)
$$
with quadratic Lyapunov function
$$
V(x) = (x-x^*)^T \mathbf{P}(x-x^*) \qquad \mathbf{P}=\mathbf{P}^T\succ0,
$$
and constraints
$$
\mathbf{A}_k^T\mathbf{P}+\mathbf{P}\mathbf{A}_k=\mathbf{Q}_k,\qquad \mathbf{Q}_k=\mathbf{Q}_k^T\prec0.
$$
Here executability is identified with global asymptotic stability, online editability, and direct use as an end-effector motion policy [2503.08029].

The second is **physics-coupled execution**. DROP treats the prior not as a trajectory tracker but as a soft manifold constraint inside an implicit Euler / projective dynamics solve:
$$
\mathbf{x}_{t+1} = \arg\min_{\mathbf{x}} \frac{1}{2h^2}(\mathbf{x}-\mathbf{y})^T\mathbf{M}(\mathbf{x}-\mathbf{y}) +\sum_i E_i(\mathbf{x};\mathbf{x}_t,\mathbf{v}_t),
$$
with the learned motion-prior term
$$
E_{kin} = \frac{w_{kin}}{2}\min_{\mathbf{x}_{kin}\in\mathcal{X}_{t+1}} \|\mathbf{x}-\mathbf{x}_{kin}\|^2.
$$
In this case the prior becomes executable because it enters the same objective as momentum, contact, rigidity, and range-of-motion constraints [2309.13742].

The third is **state- or geometry-conditioned correction**. In humanoid standing imitation, EMP receives the current robot state \(\mathbf{s}_t\) and raw upper-body goal \(\mathbf{g}_t\), and predicts an adjusted target
$$
\hat{\mathbf{g}}_t = f_\theta(\mathbf{s}_t,\mathbf{g}_t).
$$
Executability is therefore not assigned to the original target, but to its corrected version after safety, balance, collision, centroid, and smoothness criteria are enforced [2507.15649].

These patterns differ operationally, but they share an architectural claim: a motion prior becomes executable when it is represented in the same space as the constraints that matter for deployment—stability certificates, metric geometry, contact mechanics, or embodiment-conditioned action variables.

## 3. Elastic Motion Policy and scene-adaptive single-demonstration execution

Elastic Motion Policy presents EMP explicitly as a one-shot imitation-learning framework. It follows the dynamical-systems paradigm, learns a stable SE(3) LPV-DS from a single demonstration, and adapts that policy online by geometric editing rather than retraining. The translational component is learned in \(\mathbb{R}^3\), the rotational component on the quaternion manifold, and both are constrained for global asymptotic stability [2503.08029].

Its distinctive mechanism is **Laplacian editing in full end-effector space \(\mathbb{R}^3\times SO(3)\)**. For the translational part, the edited motion geometry is computed through a Laplacian objective
$$
\min_{\beta_i}\ \|L\beta_i-\Delta\|_2^2,
$$
subject to endpoint constraints tied to the start and end object poses. For orientation, the method works on the tangent plane of \(S^3\), projects quaternion means and covariances into a reduced Euclidean space, performs Laplacian editing there, and maps the result back to quaternion space. This is the formal basis for the paper’s claim that the policy can be morphed online while preserving the motion’s local geometric structure [2503.08029].

A second contribution is **online convex learning of Lyapunov functions**. The paper replaces older nonconvex P-QLF fitting with the convex objective
$$
\min_{\mathbf{P}} \sum_{i=1}^{N} \mathrm{ReLU}\!\left(\dot{x}_i^T \mathbf{P}(x_i-x^*)\right)
\quad \text{subject to} \quad
\mathbf{P}\succeq \epsilon I.
$$
On LASA handwriting motions, the reported fitting times and violation rates are: for a single trajectory, baseline P-QLF **0.332 s**, **14.0%** violation; convex P-QLF **0.038 s**, **11.1%** violation; GMM-informed P-QLF **0.007 s**, **15.1%** violation. For all trajectories, the reported numbers are **2.62 s**, **14.9%**; **0.24 s**, **12.3%**; and **0.09 s**, **15.4%**, respectively [2503.08029].

The empirical evaluation uses a **Franka Research 3** arm, a customized **UMI gripper**, RGB-D sensing, **AprilTags** for frame alignment, and pretrained perception modules including **GPT-4o**, **Grounded SAM**, and **FoundationPose**. The baseline is object-centric **SE(3)-LPVDS**. In-distribution results are: Book Placing **10/10** for both methods, Cube Pouring **10/10** for both, and Pick-and-Place **8/10** for SE(3)-LPVDS versus **7/10** for EMP. Out-of-distribution results are: Book Placing **4/10** versus **8/10**, Cube Pouring **4/10** versus **9/10**, and Pick-and-Place **1/10** versus **7/10**. The obstacle-avoidance case study further combines EMP with DS modulation,
$$
\dot{x}_{new} = \mathbf{M}(x)f_p(x),
$$
showing that the edited prior can compose with classical obstacle-avoidance machinery [2503.08029].

In this formulation, EMP is not a latent style prior or a demonstration replay model. It is a stable motion law whose geometry can be re-edited online to satisfy new task constraints without collecting new demonstrations.

## 4. Physics-embedded and embodiment-conditioned EMPs

DROP and humanoid standing imitation represent two distinct but closely aligned routes to executability. DROP turns a pretrained kinematic motion prior into what the paper explicitly characterizes as an EMP by embedding the learned next-state distribution into a projective-dynamics simulator. The prior is an autoregressive generative model,
$$
\tilde{\mathbf{x}}_{t+1}=\mathcal{M}(\mathbf{x}_t,\mathbf{x}_{t-1},\ldots,\mathbf{z}),
$$
sampled repeatedly to approximate a transition manifold \(\hat{\mathcal{X}}_{t+1}\). In implementation, the motion prior is **HuMoR**, trained on **AMASS**. The body is modeled as a minimal ragdoll stick figure, and the total objective combines momentum, rigidity, contact, range of motion, and \(E_{kin}\) [2309.13742].

This design makes the prior executable because it exerts a control-like pull while remaining balanced against Newtonian dynamics and contact constraints. The solver alternates between sampling the prior, projecting the current state to the sampled manifold, projecting physics constraints, and solving the global quadratic system. The framework also introduces a soft correction for “magic” root wrench, using center-of-mass linear and angular momentum change to reduce puppet-like balance recovery [2309.13742]. The reported behaviors include push recovery, falling and getting up, dodging projectiles, two-character collision responses, tripping on obstacles, target and direction following, adaptation to a tilting platform, and behavior changes from skeleton modifications.

The humanoid imitation paper uses EMP in a narrower but highly operational sense. Its target setting is a standing humanoid performing upper-body actions such as grasping, hammering, or screwing while maintaining whole-body stability. Here EMP is a **learned, state-conditioned motion filter** inserted before the RL controller. The core mapping is
$$
\hat{\mathbf{g}}_t = f_\theta(\mathbf{s}_t,\mathbf{g}_t),
$$
where \(\mathbf{s}_t\) is the robot state and \(\mathbf{g}_t\) the desired upper-body motion target. EMP is implemented as a **VAE-style latent model** with state encoder \(f_s\), target encoder \(f_t\), fusion network \(f_\psi\), latent prior \(\mathbf{z}\sim\mathcal{N}(0,1)\), and a decoder that outputs the corrected target. A learned world model
$$
\hat{\mathbf{s}}_{t+1} = f_w(\mathbf{s}_t,\mathbf{a}_t)
$$
with prediction loss
$$
L_{\text{pre}} = \left\|\mathbf{s}_{t+1}-\hat{\mathbf{s}}_{t+1}\right\|_2^2
$$
provides differentiable rollouts for training [2507.15649].

The EMP training objective is
$$
L = \lambda_{\text{rec}}L_{\text{rec}} + \lambda_{\text{ori}}L_{\text{ori}} + \lambda_{\text{col}}L_{\text{col}} + \lambda_{\text{cen}}L_{\text{cen}} + \lambda_{\text{smo}}L_{\text{smo}} + \lambda_{\text{reg}}L_{\text{reg}},
$$
with weights \(\lambda_{\text{rec}}=20\), \(\lambda_{\text{ori}}=10\), \(\lambda_{\text{col}}=1\), \(\lambda_{\text{cen}}=10\), \(\lambda_{\text{smo}}=100\), and \(\lambda_{\text{reg}}=1\). The losses respectively preserve the original motion, encourage upright base orientation, penalize self-collision among \(\{\text{torso, hand, sacrum, thigh}\}\), keep the centroid inside the support region, enforce temporal smoothness, and regularize the latent code [2507.15649].

The simulation comparison includes Privileged Policy, Whole-Body Policy, Decoupled Policy, PMP baseline, EMP, and “EMP when Danger.” From Table 1, EMP reaches **98.1%** success rate, compared with **97.0%** for Decoupled Policy and **97.4%** for PMP. It reduces MSC to **0.1494**, versus **0.3668** and **0.3741**, and preserves upper-body motion quality with MJP **0.8221** versus **0.8295** for the best decoupled baseline. The ablation study shows that removing the smoothness loss drops success to **27.0%**, removing the orientation loss drops success to **2.6%**, and removing the centroid loss drops success to **10.7%**. On hardware, the RL policy and EMP run at **50 Hz**, the PD controller at **1 kHz**, and the system relies only on onboard proprioception [2507.15649].

Taken together, these two formulations show that EMP can be placed at different interfaces: inside the simulator as a soft manifold constraint, or before the controller as a target executability filter. This suggests that “executability” is not tied to one control stack location, but to whichever representation directly mediates between desired motion and feasible embodied behavior.

## 5. EMP-like intermediaries between imagination, action, and compositional programs

TC-IDM is explicitly described as being very close in spirit to an EMP, with the important distinction that it does not learn a monolithic motion prior directly from robot trajectories. Instead, it treats the world model’s **imagined tool trajectory** as the structured intermediate representation between visual planning and action. The motivating problem is the “last-mile” gap between pixel-level plans and physically executable robot control: generated videos may contain hallucinations, viewpoint-dependent distortions, occlusions, or kinematic inconsistencies, and a robot cannot execute a sequence of RGB frames [2601.18323].

The pipeline begins with a generated RGB video sequence conditioned on the initial observation and task instruction. The method uses **SAM 3** for gripper or tool segmentation, **VGGT** with depth completion/alignment for metric depth and metric camera trajectory recovery, and then 3D point tracking in the generated video. Dense tracked trajectories are defined as
$$
\mathcal{T}_{\text{dense}}= \text{PointTracker}\!\left( V_{\text{spatial}}, M_{\text{gripper}} \right),
$$
followed by rigid-motion filtering
$$
\mathcal{T}_{\text{gripper}}= \text{Select}_{\text{rigid}}\!\left(\mathcal{T}_{\text{dense}}\right).
$$
The executable translation step is analytic:
$$
(\mathbf{R}_t, \mathbf{p}_t) = \arg\min_{\mathbf{R} \in SO(3),\, \mathbf{p} \in \mathbb{R}^3} \sum_{i=1}^{K} \left\| \mathbf{R}\,\mathbf{x}_i^t + \mathbf{p} - \mathbf{x}_i^{t+1} \right\|^2,
$$
and the recovered rigid transform is interpreted as the 6-DoF end-effector action,
$$
\text{A}_{TCP}^{t+1} = (\mathbf{R}_t, \mathbf{p}_t).
$$
The architecture is decoupled into a vision-driven state generation branch, using frozen **DINOv3** features and an MLP **GripperHead** for gripper aperture, and a geometry-grounded gesture generation branch for 6-DoF TCP motion [2601.18323].

The reported real-world results are an average success rate of **61.11 percent**, with **77.7 percent on simple tasks** and **38.46 percent on zero-shot deformable object tasks**. In replay evaluation, TC-IDM reports **93.3%** easy-task replay accuracy, **53.3%** medium-task replay accuracy, and **28.9%** hard-task replay accuracy, and the paper states that easy-task replay accuracy is the upper bound for task success in that evaluation. The method is reported to outperform ResNet-MLPs, AVDC, AnyPos, and a 2D-tracker IDM, and to remain effective across different world models [2601.18323]. In EMP terms, this is an executable intermediary: the imagined future becomes control-ready only after being lifted into metric 3D tool motion.

A second EMP-like extension appears in executable action models for human understanding. The **ExAct** DSL represents motion as underspecified programs, with grammar
$$
\langle \text{program} \rangle ::= \langle \text{motion} \rangle \mid \langle \text{motion} \rangle ; \langle \text{motion} \rangle,
$$
$$
\langle \text{motion} \rangle ::= [t_1,t_2] \, \langle \text{sensors} \rangle,
$$
$$
\langle \text{sensor} \rangle ::= \mathtt{s}^{pos}(x).
$$
Programs are compiled into reward-generating functions and executed through forward-backward representations with
$$
Q_\phi(s_t,a_t)=F(s_t,a_t)^\top r_t^\phi.
$$
The parser uses an **ST-GCN encoder** and a **Qwen2.5-Coder-3B decoder** fine-tuned with **LoRA**, and the parser objective is
$$
L = L_{\rm LM} + \lambda L_{\rm InfoNCE}.
$$
The evaluation covers **HumanAct12** and **EPFL-Smart-Kitchen**, with synthetic motion programs up to **1024 timesteps**, and reports that executable action models improve data efficiency and capture intuitive relationships between actions in segmentation and anomaly detection settings [2604.18064].

These two lines of work broaden the EMP concept beyond imitation learning. In one case, the executable object is a tool trajectory extracted from a generated video; in the other, it is a reward-generating motion program. Both preserve the defining property that the prior is operational rather than merely descriptive.

## 6. Terminological boundaries, misconceptions, and limitations

A common source of confusion is acronym overload. In other domains, **EMP** means **exciton magnetic polaron** in Mn-doped perovskite nanorods [1612.02517] or **extremely metal-poor** stars in Galactic archaeology [2311.02297]. Those usages are unrelated to robotics and motion priors. For literature retrieval, this ambiguity is nontrivial.

Within robotics, another misconception is that an EMP must be a single end-to-end neural policy trained directly from robot trajectories. The recent literature does not support that restriction. Elastic Motion Policy uses a stable dynamical system and Laplacian editing rather than end-to-end behavior cloning [2503.08029]. DROP keeps the motion prior fixed and only embeds sampled next states into a projective-dynamics solver [2309.13742]. The humanoid standing imitation formulation treats EMP as a pre-controller target-correction module rather than the controller itself [2507.15649]. TC-IDM is explicitly framed as close in spirit to EMP while grounding imagined tool motion into executable control rather than learning a monolithic prior [2601.18323].

The limitations reported across these works are also heterogeneous. Elastic Motion Policy notes that multi-step performance is reduced by segmentation and grasping inaccuracies for both EMP and its baseline [2503.08029]. The executable human-action model reports weaker benefits on coarse kitchen activities because the parser tends to produce simpler programs than the true complexity of activities like cooking, and states that none of the evaluated methods fully captures the complexity of long-horizon activities [2604.18064]. The humanoid standing imitation paper explicitly notes that it does **not yet achieve full-body motion imitation**, that full-sized humanoids have high DoF and complex dynamics, and that joint limits cause noticeable differences between retargeted motions and original human motions [2507.15649]. TC-IDM begins from the premise that direct pixel-to-action translation is brittle because generated videos may contain hallucinations, viewpoint-dependent distortions, occlusions, or kinematic inconsistencies [2601.18323].

A plausible implication is that future EMP research will continue to combine three ingredients that are currently distributed across different papers: semantic task structure, metric or physically grounded execution interfaces, and embodiment-specific feasibility correction. The existing literature already shows that the core question is not simply how to model motion, but how to represent motion in a form that remains stable, controllable, and executable under the actual constraints of deployment.

Source: https://www.emergentmind.com/topics/executable-motion-prior-emp