Motus2: Dexterous Manipulation Model

• The Motus2 system is a general world model for dexterous manipulation designed to unify policy generation, action-conditioned visual simulation, and value-based evaluation within a shared parameter video-action model.
• Tailored for language-conditioned partially observable manipulation, Motus2 leverages curated demonstrations for policy improvement, while failed and suboptimal interactions enhance dynamics and value learning through a closed-loop process
• Motus2 demonstrates significant improvements in task success rates, with an average of 84% on a principal task suite, achieved through large-scale egocentric pretraining and robot-domain mid-training

Motus2 is a self-evolving general world model for dexterous manipulation that unifies policy generation, action-conditioned visual simulation, and value-based evaluation within a shared-parameter video–action model. Proposed for language-conditioned partially observable manipulation, it uses curated demonstrations for action learning, failed and suboptimal interactions for dynamics and value learning, and a closed loop in which candidate actions are imagined, evaluated, selected, and used to improve the policy (Bi et al., 31 Aug 2026). The name should be distinguished from other motion-related works: the supplied papers on MR-MOTUS (Huttinga et al., 2019), Motion-Aware Tracker (Han et al., 2020), Motion2Language (Radouane et al., 2023), mobile motion amplification (Voss, 2022), and a 2D Motion Interface (Yokoyama et al., 17 Aug 2026) do not define this system.

1. Problem formulation and design objectives

Motus2 formulates dexterous manipulation as a language-conditioned partially observable Markov decision process,

$$\mathcal{M} = (\mathcal{S},\mathcal{A},\mathcal{O},\mathcal{T},\Omega,r,\gamma),$$

where $s_t$ is the latent physical state, $a_t$ is the robot action, and the observation

$o_t=(I_t,q_t,\tau_t)$

contains visual observations $I_t$, proprioception $q_t$, and optional tactile measurements $\tau_t$. Because object states, contacts, and prior events may be hidden or occluded, the policy operates on a history

$h_t=(\ell,o_0,\ldots,o_t),$

where $\ell$ is the language instruction. In implementation, this history is summarized by a context $c_t$ containing language, proprioception, and visual working memory.

The basic prediction unit is an action chunk,

$s_t$0

together with future latent observations $s_t$1 and a discretized task-progress variable $s_t$2. Beginning with robot-domain mid-training, Motus2 uses an action-first factorization:

$s_t$3

The three factors correspond to a world–action model (WAM), an action-conditioned world model (AC-WM), and a value model (VM). Their intended roles are distinct but coupled:

• Policy/WAM: proposes executable action chunks.
• Simulator/AC-WM: predicts future visual observations resulting from candidate actions.
• Evaluator/VM: estimates task progress or desirability for the imagined outcome.

The evaluator’s scalar score is the expectation of its categorical value distribution:

$s_t$4

This organization addresses a limitation of pure imitation learning. Demonstrations identify desirable actions, whereas failures and mediocre executions provide information about transitions, recoverability, and negative outcomes. Motus2 therefore treats recorded actions differently according to their evidential role: successful demonstrations activate action supervision, while failed and suboptimal trajectories are retained primarily for dynamics and value learning rather than imitated as policy targets.

2. Shared architecture and causal interfaces

Motus2 is initialized from the Wan 2.2 TI2V-5B video foundation model. The supplied paper does not report an independent parameter count for the final Motus2 checkpoint; the documented scale is therefore the 5B-parameter initialization rather than a separately stated final model size.

Video, action, proprioceptive, language, and value information are processed within a shared transformer computation. Stereo video latents share temporal and vertical positional coordinates while using distinct horizontal rotary-position ranges. Language enters through cross-attention, while proprioception, action tokens, and value queries enter the shared transformer sequence.

The policy samples action chunks using flow matching:

$s_t$5

Within an action chunk, low-level actions are generated jointly rather than strictly autoregressively. During ordinary control, only the policy factor is required; future video and values are generated when planning or evaluation is invoked.

The simulator conditions on a candidate action:

$s_t$6

and predicts latent future visual observations. These predictions describe expected consequences for the robot and manipulated objects rather than necessarily producing decoded images for display.

The evaluator conditions on the context, proposed action, and imagined future:

$s_t$7

Its output is a ranking or progress signal, not a calibrated probability of task success.

Action-first causal masking

During joint pretraining, video and action tokens may interact bidirectionally within a prediction chunk. During robot-domain adaptation, Motus2 adopts an action-first layout,

$s_t$8

where $s_t$9 is a read-only value query. The causal restrictions are:

• action tokens cannot access future video or value tokens from their own chunk;
• future-video tokens can read the current action;
• value queries can read both the action and predicted future video;
• later chunks cannot be used to predict earlier chunks;
• causal attention is maintained across chunks.

A representative sequence is

$a_t$0

This masking prevents future visual information from leaking into action prediction while preserving the intended causal direction from candidate action to imagined consequence and then to value.

3. Learning objectives and supervision

Motus2 uses flow matching for video and action generation. Given a clean target $a_t$1, Gaussian noise $a_t$2, and noise level $a_t$3,

$a_t$4

The shared backbone predicts video and action velocity fields, $a_t$5 and $a_t$6, along with a categorical value distribution. The general objective is

$a_t$7

Trajectory-dependent gates determine which components are active.

Training mode	Action stream	Future video	Value model	Function
Policy	Noisy or generated	Noisy or generated	Off	Action generation with future modeling
Simulation	Clean conditioning	Noisy or target	Off	Action-conditioned dynamics
Evaluation	Clean conditioning	Clean conditioning	On	Task-progress assessment

Curated successful demonstrations provide action targets. Failed trajectories are not treated as desirable action-imitation targets; their actions remain conditioning inputs for transition and negative-outcome learning. Suboptimal trajectories are used similarly. Task-irrelevant interactions supply additional negative evidence for value estimation.

For a successful segment beginning at time $a_t$8, with segment length $a_t$9 and total horizon $o_t=(I_t,q_t,\tau_t)$0, the progress target is

$o_t=(I_t,q_t,\tau_t)$1

Failed or task-irrelevant segments receive

$o_t=(I_t,q_t,\tau_t)$2

These values are discretized into 201 bins during mid-training. The evaluator is thus trained to represent temporally localized progress, including cases in which value initially rises during genuine progress and later falls when an execution becomes unrecoverable.

4. Closed-loop planning and policy improvement

At test time, Motus2 can use best-of-$o_t=(I_t,q_t,\tau_t)$3 planning. It samples candidate action chunks,

$o_t=(I_t,q_t,\tau_t)$4

simulates their consequences,

$o_t=(I_t,q_t,\tau_t)$5

and evaluates them:

$o_t=(I_t,q_t,\tau_t)$6

The selected candidate is

$o_t=(I_t,q_t,\tau_t)$7

Motus2 executes $o_t=(I_t,q_t,\tau_t)$8, observes the real outcome, appends it to memory, and replans. This is a receding-horizon procedure rather than an open-loop rollout. In the reported model-based reinforcement-learning setup, eight policy-generated candidates and one ground-truth candidate were used with an imagined horizon of one action chunk.

Best-of-$o_t=(I_t,q_t,\tau_t)$9 selection changes the executed action but does not itself alter the policy distribution. Motus2 therefore uses DiffusionNFT to update the action pathway. For candidate $I_t$0,

$I_t$1

The online policy and an exponential-moving-average reference policy predict $I_t$2 and $I_t$3. Candidate values are normalized into weights:

$I_t$4

High-value candidates receive weights above $I_t$5, low-value candidates receive weights below %%%%46$s_t$47%%%%$I_t$8$</p> <p>$I_t$9

The policy-improvement objective is

$q_t$0

High-value candidates pull the policy toward similar actions, whereas low-value candidates push it away. Only action-related parameters are updated during this model-based reinforcement-learning stage; the shared video backbone and evaluator are frozen, and the simulator is effectively held fixed.

The reported implementation is macro-asynchronous, using Ray-based actors and FIFO buffers for rollout, simulation, value scoring, reference construction, and optimization. Stale samples are discarded. The exponential-moving-average reference uses decay $q_t$1, is published every ten trainer updates, and candidate samples older than 600 seconds are discarded.

The complete loop is therefore:

$q_t$2

5. Data scaling, memory, and tactile modeling

Motus2 uses a staged data curriculum extending from large-scale egocentric video to robot-specific demonstrations and alignment data.

Egocentric and stereo data

The corpus contains approximately 130,000 raw recording hours before filtering, segmentation, and annotation. It includes monocular collections such as Egocentric-100K, Egocentric-10K, EgoVerse, and EgoDex, together with stereo collections including Ropedia, EgoScale, LightWheel, JD-Group, and CyberOrigin.

Stage 1 trains the visual pathway on monocular video for 500K low-resolution steps and 340K high-resolution steps. Action and value tokens are absent at this stage. Stage 2 introduces synchronized stereo egocentric video paired with human actions for 450K steps. Stereo provides implicit depth cues and supports improved 3D hand-pose estimation.

For stereo-data scaling over 2K, 4K, 10K, and 20K-hour subsets, held-out human-action prediction error decreases approximately according to

$q_t$3

where $q_t$4 is the number of raw stereo-data hours. This is an approximately log-linear trend over the measured range rather than an unlimited scaling law.

Robot-domain mid-training uses more than 100 hours of robot trajectories and human–robot alignment data collected with Wuji Human Gloves, stereo cameras, and wrist trackers. The training mixture contains 85% policy-mode samples, 10% simulation-mode samples, and 5% evaluation-mode samples.

Sliding-window and extended memory

The default policy uses a bounded sliding window of recent clean observations in a key–value cache. After an action chunk is executed, the real observation is appended and the oldest observation is evicted when the cache is full. Imagined observations are not inserted into memory, and temporal rotary-position coordinates are rebased after eviction.

The deployed configuration uses eight latent frames, two frames per action chunk, one-to-three chunks of training history, a two-chunk streaming cache, and five action-denoising steps. A global-autoregressive extension retains all preceding clean visual latents, avoiding sliding-window information loss at the cost of growing memory and attention requirements. A hybrid variant retains two full-resolution anchor frames, four full-resolution recent frames, and eight persistent memory tokens per latent frame for older observations.

Tactile feedback

Motus2 adds a lightweight tactile expert for contact-sensitive control. The shared backbone first denoises a 48-action chunk to an intermediate noise level $q_t$5. The chunk is divided into eight six-action sub-chunks, each lasting 0.2 seconds at a 30 Hz action rate. Before each sub-chunk, the tactile expert uses the preceding tactile window and detached backbone key–value states to refine the action:

$q_t$6

The refinement loss is

$q_t$7

The tactile expert also predicts the force window following each action. Its total objective is

$q_t$8

with $q_t$9.

The tactile expert is a 30-layer transformer with hidden width 128, four attention heads, and feed-forward width 512. The evaluated tactile embodiments are Sharpa Wave, providing a 60-dimensional bimanual force–torque vector and deformation images from ten finger pads, and Wuji Hand 2, providing a 1,086-dimensional bimanual signal containing 1,056 pointwise force values and 30 aggregate per-finger force values.

6. Platforms, experiments, and limitations

Motus2 is evaluated on biomimetic bimanual platforms including the Tianji Marvin dual-arm system with two 20-DoF Wuji dexterous hands, the Tianji Gento Luna with Wuji Hand 2, and the Tianji Gento Luna with Sharpa Wave hands. These systems combine stereo egocentric perception, dual-arm coordination, high-DoF hands, proprioception, tactile feedback, and human teleoperation or alignment data.

Main manipulation comparison

The principal task suite consists of Place Ball, Multi-Finger, Attach Eraser, Screw Bulb, and Put Phone. The reported success rates are:

Method	Place Ball	Multi-Finger	Attach Eraser	Screw Bulb	Put Phone	Average
$\tau_t$0	0%	0%	0%	0%	0%	0%
WAN-SFT	0%	0%	0%	0%	0%	0%
Pretrain-SFT	60%	35%	90%	55%	15%	51%
Motus2 Midtrain-SFT	100%	70%	100%	90%	60%	84%

Under this controlled setup, the reported results associate large-scale egocentric pretraining with an increase from 0% to 51% average success and robot-domain mid-training with a further increase to 84%.

Planning, model-based learning, memory, and touch

On Put Phone and Multi-Finger, the reported averages are:

Configuration	Put Phone	Multi-Finger	Average
Motus2	60%	70%	65.0%
+ Planning	65%	70%	67.5%
+ MBRL	65%	80%	72.5%
+ MBRL + Planning	70%	80%	75.0%

Planning alone improves the average by 2.5 percentage points, model-based reinforcement learning alone by 7.5 points, and their combination by 10 points over the base Motus2 result.

Global autoregression outperforms hybrid memory in the reported delayed-evidence tasks, although it requires growing memory and attention cost:

Setting	Context	Find Square	Press Button	Average
Simulation	Hybrid memory	64%	40%	52%
Simulation	Global autoregression	84%	72%	78%
Real robot	Hybrid memory	30%	20%	25.0%
Real robot	Global autoregression	65%	50%	57.5%

Tactile feedback improves performance on contact-sensitive tasks:

Variant	Pull Out Paper Cup	Tear Paper	Average
Without tactile	65%	55%	60.0%
With tactile	75%	70%	72.5%

Scope and unresolved issues

The reported evidence demonstrates gains from monocular and stereo egocentric pretraining, robot-domain mid-training, best-of-$\tau_t$1 planning, model-based policy improvement, extended context, and tactile refinement under the specified evaluations. It does not establish indefinite data scaling, calibrated value prediction, universal cross-embodiment generalization, or broad superiority over all contemporary world models and dexterous policies.

Several limitations remain. The evaluator is trained from progress-derived targets and is used mainly for ranking; its outputs are not calibrated success probabilities. Model-based policy improvement depends on the fidelity of the learned simulator and value model. Sliding-window memory has bounded cost but discards old evidence, whereas global autoregression preserves history with increasing computational requirements.

Tactile transfer also faces a morphology gap. Wearable tactile gloves capture both external contact and internal material deformation, and human and robotic hands differ in geometry, proportions, and kinematics. Consequently, human tactile patterns cannot be transferred directly to every robot embodiment.

The paper does not fully specify the final Motus2 parameter count, total training compute, inference latency, exact tokenization details, all cross-expert projection layers, or safety mechanisms. It also does not report collision avoidance, uncertainty estimation, action shielding, force limits, recovery policies, or safeguards against physically incorrect imagined futures. The system is therefore best characterized as a shared-weight, model-based manipulation architecture whose demonstrated self-evolution remains bounded by its data curriculum, simulator fidelity, value reliability, embodiment adaptation, and deployment safeguards.