WOLF-VLA: Humanoid Locomotion Framework

• WOLF-VLA is a unified robotics framework that generates dynamically feasible humanoid locomotion trajectories from natural-language instructions using optimal-control and multimodal learning.
• The framework employs constrained optimal control for trajectory synthesis and a rigorously designed dataset that integrates ego-centric vision, proprioception, and spatially tagged language.
• WOLF-VLA bridges classical model-based planning with imitation learning, outperforming baselines in dynamic tasks while maintaining low range-of-motion errors.

WOLF-VLA, short for Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning, is a unified robotics pipeline that combines whole-body optimal-control motion synthesis with large-scale multimodal data collection to train a Vision-Language-Action model for humanoid locomotion from natural-language instructions. In the formulation reported in "WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning" (Boukheddimi et al., 24 Jun 2026), the framework addresses a specific gap in VLA research: strong progress in manipulation has not been matched by comparable progress in whole-body, contact-rich humanoid locomotion, largely because of data scarcity, the lack of dynamically consistent demonstrations, and the difficulty of encoding optimality and safety in learning pipelines.

1. Scope and problem setting

WOLF-VLA is designed around two coupled components: an optimal-control generator that synthesizes dynamically feasible whole-body locomotion trajectories, and a multimodal learning system that consumes those trajectories as demonstrations. The stated objective is not merely to imitate motion, but to train a single policy that can execute locomotion behaviors directly from ego-centric visual input, proprioception, and language instructions. This places the framework at the intersection of humanoid motion planning, imitation learning, multimodal representation learning, and instruction-conditioned control (Boukheddimi et al., 24 Jun 2026).

The motivating problem is narrowly defined. Prior VLA datasets are described as often teleoperated or video/mocap-based and as not necessarily optimal with respect to energy, torque, or smoothness. WOLF-VLA instead generates training data through constrained optimal control, so that dynamic feasibility, contact coherence, torque limits, and joint limits are embedded in the demonstrations themselves. The framework therefore treats optimal control not as a downstream stabilizer layered on top of a learned policy, but as the source of the supervisory distribution.

The benchmark centers on six locomotion-related task families: target-directed forward walking, lateral locomotion toward a target, stair ascent, compound stair ascent and descent, 180° turning motions, and variable-height squatting. The platform is the RH5 humanoid in MuJoCo, with whole-body locomotion executed through low-level joint-space actions represented as delta joint rotations. A plausible implication is that WOLF-VLA is intended as a bridge between classical model-based locomotion synthesis and scalable multimodal policy learning, rather than as a purely end-to-end perception-control system.

2. Dataset design and multimodal structure

The WOLF-VLA-dataset contains 277 hours of optimal-control-generated locomotion demonstrations across the six task families. Each sample consists of synchronized ego-centric RGB observations, proprioceptive states, action commands, and a natural-language instruction. The visual stream is recorded from a head-mounted camera with 120° field of view at 33.33 Hz and stored at $224 \times 224$ resolution. Proprioception includes 32 joint rotations and 31 joint velocities, including the floating base. Actions are represented as $\Delta q_t$, namely delta joint rotations at each control step (Boukheddimi et al., 24 Jun 2026).

Environmental variation is a central part of the dataset design. Target types include a box, cylinder, sphere, ground marker, a single staircase with three steps, and a double staircase with three ascending and three descending steps. Six distinct color variants are applied to all target types. Placements are varied over an approximately $40 \times 40$ grid along $X$ and $Y$, and random non-target objects are inserted as distractors. The resulting dataset is therefore not a narrow collection of repeated nominal gait cycles, but a structured space of locomotion problems with systematic visual and spatial perturbations.

The language channel is generated automatically from optimal-control metadata and augmented with structured spatial tags. Two explicit tags are reported: <DIST>…</DIST> for target distance and <HEIGHT>…</HEIGHT> for step height. This means that natural language is not used as an unconstrained annotation layer; it is partially grounded in symbolic task parameters that can condition the policy on geometry relevant to locomotion.

Average episode durations are reported for several tasks: WF at 13.5 s, WA at 43.2 s, W.CS.U at 21.6 s, and W.CS.U/D at 33.6 s. Rotation and squatting are part of the dataset, but the main reported evaluations focus on WF, WA, W.CS.U, and W.CS.U/D. Dataset quality is maintained through automatic success checks and manual validation, with the stated aim of minimizing numerical artifacts and preserving dynamic consistency.

3. Optimal-control formulation and trajectory synthesis

The demonstration generator is a multi-phase optimal-control problem with whole-body dynamics, contact constraints, and actuator limits. The continuous dynamics are written as

$$M(q)\ddot{q} + b(q,\dot{q}) = S^\top \tau + \sum_{k=1}^{K} J_k(q)^\top \lambda_k$$

where $q$ are generalized coordinates, $M$ is the inertia matrix, $b$ collects Coriolis, centrifugal, and gravity terms, $S$ selects actuated joints, $\Delta q_t$0 are torques, and $\Delta q_t$1 are the contact Jacobian and force for contact $\Delta q_t$2. Contacts are enforced as second-order kinematic constraints in acceleration space (Boukheddimi et al., 24 Jun 2026).

The multi-phase OCP is defined over contact modes $\Delta q_t$3:

$\Delta q_t$4

subject to admissible sets $\Delta q_t$5, $\Delta q_t$6, $\Delta q_t$7, and state dynamics $\Delta q_t$8. The reported transcription is multiple shooting, solved by Differential Dynamic Programming using Crocoddyl’s Box-FDDP, with Pinocchio used for dynamics and derivatives.

The phase cost structure is

$\Delta q_t$9

with four named components: center-of-mass tracking, swing-foot placement, torque minimization, and posture regularization. The CoM term pulls the center of mass toward a phase-end target; the foot term drives the swing foot to a step or ground target; the torque term promotes energy efficiency and feasibility; the posture term resolves redundancy and keeps the system near a nominal whole-body posture.

This construction is important because it formalizes the paper’s notion of “optimality and safety” at data-generation time. The demonstrations are not only kinematically valid but explicitly constrained by joint, velocity, and torque bounds through $40 \times 40$0, $40 \times 40$1, and $40 \times 40$2. The paper does note a limitation: explicit friction cones and complementarity are not detailed in the formulation, which may matter for aggressive gaits or more difficult contact regimes. That caveat narrows the claim: WOLF-VLA provides contact-consistent, torque-limited locomotion data, but not a full treatment of all possible contact-physics formalisms.

4. Policy architecture, flow matching, and training

The learning component is a VLA model initialized from GR00T-N1.5-3B. Vision and language are encoded by frozen backbones, while trainable projector modules and an action diffusion module learn to map multimodal observations to action chunks. The fused input comprises ego-centric vision tokens, language tokens, and proprioceptive embeddings. The output is a chunk of low-level actions in joint space, rather than a high-level gait command or latent skill code (Boukheddimi et al., 24 Jun 2026).

The action model uses flow matching. Let $40 \times 40$3 denote multimodal observations and $40 \times 40$4 an expert action chunk. Noise $40 \times 40$5 and interpolation time $40 \times 40$6 are sampled, and the interpolated action is

$40 \times 40$7

The model predicts a conditional vector field $40 \times 40$8, trained with

$40 \times 40$9

At inference, denoising proceeds by forward Euler updates

$X$0

starting from $X$1. The final denoised chunk is executed over $X$2 control ticks in a receding-horizon loop.

Training is reported on $X$3 NVIDIA A100 GPUs for 200,000 gradient steps with effective batch size 128. The learning rate uses a 500-step warmup and cosine decay from $X$4 to $X$5. Optimization uses AdamW with $X$6, $X$7, $X$8, weight decay $X$9, gradient clipping at 10, and bfloat16 precision. Data are stored in LeRobot format for standardized loading and reproducibility. The stated claim is that this training procedure transfers the dynamic feasibility, smoothness, and optimality of the OC expert into the learned policy.

5. Evaluation protocol and empirical performance

Evaluation uses 20 rollouts per task under two visual conditions, with and without distractors, and three horizon settings: Short, Medium, and Long. Generalization is tested by holding out specific object-type and color combinations while allowing the individual object types and colors to appear in training. The principal metrics are binary success rate, Soft Success Rate for stair tasks, and AROM, the normalized average range-of-motion error of hip, knee, and ankle relative to unseen optimal-control references (Boukheddimi et al., 24 Jun 2026).

For stairs, Soft Success Rate is defined as

$Y$0

with $Y$1 per stair. Range of motion is

$Y$2

and the normalized ROM error is

$Y$3

These metrics distinguish full task completion from partial progress and also test whether the learned controller uses joints in a manner consistent with the expert.

The headline empirical pattern is asymmetric across tasks. WF achieves approximately 95–100% success across short, medium, and long horizons, with and without distractors. On successful runs, hip, knee, and ankle AROM errors are often approximately 1–2%, indicating close reproduction of expert joint usage patterns. WA is much harder: it reaches 100% only on short-horizon no-distractor trials, 60% on medium-horizon no-distractor trials, and near 0% otherwise. W.CS.U yields SSR of approximately 47–63% across conditions. W.CS.U/D yields SSR of approximately 35–68% depending on setting, with average SSR approximately 44% and average standard success rate approximately 10%. Overall average success across all tasks, conditions, and horizons is approximately 55%.

Baseline comparisons are strongly unfavorable to the alternative models tested on the same observation and action spaces. ACT reaches approximately 8.3% only in the easiest short/no-distractor setting and is otherwise near 0%, while TTO.5 is approximately 0% across the board. The reported conclusion is therefore not that WOLF-VLA solves instruction-conditioned humanoid locomotion in general, but that it substantially outperforms two baselines under the benchmark’s conditions and does so while preserving low ROM error relative to OC references.

6. Ablations, limitations, reproducibility, and naming

The ablation study identifies modality hierarchy rather than simple modality complementarity. The full model, which uses vision, language with spatial tags, and proprioception, performs best overall. Removing spatial tags causes a small reduction. Removing language causes a moderate drop on complex tasks such as WA and the stair tasks. Removing vision causes a dramatic collapse: even WF falls to 20% on short/no-distractor trials and 0% elsewhere, while WA and the stair tasks are near 0%. Instruction paraphrasing using GPT-4o leaves performance broadly robust on WF and stairs, but WA remains the most fragile under distractors and longer horizons (Boukheddimi et al., 24 Jun 2026).

Several limitations are explicit. All reported results are in simulation, so sim-to-real transfer is not demonstrated. Perception-heavy turning and compound stair tasks show sensitivity to clutter and long-horizon error accumulation. Robustness to severe occlusion or novel textures is not guaranteed. Contact modeling is enforced kinematically in acceleration space, but explicit friction cones and complementarity are not detailed. The paper therefore positions WOLF-VLA as a benchmark and training framework rather than as a complete deployment stack for real humanoid hardware. Future directions named in the paper include richer contact models, improved clutter robustness, domain randomization, photorealistic rendering, and real-world deployment with safety supervisors such as WBC or MPC filters.

Reproducibility is a central stated objective. The authors indicate that they will release the WOLF-VLA-dataset, model checkpoints, and a Gymnasium-based benchmarking simulation suite built around RH5 in MuJoCo. This is presented as a reproducible, dynamically consistent benchmark for humanoid locomotion-rich VLA control.

The acronym has an additional source of ambiguity in arXiv-adjacent literature. In the supplied corpus, “WOLF-VLA” also appears in unrelated Wolf–Rayet contexts, including a consolidated Wolf–Rayet Velocity Law Analysis derived from PoWR $Y$4-law experiments (Lefever et al., 2022) and a 22 GHz VLA imaging program of local Wolf-Rayet galaxies (Ferraro et al., 2024). In robotics usage, however, WOLF-VLA specifically denotes the whole-body humanoid locomotion framework introduced in 2026.