DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation

Abstract: World Action Models (WAMs) provide a promising alternative to Vision-Language-Action (VLA) policies by using video-based world modeling as dense supervision for robot action learning. Existing WAMs excel at physically grounded execution, but typically lack the explicit language-level planning interface in VLM-based VLAs for decomposing coarse instructions. Such decomposition becomes important when household tasks involve complex multi-step goals, where coarse user commands need to be converted into sequences of fine-grained executable subtasks. Meanwhile, the field still lacks a fair real-robot comparison between VLA and WAM execution capabilities, since existing systems often differ in data, robot embodiments, and task protocols. To address both the decomposition gap and the need for a controlled WAM-VLA comparison, we introduce DSWAM, a Dual-System World Action Foundation Model for fine-grained robot manipulation. DSWAM keeps a System 1 WAM executor as the default control path and optionally activates a System 2 vision-language subtask planner only when task decomposition is useful. The planner predicts executable subtasks from short-term visual history and a global task prompt, while the WAM executor performs world-aware action generation for each instruction or subtask. The executor is trained with action prediction and video co-training, but inference directly predicts action chunks without explicit future video generation. To make this execution path practical on real robots, we further integrate TensorRT acceleration, asynchronous execution, and real-time chunking (RTC) so that policy queries do not block robot control. To provide a fair real-robot comparison with VLA policies, we build and evaluate DSWAM under the DeMaVLA real-world deformable manipulation setting with matched robot platform, pretraining data, post-training data, and evaluation criteria.

• The paper presents DSWAM, which couples a world-modeling executor (System 1) with an optional vision-language planner (System 2) for decomposing multi-step robotic tasks.
• The model achieves up to 96.3% success on real-world benchmarks and reduces latency from 198 ms to 74 ms through efficient action inference and TensorRT acceleration.
• The study shows that decoupling semantic planning from world-aware execution minimizes errors, with System 2 boosting task success rates from 75.7% to 100% in coarse-to-fine sorting.

DSWAM: Dual-System World Action Model for Fine-Grained Robot Manipulation

Context and Motivation

World Action Models (WAMs) have established themselves as robust alternatives to Vision-Language-Action (VLA) policies for robotic manipulation, owing to their superior temporal modeling of physical dynamics and contact-rich interactions. WAMs benefit from dense, video-based supervision that strengthens low-level execution. Conversely, VLAs offer semantic interpretability and planning via language-grounded decomposition, but typically lack explicit temporal world modeling. However, the field lacks a fair, matched evaluation protocol comparing WAMs and VLAs across identical hardware, data, and task regimes; further, WAMs do not expose a planner interface for decomposing multi-step, coarse-grained tasks. DSWAM is proposed to fill both these critical gaps by coupling a WAM executor (System 1) with an optional, VLM-based planner (System 2) within a unified foundation model architecture.

Figure 1: DSWAM system schematic: large-scale real-robot pretraining, an optional Vision-Language System 2 planner for subtask decomposition, and a default System 1 WAM executor; video-based co-training is used only at training time, and inference remains efficient with TensorRT acceleration and real-time chunking.

Architecture: Decoupling Planning from World-Aware Execution

System 1: WAM Executor

The default controller in DSWAM is a WAM executor that takes as input the current episode context—multi-view RGB, current instruction or subtask prompt, and proprioception—and predicts a chunk of continuous dual-arm actions. The architecture leverages a pretrained video backbone with dense temporal and physical priors, and action prediction is framed as a conditional flow-matching task over action chunks. Crucially, while the video backbone is co-trained for future frame prediction (world modeling), the deployment regime follows efficient, direct action inference without explicit future-video generation, thus eliminating the runtime penalty of test-time rollout and denoising. Chunked action prediction, with RTC and TensorRT acceleration, further enables low-latency policy execution even for contact-rich, bimanual manipulation.

System 2: Optional Vision-Language Planner

For tasks that benefit from decomposition—for example, those with vague or high-level user commands—DSWAM optionally invokes a Vision-Language System 2. Inspired by Rynnbrain4B-style VLMs, System 2 observes a recent visual window and the global prompt, and emits the next fine-grained executable instruction, which then conditions System 1 for execution. Subtask supervision is provided via explicit transition-aware segmentation of demonstration videos and autoregressive language modeling. Importantly, this planner is only activated when necessary: for atomic or reliably solved tasks, DSWAM operates in a pure WAM (System 1) mode, avoiding unnecessary semantic planning latency.

Empirical Results

RoboTwin 2.0 Simulation Benchmark

On the RoboTwin 2.0 suite—comprising 50 bimanual tasks with both clean and randomized initializations—DSWAM achieves 92.38% success under clean and 91.90% under randomized settings, outperforming both VLA and prior WAM baselines, including Fast-WAM and DeMaVLA. The model exhibits strong robustness to scene randomization, indicating effective physical state encoding and chunked execution.

Matched Real-World Folding Benchmark

Utilizing the DeMaVLA protocol with strict alignment in platform, pretraining, data, evaluation, and success criteria, DSWAM (System 2 disabled) attains a 96.3% average real-world success rate (SR), surpassing DeMaVLA’s 92.5%, and reduces the average completion time from $2'18''$ to $1'44''$. The improvement is most pronounced for more challenging manipulation (hard pants), revealing a substantial execution benefit directly attributable to the WAM policy design, not to differences in setup or planning overhead.

System 2 Subtask Supervision Effects

On a coarse-to-fine sorting task, the optional System 2 planner is shown to facilitate error-free rollout: explicit subtasking boosts SR from 75.7% to 100% and reduces errors from 3.53 to 0.65 per rollout. This controlled ablation demonstrates that high-level subtask inference reduces error propagation in multi-step tasks without harming atomic-instruction performance.

Real-World Deployment Efficiency

DSWAM achieves efficient real-world inference due to the direct action policy, RTC, and TensorRT-compiled execution. End-to-end policy latency is reduced from 198 ms (PyTorch) to 74 ms (BF16 TensorRT), yielding a 2.69x speedup and enabling unblocked real-time robot control. Asynchronous RTC further lifts success rates and completion times for challenging tasks in deployment.

Theoretical and Practical Implications

By explicitly separating physical dynamics modeling (System 1) and semantic planning (System 2), DSWAM offers modularity—exposing an optional high-level planner only when the task requires decomposition, while preserving high-frequency, temporally aware action for all tasks. This design enables high success rates on contact-rich, long-horizon tasks, robust real-world deployment, and extensibility to broader semantic planning regimes. The results demonstrate that world-modeling video pretraining can offer key execution advantages over vision-language direct mapping, particularly when controlled for all confounding factors.

Practically, the architecture and deployment pipeline pave the way for scalable robot foundation models that can operate efficiently in diverse, unstructured environments, and can be extended to more complex task grammars, additional sensor modalities (e.g., force-torque), and even preemptive error recovery via planner integration.

Future Directions

Ongoing developments could include tighter planner-executor feedback, adaptive planner invocation based on execution confidence, and direct integration of corrective policies leveraging both video and subtask context. Multi-agent extensions and hierarchical action abstraction may further benefit long-horizon household or industrial tasks where world-aware modeling and task decomposition are both essential.

Conclusion

DSWAM introduces a dual-system robot foundation policy, combining a world-modeling action executor and an on-demand vision-language planner. Extensive simulation and real-world evaluations, under matched protocols, provide compelling evidence that WAM-style, video-co-trained execution is a strong and efficient baseline for fine-grained, deformable manipulation. The optional planner interface enhances robustness for multi-step and ambiguous tasks. The design achieves practical policy latency for closed-loop robotic control and points toward a modular, scalable blueprint for next-generation robot foundation models (2607.04927).