UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling

Abstract: Occluded tasks remain a bottleneck in robot manipulation. Existing solutions either deploy additional physical cameras requiring training-inference camera parity, or rely on explicit 3D reconstruction with high computational cost. Moreover, both approaches rely on standard agent-view and wrist-view observations, while failing to capture occlusion information and future scene evolution. To this end, we propose UniviewVLA, a unified multiview Vision-Language-Action model with world modeling, which infers multiview scene evolution for action prediction from only standard two-camera observations. We demonstrate that by leveraging generated multiview future views from the world model, UniviewVLA reveals occluded cues and models future scene evolution, improving action prediction and removing the need for extra hardware or explicit reconstruction. Besides, to accelerate inference while preserving prediction accuracy, UniviewVLA develops Motion-Informative Token Compression, which compresses each generated view from 625 to 16 tokens and reduces per-view latency from 6-7s to 0.2-0.3s. UniviewVLA also proposes training-free Action-Entropy View Selection, which dynamically identifies the most action-informative view at different inference stages. Extensive experiments show that UniviewVLA achieves 95.8% on LIBERO and 4.60 on CALVIN ABCD to D, both standard occlusion-free benchmarks. On customized occlusion-focused tasks, it improves success rate from 40.0% to 73.3%, and average real-robot success rate by 33.4 points, demonstrating stronger occlusion-focused performance without sacrificing standard occlusion-free benchmarks.

• The paper presents a generative multiview world model that synthesizes occluded and future views from standard dual-camera inputs.
• Motion-informative token compression reduces computational latency by retaining only key tokens that capture dynamic workspace changes.
• Dynamic entropy-based view selection adapts to evolving occlusions, achieving state-of-the-art performance on both simulated and real-world manipulation tasks.

UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling

Introduction

UniviewVLA addresses a critical limitation in contemporary Vision-Language-Action (VLA) models utilized for robotic manipulation—specifically, the severe performance degradation that arises when action-critical visual cues are occluded from deployed camera viewpoints. Conventional approaches mitigate occlusions by either scaling the number of physical cameras, thereby imposing maintenance and calibration burdens, or by explicit 3D scene reconstruction, at significant computational expense. UniviewVLA introduces a unified pipeline that leverages generative multiview world modeling to infer and synthesize occluded and future workspace views from standard dual-camera (agent-view and wrist-view) setups alone, obviating the need for additional hardware or explicit reconstruction.

Methodology

UniviewVLA operates via a two-stage training regime grounded in the unified, discrete-token transformer paradigm. In the first stage, a multiview world model is trained to autoregressively generate future auxiliary-view images—represented as sequences of visual quantization (VQ) tokens—conditioned solely on histories from the agent-view, wrist-view, and the language instruction. In the subsequent fine-tuning stage, action decisions are predicted from compact, motion-informative view-token subsets, dramatically reducing computational latency. Candidate auxiliary views are selected via a dynamic, entropy-based scheme that optimizes for low action-token entropy at inference, allowing rapid adaptation to evolving occlusion patterns throughout manipulation trajectories.

Figure 1: The UniviewVLA pipeline: language, multiview images, and actions are unified as token sequences for autoregressive prediction, incorporating world modeling and dynamic view selection.

Multiview World Model Synthesis

The generative model predicts workspace appearances from alternative physical viewpoints and anticipated future states, thereby integrating both static and temporally-evolved evidence that standard views omit. Predictive supervision is enforced at the token level across all auxiliary viewpoints, ensuring robust world-modeling capabilities, including for severe occlusion cases.

Figure 2: Example of auxiliary-view token generation, visualizing ground-truth and generated future views from occluded standard observations.

Motion-Informative Token Compression

Full auxiliary views (originally 625 tokens per view) present an acute trade-off between spatio-temporal coverage and policy latency. UniviewVLA introduces motion-informative token selection: at each step, only the top-K (K=16) tokens reflecting maximal frame-to-frame visual change are retained, ensuring critical workspace dynamics are available at negligible additional latency (0.2–0.3s per view versus 6–7s).

Action-Entropy Dynamic View Selection

Crucially, the informativeness of a viewpoint is non-stationary with respect to the manipulation episode. UniviewVLA dynamically enumerates auxiliary view candidates, computes the mean action-token entropy per candidate, and selects the view minimizing this entropy—effectively and adaptively maximizing actionable scene evidence.

Figure 3: Multiview observations under occlusion. Auxiliary viewpoints are evaluated by action entropy to select the view containing maximal decisive information.

Experimental Results

Occlusion-Free and Challenging Benchmarks

UniviewVLA demonstrates state-of-the-art results on LIBERO and CALVIN ABCD$\to$D, with a 95.8% average success rate and 4.60 average chain length, respectively. These metrics surpass prior methods employing two (agent/wrist) or three (agent/wrist/side) physical cameras, indicating that generative multiview world modeling not only mitigates the occlusion bottleneck but can also enhance standard manipulation performance.

Occlusion-Focused Simulation

On customized occlusion-centric manipulation tasks, the standard two-camera baseline delivers only 40.0% success. The addition of a physical third camera yields 66.3%, while UniviewVLA with dynamic entropy-based view selection achieves 73.3%. These results empirically demonstrate that synthesized multiview evidence can exceed the coverage benefits of third physical cameras without deployment or calibration overhead.

Figure 4: Six customized occlusion tasks, each designed to obscure task-critical cues from default agent cameras.

Real-World Deployment

Deployed on the ALOHA platform, UniviewVLA attains success rates of 53.3% (Oreo-to-Plate) and 46.7% (Occluded-Doll Move) using only dual standard camera inputs, outperforming the two-camera policy (13.3% and 20.0%) and approaching or matching the three-camera setup (40.0% and 53.3%). These results validate the transferability and closed-loop efficacy of token-compressed generative views and entropy-based selection in physical robot systems.

Figure 5: Real-robot occlusion tasks illustrate practical occlusion where critical objects are hidden from standard viewpoints.

Analysis and Model Implications

The principal innovation of UniviewVLA is to decouple observation-space augmentation from the core hardware stack by leveraging transformer-based autoregressive scene evolution. The generative auxiliary views provide three tangible advantages:

1. Enhanced Observability Without Hardware Scaling: Auxiliary-view tokens synthesized via world modeling confer the benefits of diverse camera coverage for manipulation without requiring a proliferation of cameras.
2. Computational Efficiency: Motion-informative token compression ensures minimal inference overhead, a persistent challenge for token-based visual world modeling.
3. Adaptive Perception: Entropy-driven view selection enforces context-sensitive, stage-dependent perception, outperforming static multiview configurations.

Theoretically, this approach concretizes a notion of "perceptual imagination" for policy deployment, wherein the model not only recalls but predicts the workspace from multiple virtual perspectives—implicit reasoning unattainable with fixed camera hardware or static models. The positive correlation between entropy minimization and action outcome success further substantiates model uncertainty as a viable selection heuristic for perceptual control in closed-loop tasks.

Limitations and Future Work

UniviewVLA shares limitations inherent to current generative policies: inference cost (relative to non-generative policies), dependence on the density of supervised multiview training data, and primary validation on tabletop manipulation scenarios. Extensions to mobile, multi-robot, and long-horizon environments remain open. Future directions include scaling generative viewpoints to mobile sensor arrays, integrating richer spatial-temporal context modeling, and combinatorial policy improvement through interaction with environment-level to agent-level uncertainty quantification.

Conclusion

UniviewVLA establishes a robust framework for VLA-based robotic manipulation that integrates generative, compressed, and dynamically selected multiview representations. Quantitative evaluations across simulation and real-world domains demonstrate that this method can supplant the need for deployment-time camera scaling or explicit 3D reconstruction, yielding measurable improvements on both occlusion-free and occlusion-focused benchmarks. These findings chart a practical path forward for scalable, adaptive, and hardware-agnostic visuomotor control.

Figure 6: Example real-robot multiview configuration for fair comparison of hardware-based and model-based auxiliary viewpoint strategies.