WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving

Abstract: Vision-Language-Action (VLA) models have advanced end-to-end autonomous driving. However, existing methods either lack comprehensive world cognition or suffer from fragmented world foresight, inherently confining these models to reactive driving. To address this limitation, we propose WCog-VLA, a novel dual-level World-Cognitive VLA framework that successfully bridges semantic world forecasting with generative world evolution to achieve proactive autonomous driving. At the semantic level, WCog-VLA unifies world cognition and reasoning by incorporating 3D spatial perception and injecting agent tokens to capture the world dynamics, while concurrently enabling Game-theoretic Chain-of-Thought (Game-CoT) reasoning. At the generative level, we introduce the Aligned Decoupled Diffusion Transformer (ADDT) as a powerful generative world model that synthesizes physically-plausible joint multi-agent trajectories. Through scene representation alignment, ADDT reduces the number of denoising steps required and thus significantly accelerates inference. To facilitate strategic reasoning, we further construct a large-scale dataset featuring 85k Game-CoT annotations. Extensive experiments on the NAVSIM benchmark demonstrate that WCog-VLA achieves a State-Of-The-Art (SOTA) PDMS score of 92.9.

• The paper presents a dual-level framework that fuses semantic world cognition with generative trajectory modeling for proactive end-to-end autonomous driving.
• It integrates multi-view vision, textual inputs, and 3D spatial perception using models like InternVL3-2B and BEVFormer to produce structured agent-environment representations.
• Experimental results demonstrate state-of-the-art performance with a PDMS of 92.9 and up to 10.7× faster inference, underlining its safety and efficiency benefits.

WCog-VLA: Dual-Level World-Cognitive VLA Model for End-to-End Autonomous Driving

Motivation and Problem Statement

End-to-end (E2E) autonomous driving paradigms, which map sensor inputs directly to actions or trajectories, are proficient in canonical scenarios but exhibit limited causal reasoning and world modeling, especially in complex, long-tail settings. Current Vision-Language-Action (VLA) models, though bolstered by multimodal reasoning, are constrained to reactive behavior due to fragmented semantic forecasting and lack of generative world foresight. They typically fail to model joint agent-environment interactions or strategic, game-theoretic reasoning, thus impeding proactive, human-like decision making.

WCog-VLA Architecture Overview

WCog-VLA introduces a dual-level World Cognition framework, establishing a comprehensive bridge between semantic-level forecasting and generative-level world evolution, aiming for proactive E2E driving. The model comprises:

• VLM Backbone: Built on InternVL3-2B, integrating multi-view vision, textual instructions, and ego states. It is enriched by 3D spatial perception using BEVFormer and TrackFormer, yielding agent-centric tokens for structured environment representation.
• Semantic World Cognition: The VLM's hidden states are segmented into cognition (current and future agent world states) and reasoning (game-theoretic Chain-of-Thought [Game-CoT]). This produces explicit scene understanding and strategic foresight.
• Generative World Model (ADDT): The Aligned Decoupled Diffusion Transformer bridges semantic intent to physically-plausible joint trajectories by aligning latent representations with a pre-trained VAE’s agent-interaction space, enabling efficient and interactive multi-agent trajectory synthesis.

Game-CoT Reasoning Dataset

A major contribution is the construction of Game-CoT, an 85k-sample dataset for game-theoretic reasoning, filling supervision gaps in strategic interaction. Reasoning steps are structured as scene analysis, critical object identification, Stackelberg-game-based action enumeration, and payoff evaluation. GT actions are included to minimize hallucination and enforce explicit causal links, leading to rigorous and interpretable strategic annotations.

Training Paradigm

WCog-VLA employs a four-stage training regime:

1. 3D Spatial Perception Pre-Training for BEV and agent token extraction.
2. Supervised Fine-Tuning of VLM on multimodal VQA, Game-CoT, and trajectory datasets.
3. ADDT Supervised Fine-Tuning for grounded, generative trajectory synthesis using DDPM and explicit representation alignment.
4. Reinforcement Fine-Tuning with DiffGRPO, optimizing for policy robustness beyond behavioral cloning and balancing ego and agent trajectory quality via NAVSIM PDMS and displacement penalties.

Experimental Results

WCog-VLA achieves a SOTA PDMS score of 92.9 on NAVSIM v1 and EPDMS of 85.9 on NAVSIM v2, outperforming a suite of established E2E and VLA baselines, including large-scale VLMs, diffusion-based planners, and multi-modal approaches. Notably, despite relying solely on camera inputs, WCog-VLA surpasses lidar-enhanced methods in both planning and safety (NC, TTC). Ablation studies reveal:

• Each training stage is essential, with ADDT contributing substantial improvement (3.8 PDMS) over discrete VLM output.
• Synergistic coupling of semantic and generative cognition is necessary for robust multi-agent planning (89.3 PDMS vs 86.5 baseline).
• ADDT accelerates inference by up to 10.7× relative to text-based VLMs; alignment regularization ensures trajectory plausibility and encoder consistency.
• Incorporation of diverse VQA sources—including Game-CoT—yields optimal planning scores, validating the necessity of strategic reasoning supervision.
• Explicit 3D perception boosts spatial precision, with a 3.3 PDMS gain over plain vision-language features.

Practical and Theoretical Implications

WCog-VLA demonstrates that proactive E2E autonomous driving requires both structured semantic world understanding and generative mutual agent evolution. The explicit dual cognition model enables anticipation of agent intents and tactical negotiation, marking a departure from purely reactive frameworks. Game-theoretic reasoning supervision cultivates strategic foresight, imperative for dense-urban and adversarial environments. The ADDT architecture, with its alignment and decoupling, resolves diffusion model optimization trade-offs and achieves high-fidelity, efficient multi-agent rollout, facilitating deployment viability.

From a theoretical perspective, the integration of latent alignment regularization with joint semantic-generative modeling may inform future foundation models for embodied AI, wherein interaction, foresight, and causal reasoning are central.

Directions for Future Research

While WCog-VLA achieves explicit agent-level cognition, it currently omits dynamic modeling of road geometry and topological evolution. Extending semantic cognition to encompass evolving map states, infrastructure, and environmental cues will be critical for holistic world modeling. Further research may leverage generative graph structures, multi-modal instruction following, and reinforcement learning paradigms attuned to emergent social negotiation and complex multi-agent systems.

Conclusion

WCog-VLA introduces a dual-level World-Cognitive VLA paradigm that unifies semantic forecasting and generative trajectory evolution for proactive, strategic autonomous driving. Its explicit coupling of 3D spatial perception, game-theoretic reasoning, and aligned generative modeling yields superior performance, safety, and efficiency in benchmarked urban scenarios. The framework sets a precedent for world-centric AI models and signals future directions toward comprehensive, dynamic cognition in E2E driving systems (2607.08375).