ChainFlow-VLA: Causal Flow Planning with Vision-Language Models

Abstract: Current end-to-end autonomous driving systems are fundamentally limited by a mismatch between temporal causal reasoning and global trajectory consistency. Autoregressive (AR) models capture interaction-aware temporal dependencies via causal factorization, but their step-wise decoding leads to error accumulation and suboptimal global structure. In contrast, diffusion models optimize trajectories globally but lack explicit causal constraints, making them unreliable in interactive and safety-critical scenarios. This dichotomy reveals a deeper issue: existing methods treat causal modeling and global optimization as separate paradigms, without a principled way to unify them within a single trajectory distribution. To address this, we propose ChainFlow-VLA, which unifies causal generation and global refinement within a unified probabilistic framework. We formulate planning as a mixture over AR-induced modes and learn Vision-LLM (VLM)-conditioned residual distributions over these modes. An autoregressive generator (Chain) produces a discrete set of causal trajectory modes, followed by a diffusion-based refiner (Flow) that leverages VLM hidden states as semantic priors to perform mode-conditioned correction in residual space while preserving causal structure. This straightforward conditioning seamlessly injects high-level scene understanding into fine-grained trajectory adjustments. Experiments demonstrate that ChainFlow-VLA achieves robust planning in ambiguous and long-tail scenarios, achieving a state-of-the-art score of 94.85 on the NAVSIM v1 leaderboard, matching human-level performance (94.8). Code will be available at https://github.com/AFARI-Research/ChainFlow-VLA.

• The paper presents a unified framework combining autoregressive trajectory generation with VLM-guided residual diffusion for semantic refinement.
• It achieves human-level performance with a NAVSIM v1 PDMS score of 94.85 by robustly aligning temporal causality with global geometric and semantic constraints.
• Empirical and ablation studies validate that semantic conditioning improves efficiency and safety in complex driving scenarios such as roundabouts and intersections.

ChainFlow-VLA: Unified Causal Planning and Semantic Refinement in Vision-Language-Driven Autonomous Driving

Problem Formulation and Motivation

Autonomous driving demands robust trajectory planning that reconciles temporal causality with global geometric consistency and semantic understanding. Existing end-to-end approaches bifurcate into two paradigms with intrinsic limitations: autoregressive (AR) models offer strong temporal dependencies but suffer from error accumulation and suboptimal global structure, while diffusion-based planners optimize trajectories globally yet lack explicit causal constraints, undermining their reliability in interactive, safety-critical contexts. Further, attempts to leverage Vision-LLMs (VLMs) for enhanced scene semantics typically result in decoupled architectures where semantic cues are fused early but fail to directly modulate fine-grained trajectory refinement. This architectural dichotomy leaves causal and global optimization unintegrated, especially with respect to interactive semantics and error correction in long-tail driving scenarios.

ChainFlow-VLA (2605.23270) proposes a unified probabilistic framework that merges causal generation and global semantic refinement. Planning is cast as a mixture over AR-induced trajectory modes, each subsequently refined via VLM-guided diffusion in residual space. This operationalizes the law of total probability to encapsulate both temporal rollout and semantic correction, enabling high-level scene understanding to directly steer local trajectory adjustments and bridging the gap between geometric feasibility, causality, and semantic intent.

Methodology

Chain: Autoregressive Trajectory Mode Generation

The AR module constitutes the "Chain," generating discrete trajectory proposals by sequentially decoding future states conditioned on previous timesteps and BEV-style driving features. Multi-modality is represented by maintaining $K$ parallel trajectory hypotheses, each corresponding to a distinct mode. Control variables and kinematic transitions are predicted at each step, enforcing physical feasibility via a bicycle model. Latent scene context is encoded and queried for environment-aware predictions, providing structured initialization for global refinement.

Flow: VLM-Guided Residual Diffusion

The "Flow" module instantiates VLM-guided semantic refinement via residual diffusion. Rather than modeling the full trajectory, it learns corrections relative to AR proposals, parameterizing the local conditional distribution with extracted VLM hidden states. A driving-oriented VLM, fine-tuned on environment and trajectory-QA tasks, supplies semantic priors for the transformer-based residual denoiser, which modulates trajectory refinement without task-specific adaptation.

Residual targets are constructed as the difference between the expert trajectory and the AR proposal, with noisy samples generated for diffusion modeling. Cross-attention injects VLM hidden states to condition each transformer block, and inference proceeds via DDIM to reconstruct refined, mode-conditioned trajectories. The Flow module thus implements proposal-centered correction, tightly coupling semantic information with causal rollouts.

Scoring and Training

Candidate trajectories are ranked by a proxy utility function, selecting the hypothesis with the highest score. Training follows a two-stage protocol: AR supervision employs WTA loss selecting the closest mode to ground-truth, while diffusion refinement is trained within modes nearest to the expert trajectory, optimizing both trajectory loss and noisy residual denoising via asymmetric assignment and direct output supervision.

Empirical Results and Ablation Analysis

ChainFlow-VLA achieves a NAVSIM v1 PDMS score of 94.85, statistically matching human expert performance (94.8) and exceeding previous state-of-the-art for both end-to-end and VLA-based models, including DriveSuprim, RAP-DINO, and LatentVLA. Component-wise ablations demonstrate consistent gains from AR generation, residual diffusion, and especially VLM-guided refinement: semantic conditioning in residual space increases efficiency and safety, notably improving ego progress (EP metric) and collision avoidance in ambiguous scenarios.

Qualitative evaluations illustrate robustness across roundabouts, ramps, and intersections—scenarios where baseline models (ReCogDrive, DrivoR) either fail due to geometric drift or collision, whereas ChainFlow-VLA aligns precisely with navigation routes and avoids static barriers. Design ablations validate the superiority of residual-space modeling (over trajectory-space), the utility of deeper transformer blocks, and the benefit of environment- and trajectory-QA sourced semantic guidance.

ChainFlow generalizes as a modular action expert across multiple backbones (DiffusionDrive, iPad), improving their performance irrespective of underlying planner architecture. The effect of VLM guidance is pronounced: refinement conditioned on VLM features delivers collision-free, semantically coherent trajectories, outstripping BEV-only variants in handling intent, boundaries, and car-following tasks.

Theoretical and Practical Implications

ChainFlow-VLA establishes a principled paradigm for integrating causal temporal reasoning, geometric consistency, and vision-language semantics within a single trajectory distribution. This formulation reframes the role of VLMs—from early-stage injection or direct trajectory generation to mode-conditioned semantic controllers for residual refinement. It highlights that end-to-end driving systems benefit most from semantic guidance in incremental correction—addressing error accumulation and environmental context in real-world, long-tail distributions.

The framework's probabilistic mixture decomposition suggests a more generalizable foundation for planning in interactive, multimodal environments. Residual diffusion conditioned on VLM states can support scalable, safety-critical correction, while the architectural modularity enables cross-backbone integration and adaptation. Empirically, the approach sets a new performance ceiling for vision-language-action planning, validating the tight coupling of scene understanding and fine-grained trajectory optimization.

Future Directions

Although ChainFlow-VLA leverages general driving-oriented VLMs for refinement, further tailoring of VLMs towards score-oriented or judge-oriented supervision may yield stronger alignment with trajectory evaluation and correction tasks. Furthermore, there are potential avenues for integrating reinforcement-driven semantic constraints and expanding the semantic grammar of VLMs. The mixture-based refinement process could also extend to multi-agent coordination, uncertainty estimation, and test-time adaptation for robust deployment.

Conclusion

ChainFlow-VLA presents a unified approach to trajectory planning in autonomous driving, synthesizing causal AR generation and semantic diffusion-based refinement within a vision-language framework. By operationalizing VLMs as semantic conditioners for mode-centered correction, the method attains human-level planning quality, robust safety, and real-world generalization, offering a principled foundation for future research in interactive, semantically-driven autonomy.