--- title: DiffusionDriveV2 Autonomous Driving Framework url: https://www.emergentmind.com/topics/diffusiondrivev2 type: topic --- # DiffusionDriveV2 Autonomous Driving Framework DiffusionDriveV2 is a reinforcement learning-constrained truncated diffusion modeling framework for end-to-end autonomous driving, designed to resolve the persistent “diversity–quality dilemma” that arises when leveraging generative diffusion planners. It accomplishes this via a combination of Gaussian mixture modeling, anchored intent representations, scale-adaptive multiplicative exploration noise, and a dual-level reinforcement learning objective, achieving state-of-the-art closed-loop driving performance while preserving trajectory multimodality [2512.07745]. ## 1. End-to-End Trajectory Generation with Anchored Diffusion The autonomous driving policy in DiffusionDriveV2 is formulated as a mapping \[ \pi_\theta: z \mapsto \tau = \{(x_n, y_n)\}_{n=1}^{N_f} \] where $z$ represents processed sensor inputs and $\tau$ is a sequence of future waypoints over a fixed planning horizon $N_f$. Traditional imitation learning approaches produce single-mode outputs, failing to capture real-world multimodal intent. Vanilla diffusion models, though multimodal, suffer from mode collapse, generating conservative, mean-like behaviors on diverse driving scenarios. DiffusionDrive introduced $N_{\text{anchor}}$ discrete intent anchors $\{a^k\}$ to partition the action space—e.g., “turn left,” “go straight”—with a truncated diffusion decoder acting over a Gaussian Mixture Model (GMM) prior. Supervision, however, was limited to the anchor closest to the ground truth, leaving other modes unconstrained and yielding low-quality, sometimes invalid trajectories. DiffusionDriveV2 extends this strategy by incorporating reinforcement learning (RL) based constraints, both to penalize unsafe or low-quality modes and to encourage exploration toward higher-reward behaviors. The framework’s core advancements are its use of scale-adaptive multiplicative exploration noise, Intra-Anchor Group Relative Policy Optimization (GRPO), and Inter-Anchor Truncated GRPO. ## 2. Gaussian Mixture Modeling and Anchored Trajectory Diffusion The conditional distribution over future trajectories, given perception $z$, is modeled as a Gaussian mixture: \[ p(\tau \mid z) = \sum_{k=1}^{N_{\text{anchor}}} s(a^k \mid z) \, \mathcal{N}(\tau ; a^k + \mu^k(z), \Sigma^k(z)) \] with $s(a^k\mid z)$ as softmax-weighted intent probabilities, $\mu^k(z)$ as scene-dependent offsets, and $\Sigma^k(z)$ as offsets' covariances. At inference, each anchor $a^k$ initiates a truncated diffusion chain: \[ \tau_t^k = \sqrt{\bar\alpha_t} a^k + \sqrt{1-\bar\alpha_t}\, \epsilon,\quad \epsilon \sim \mathcal{N}(0, I),\quad t=1 \ldots T_{\text{trunc}} \] producing a pool of candidate trajectories that reflect distinct high-level driving intents. ## 3. Scale-Adaptive Multiplicative Noise for Exploration Exploration in continuous control is a central challenge. DiffusionDriveV2 replaces standard additive Gaussian noise with two-degree-of-freedom multiplicative noise, tailored for vehicle trajectory planning: \[ \tau' = (I + \epsilon_{\text{mul}}) \tau \] where \[ \epsilon_{\text{mul}} = \mathrm{diag}(\epsilon_{\text{long}}, \epsilon_{\text{lat}}), \quad \epsilon_{\text{long}}, \epsilon_{\text{lat}} \sim \mathcal{N}(0, \sigma^2) \] This scheme injects controlled scaling in both longitudinal and lateral directions. During RL-based training (DDIM/DDPM noise scale $\eta=1$), a minimum standard deviation constraint $\sigma_{\min} \ge 0.04$ is enforced for both dimensions, preventing entropy collapse and fostering robust exploration. Deterministic sampling ($\eta=0$) is used during inference. ## 4. Reinforcement Learning Constraints: Intra- and Inter-Anchor GRPO Trajectories for each anchor are generated by applying a Gaussian denoising policy via the truncated diffusion chain: \[ \pi_\theta(\tau_{t-1}^k \mid \tau_t^k, z, a^k) = \mathcal{N}(\tau_{t-1}^k ; \mu_\theta(\tau_t^k, t, z, a^k), \eta(1-\alpha_t)I) \] ### Intra-Anchor Group Relative Policy Optimization (GRPO) For each anchor $k$, $G$ sample trajectories $\{\tau_0^{k,i}\}_{i=1}^G$ are generated. Relative advantages are calculated within the anchor group: \[ A^{k,i} = \frac{r(\tau_0^{k,i}) - \overline{r_k}}{\mathrm{std}(\{r(\tau_0^{k,1}), ..., r(\tau_0^{k,G})\})}, \] with $\overline{r_k}$ the mean reward across group samples. The RL gradient for training is \[ \nabla_\theta J = \mathbb{E}\Big[\sum_{t=1}^{T_{\text{trunc}}} \gamma^{t-1} \nabla_\theta \log\pi_\theta(\tau_{t-1}^{k,i} \mid \tau_t^{k,i}) A^{k,i}\Big] \] and the intra-anchor loss is \[ L_{RL} = -\frac{1}{N_{\text{anchor}}} \sum_{k=1}^{N_{\text{anchor}}} \frac{1}{G} \sum_{i=1}^G \frac{1}{T_{\text{trunc}}} \sum_{t=1}^{T_{\text{trunc}}} \gamma^{t-1} \log \pi_\theta(\tau_{t-1}^{k,i} \mid \tau_t^{k,i}) A^{k,i} \] ### Inter-Anchor Truncated GRPO To provide a global reward signal without mode collapse, negative intra-anchor advantages are truncated to zero; colliding trajectories are assigned a hard penalty of $-1$: \[ A_{\text{trunc}}^{k,i} = \begin{cases} -1, & \text{if } \tau_0^{k,i} \text{ collides} \\ \max(0, A^{k,i}), & \text{otherwise} \end{cases} \] These are substituted into the RL loss. ## 5. Learning Algorithm and Objective Structure The overall training objective sums the RL loss and a regularized imitation learning (IL) loss computed over the denoising-reconstruction and anchor-classification binary cross-entropy (BCE): \[ L_{\text{total}} = L_{RL} + \lambda L_{IL}, \quad \lambda \in (0,1) \] High-level learning proceeds as follows: ```python for epoch in 1..E_rl: for batch in dataset: features ← PerceptionNet(batch.sensors) for each anchor k: for i = 1..G: τ_T ← noisy_anchor(a^k; features) τ_0 ← RunTruncatedDiffusion(τ_T; θ, η=1, multiplicative_noise) r[k,i] ← SimulatorReward(τ_0) for each anchor k: compute A^{k,i} and then A_trunc^{k,i} L_RL ← sum_{k,i,t} [ -γ^{t-1} log π_θ(·) A_trunc^{k,i} ] L_IL ← imitation‐learning loss on GT anchor θ ← θ − AdamW(∇θ[L_RL + λ L_IL]) ``` Following RL-stage training, a two-stage mode selector is trained on frozen generator outputs for 20 epochs, using BCE and margin-rank losses. ## 6. Experimental Protocol and Metrics Key experimental details include: - Datasets: NAVSIM v1 (1,192 train, 136 test scenes) and NAVSIM v2 (with extended closed-loop metrics). - Inputs: ResNet-34 backbone aligned on BEV-LiDAR plus three front cameras (1024×256). - PDMS is the main closed-loop score: \[ \mathrm{PDMS} = \mathrm{NC} \times \mathrm{DAC} \times \frac{5\,\mathrm{EP} + 5\,\mathrm{TTC} + 2\,C}{12} \] where NC: no-at-fault collisions, DAC: drivable-area compliance, EP: ego progress, TTC: time to collision, C: comfort. - Extended score on v2, EPDMS: \[ \mathrm{EPDMS} = \mathrm{NC} \times \mathrm{DAC} \times \mathrm{DDC} \times \mathrm{TL} \times \frac{5\,\mathrm{TTC} + 2\,C + 5\,\mathrm{EP} + 5\,\mathrm{LK} + 5\,\mathrm{EC}}{22} \] - Diversity: average normalized pairwise waypoint distance: \[ Div_{\text{raw}}^n = \tfrac{2}{M(M-1)}\sum_{i