---
title: 'DrivoR: Transformer for Autonomous Driving'
url: https://www.emergentmind.com/topics/drivor
type: topic
---

# DrivoR: Transformer for Autonomous Driving

DrivoR is a transformer-based, end-to-end autonomous driving architecture that leverages pretrained Vision Transformers (ViTs) and camera-aware register tokens to achieve accurate, efficient, and behavior-conditioned driving. Its design combines multi-camera perception, compact feature compression, decoupled generation and evaluation of candidate trajectories, and interpretable sub-score prediction for decision-making. DrivoR demonstrates superior or on-par performance with contemporary baselines across NAVSIM-v1, NAVSIM-v2, and HUGSIM closed-loop evaluation benchmarks, while maintaining a minimalist architectural footprint and high computational efficiency [2601.05083].

## 1. Architecture Overview

DrivoR processes raw sensory inputs from four surround cameras (front, front-left, front-right, rear), utilizing a pretrained ViT (typically ViT-S or larger). Each input image is divided into $16\times 16$ patches; these patches are embedded and, in parallel, supplemented with $R$ learnable camera-specific “register” tokens $Q_\textrm{reg}^c \in \mathbb{R}^{R \times D_{\textrm{ViT}}}$. The entire set of image and register tokens undergoes $L$ transformer layers, after which only the register tokens are retained for downstream modules; patch tokens are discarded.

The architecture is composed of two primary lightweight transformer decoder stacks:
- **Trajectory Generator:** Proposes $K$ future action trajectories given the compact, multi-camera scene encoding.
- **Scoring Decoder:** Assigns interpretable, aspect-specific sub-scores—learned to mimic an external oracle—to each candidate trajectory via cross-modal attention.

## 2. Register Tokens and Scene Representation

DrivoR’s camera-aware register tokens provide significant reduction in sequence length and computational overhead. For each camera view:
- Patch tokens $X_{\textrm{patch}}^c \in \mathbb{R}^{N_p \times D_{\textrm{ViT}}}$ and register tokens $Q_\textrm{reg}^{c(L)} \in \mathbb{R}^{R \times D_{\textrm{ViT}}}$ are produced by the final ViT layer.
- The input to the planning stack is $S = \textrm{concat}_{c=1..4} Q_\textrm{reg}^{c(L)} \in \mathbb{R}^{(4R) \times D_{\textrm{ViT}}}$.
- Register tokens are camera-specific, allowing the system to learn specialization by viewpoint.

Compared with spatial average pooling or cross-attention-based fusion, register tokens permit focused extraction of planning-relevant information, compressing $\sim$4K tokens per camera to $R$ (e.g., 16) and expediting all downstream computation.

## 3. Candidate Trajectory Generation and Scoring

### 3.1 Trajectory Generation

- The trajectory generator decoder accepts:
  - Concatenated register tokens (scene encoding) $S$.
  - $K$ learnable trajectory queries $Q_{\textrm{traj}} \in \mathbb{R}^{K \times D_{\textrm{traj}}}$.
  - Ego vehicle state (pose, velocity, acceleration, command), which is embedded and added to each query.
- Four attention blocks combine trajectory self-attention with cross-attention to the scene.
- Output: $K$ tokens $T_i$, each mapped via MLP to trajectory $\tau_i \in \mathbb{R}^{n_p \times 3}$ over $n_p$ future steps (position, heading).

The learning objective is a minimum-of-N (MoN) or winner-take-all (WTA) L1 regression to the human reference trajectory:
$$
\mathcal{L}_{\textrm{traj}} = \min_{i=1..K} \|\tau_i - \hat{\tau}\|_{1} \,.
$$

### 3.2 Scoring Decoder

- Each candidate trajectory $\tau_i$ is re-embedded via an MLP to provide the input query $q_i^{sc}$.
- The scoring decoder processes these queries with the same scene tokens and self-/cross-attention structure.
- Six independent MLP heads predict sub-scores $G_{\theta_c}(\tau_i) \in [0,1]$ corresponding to safety, comfort, efficiency, and traffic compliance metrics (e.g., NC, DAC, TTC, EP, Comf).
- The scoring loss is the binary cross-entropy with oracle-provided sub-scores.

Structural disentanglement between trajectory generation and scoring (as opposed to sharing queries/weights) is empirically critical: combining both into a single decoder degrades the primary driving metric (PDMS) by 5–6 points.

## 4. Training, Behavior Conditioning, and Inference

The total loss combines the trajectory regression and scoring components (with $\lambda_s = 1$ in practice):
$$
\mathcal{L} = \mathcal{L}_{\textrm{traj}} + \lambda_s \mathcal{L}_{\textrm{score}}
$$

At inference, per-trajectory sub-scores are synthesized into a global meta-score using user-configurable weights $\lambda_c^{test}$:
- PDMS-style (product of penalties, sum of behavior metrics) or simple weighted sum.
- The selected trajectory is $\tau^* = \arg\max_{i} \mathrm{Score}(\tau_i)$.

This reward re-weighting at inference time enables flexible behavior conditioning—aggressive, defensive, or otherwise—without retraining.

## 5. Benchmark Evaluation and Performance

DrivoR matches or exceeds strong neural and classical baselines across a range of closed-loop and open-loop end-to-end driving benchmarks:

| Benchmark       | Metric              | DrivoR (ViT-S) | Best Baselines      |
|-----------------|---------------------|:--------------:|:-------------------:|
| NAVSIM-v1       | PDMS $\uparrow$     | 93.7           | DriveSuprim 93.5, Human 94.8 |
| NAVSIM-v2       | EPDMS $\uparrow$    | 48.3           | ZTRS (ViT-99) 48.1  |
| HUGSIM          | RC / HD-Score       | 49.8 / 35.7    | UniAD 45.9 / 32.7   |
| NAVSIM-v2 Speed | ms/fwd (A100)       | 110            | GTRS-Dense 400      |
| NAVSIM-v2 Memory| GB peak             | 0.5            | GTRS-Dense 1.6      |

DrivoR achieves state-of-the-art performance on navigation and closed-loop metrics, while operating with $\leq 40$ million parameters and demonstrating $3$–$4\times$ faster throughput compared to competitive models [2601.05083].

## 6. Design Principles and Comparative Analysis

DrivoR’s empirical successes derive from several core principles:
- Pure-transformer, query-based computation allows flexible conditioning.
- Token reduction via camera-aware register tokens enables high efficiency without significant sacrifice in accuracy.
- Decoupling candidate generation and scoring prevents mode collapse and ensures interpretable trajectory selection.
- Sub-score prediction offers transparency and control, supporting requirement-aware and context-sensitive decision-making.

A plausible implication is that similar register-based compression mechanisms could benefit other high-dimensional multi-camera perception and planning tasks. The lightweight, modular design contrasts with heavily fused, monolithic encoding approaches that are more resource-intensive.

## 7. Limitations and Directions for Future Research

While DrivoR demonstrates that performance and interpretability can be achieved with a small, modular transformer, several open challenges remain:
- Further scaling of camera coverage, scene complexity, and behavior diversity may require larger ViTs or alternative fusion strategies.
- The approach is predicated on oracle sub-scores for training; generalization under domain shift or degraded sensor input is unstudied.
- Scenario-driven stress-testing and integration into large-scale search-based evaluation infrastructures (e.g., Drivora [2601.05685]) remains to be thoroughly explored.

This suggests extensibility toward multi-agent, multi-modal, and language-conditioned settings using similar register-based abstractions, as well as broader adoption in efficiency-critical robotics pipelines.

Source: https://www.emergentmind.com/topics/drivor