Bench2Drive-Speed Benchmark

• Bench2Drive-Speed is a benchmark for desired-speed conditioned autonomous driving that integrates explicit target-speed inputs and overtaking commands.
• It employs closed-loop evaluation in CARLA using metrics such as Speed-Adherence Score, Driving Score, and Overtake Score to assess system performance.
• The benchmark addresses user-control gaps with customized datasets and virtual target-speed supervision to enhance training and measurable compliance.

Searching arXiv for the benchmark and closely related papers to ground the article in current literature. Bench2Drive-Speed, also termed “Drive,” is a benchmark with metrics, dataset, and baselines for desired-speed conditioned autonomous driving. It extends the Bench2Drive ecosystem with explicit target-speed inputs and overtake/follow instructions to driving policy models, and evaluates whether an end-to-end autonomous driving system can follow user specifications while remaining compatible with standard autonomous driving metrics such as Driving Score, Success Rate, Efficiency, Comfort, and multi-ability scores (Shao et al., 26 Mar 2026). Within the broader Bench2Drive literature, the benchmark occupies the specific niche of controllability: it measures not only whether a vehicle drives safely and completes routes, but also whether it obeys a desired cruising speed and executes overtaking commands in closed loop (Shao et al., 26 Mar 2026).

1. Scope, motivation, and problem formulation

Bench2Drive-Speed is motivated by a practical gap in end-to-end autonomous driving. The benchmark paper states that end-to-end autonomous driving systems rarely expose user controls for desired cruising speed or explicit overtake/follow decisions, despite real use cases where users want faster or more cautious styles. It further argues that desired-speed adherence and controlled overtaking require long-horizon interaction and must remain consistent with safety, traffic rules, and comfort (Shao et al., 26 Mar 2026).

The benchmark is framed against two limitations in prior evaluation practice. First, existing benchmarks evaluate safety and completion but do not quantify how faithfully a policy follows user-specified speed. Second, style-aware works often encode speed implicitly within abstract categories such as Conservative, Normal, and Aggressive rather than as an explicit, measurable objective (Shao et al., 26 Mar 2026). Bench2Drive-Speed therefore makes speed a directly controllable scalar and pairs it with a categorical overtaking instruction.

The central contribution is a closed-loop benchmark in CARLA that introduces explicit inputs of users’ desired target-speed and overtake/follow instructions to driving policy models, together with quantitative metrics—Speed-Adherence Score and Overtake Score—that measure how faithfully policies follow user specifications while remaining compatible with standard autonomous driving metrics (Shao et al., 26 Mar 2026). The paper states that, to the authors’ knowledge, it is the first closed-loop end-to-end benchmark to make desired-speed control explicit and measurable while remaining compatible with the Bench2Drive ecosystem (Shao et al., 26 Mar 2026).

2. Command interface, tasks, and evaluation design

Bench2Drive-Speed uses two high-level inputs per route segment: a target-speed command and an overtaking command. The target-speed command specifies the desired cruising speed. The overtaking command specifies whether the agent should overtake or follow when a slower lead vehicle is present (Shao et al., 26 Mar 2026).

The evaluation is conducted in CARLA, extended to support command-conditioned evaluation. Each route is annotated with segment-wise target speeds, and the policy receives a target speed at runtime based on the current position. Overtaking scenarios are constructed by spawning a lead vehicle at lower speed ahead of the ego vehicle; the agent must either overtake or remain behind according to the command (Shao et al., 26 Mar 2026).

The benchmark defines 48 evaluation cases total, with 16 per difficulty level. The levels are differentiated by the degree to which command adherence must be resolved under interaction and traffic complexity (Shao et al., 26 Mar 2026).

Difficulty	Definition
Easy	no interfering vehicles; pure speed adherence
Medium	single slow vehicle in ego lane; must overtake or follow as commanded
Hard	complex traffic adapted from CARLA Leaderboard v2, plus overtake/follow

Each evaluation case uses four routes with four different speed command profiles to avoid memorization and force command-driven behavior (Shao et al., 26 Mar 2026). The hard split includes 13 challenging scenarios from CARLA Leaderboard v2, such as obstacle circumvention, dense merging, and intersections with pedestrian yielding (Shao et al., 26 Mar 2026).

A notable design choice is full compatibility with Bench2Drive metrics. The benchmark reports all standard CARLA/Bench2Drive metrics unchanged, so controllability can be studied without severing comparison to prior closed-loop autonomous-driving systems (Shao et al., 26 Mar 2026). This makes it possible to ask whether improved speed adherence or command execution degrades, preserves, or improves conventional driving performance.

3. Metrics and formal target-speed specification

The principal new metric is the Speed-Adherence Score. Let the reference route be $\mathcal{R}$ with length $L$, and let the closed-loop rollout yield $\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$, where $\vec{v}_i$ is the instantaneous velocity vector at step $i$. The ego position is projected to arc length $s_i \in [0, L]$. The actual speed and target speed are defined as

$v_i^{\text{actual}} = \| \vec{v}_i \|_2$

$v_i^{\text{target}} = v^{\text{target}}(s_i).$

To downweight stationary samples, the benchmark uses distance-based weighting,

$w_i = \| (x_i, y_i) - (x_{i-1}, y_{i-1}) \|_2.$

The relative error and per-step score are

$$e_i = \frac{|v_i^{\text{actual}} - v_i^{\text{target}}|}{\max(v_i^{\text{target}}, \epsilon)}$$

$L$0

The overall score is

$L$1

For the Follow command, when the ego is constrained by a slower leader, i.e., $L$2, the penalty is softened; the exact softening formula is not specified (Shao et al., 26 Mar 2026).

The Overtake Score is defined at route level. For each route, all overtake/follow scenarios must be triggered and executed correctly. Each scenario is scored as 100 for success or 0 for failure, and scenarios that fail to trigger are counted as failures. The final route-level Overtake Score is the success ratio, namely the mean over required scenarios (Shao et al., 26 Mar 2026).

Bench2Drive-Speed also formalizes how segment-wise target speeds are assigned and queried at runtime. Let

$L$3

denote route segments with normalized progress bounds and segment target speed $L$4. Given route keypoints $L$5, the cumulative distances are

$L$6

with total length $L$7. Segment bounds are converted to arc length as $L$8, producing a global speed plan

$L$9

At runtime, the target speed is queried by nearest route keypoint,

$\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$0

This formulation makes the command-conditioned control problem spatial rather than purely temporal: the target speed is tied to progress along the route, not merely to wall-clock time (Shao et al., 26 Mar 2026). A plausible implication is that this reduces ambiguity when evaluating agents that may slow down or speed up relative to one another.

4. Dataset and supervision strategies

To support training of speed-conditioned policies, the benchmark introduces CustomizedSpeedDataset, composed of 2,100 routes collected in CARLA with a modified rule-based expert from the PDM/PDM-Lite family (Shao et al., 26 Mar 2026). The dataset is balanced across overtake and follow conditions:

• Medium split: 570 overtake and 570 follow.
• Hard split: 480 overtake and 480 follow.
• Total: 1,050 overtake and 1,050 follow.

The dataset covers all predefined CARLA weathers with uniform distribution. Its town distribution follows town size and route availability, with many routes in Town12. Clip durations and total hours are not specified (Shao et al., 26 Mar 2026).

The paper contrasts two supervision sources for target speed. The first is an expert target speed, where PDM’s internal cruising-speed hyperparameter acts as the desired target speed under hazard-free conditions; the paper explicitly notes that this requires privileged access to the expert planner. The second is a re-annotation strategy that derives a virtual target speed from regular driving data (Shao et al., 26 Mar 2026).

The re-annotation pipeline first computes a tendency speed:

$\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$1

with future horizon $\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$2 frames (Shao et al., 26 Mar 2026). It then extrapolates a virtual target speed without leakage:

$\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$3

$\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$4

where $\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$5. The update $\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$6 is clipped to $\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$7 (Shao et al., 26 Mar 2026). Two configurations are provided:

• Long: $\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$8 m/s, $\{(x_i, y_i, \vec{v}_i)\}_{i=1}^N$9 s.
• Short: $\vec{v}_i$0 m/s, $\vec{v}_i$1 s.

The paper states that $\vec{v}_i$2 is not specified (Shao et al., 26 Mar 2026).

Within-route target-speed commands vary across route segments to prevent shortcut learning. For lead-vehicle generation in overtake/follow scenarios, the lead speed is sampled as

$\vec{v}_i$3

with $\vec{v}_i$4 and $\vec{v}_i$5 described as small constants whose exact values are not specified (Shao et al., 26 Mar 2026). This guarantees a leader slower than the ego’s target speed and biases the scenario toward a genuine longitudinal-control decision.

The benchmark also evaluates different training-data regimes: Bench2Drive1K without explicit speed commands but with re-annotation support, Customized with Expert or Virtual target speed, and mixtures of Bench2Drive1K with Customized (Shao et al., 26 Mar 2026). The benchmark’s central empirical question is whether virtual target-speed supervision from regular driving logs can substitute for expensive expert demonstrations.

5. Baselines, training setup, and empirical performance

The principal baseline is the TCP trajectory-only variant. It is chosen for simplicity and Bench2Drive compatibility. Its inputs are RGB camera features, ego state, target waypoint, and the new commands, namely target speed and overtake/follow. In the reported baseline, the target-speed and overtake command are concatenated with the encoded ego-state and visual features and fed to the planning head; the target-speed command also conditions trajectory generation. Only the trajectory output is executed in closed loop (Shao et al., 26 Mar 2026).

Training uses a single NVIDIA A100 and PyTorch Lightning. The optimizer is Adam with initial learning rate $\vec{v}_i$6 and weight decay $\vec{v}_i$7. The schedule is StepLR with decay factor 0.5 every 30 epochs. Training runs for 60 epochs with batch size 300 (Shao et al., 26 Mar 2026).

On the 48-route Drive evaluation, speed conditioning improves Speed-Adherence Score substantially relative to the unconditioned baseline, and the best adherence is achieved by models trained with virtual target-speed supervision or mixtures of Bench2Drive1K and Customized data (Shao et al., 26 Mar 2026).

Training setup	Speed-Adherence A/E/M/H	Overtake A/M/H
TCP w/o Speed Command, Customized-only	41.54 / 42.00 / 40.95 / 41.67	18.75 / 37.50 / 0.00
TCP-Speed Expert2.1k	68.79 / 76.80 / 65.38 / 64.20	21.88 / 37.50 / 6.25
TCP-Speed Virtual2.1k, variant 1	69.23 / 76.18 / 68.71 / 62.81	40.63 / 56.25 / 25.00
TCP-Speed Virtual2.1k, variant 2	68.36 / 73.17 / 67.93 / 63.98	40.63 / 56.25 / 25.00
TCP w/o Speed Command, Bench2Drive1K-only	41.36 / 42.03 / 40.72 / 41.33	37.50 / 50.00 / 25.00
TCP-Speed, Bench2Drive1K-only, variant 1	67.95 / 71.01 / 69.63 / 63.21	31.25 / 50.00 / 12.50
TCP-Speed, Bench2Drive1K-only, variant 2	65.61 / 67.80 / 65.69 / 63.34	31.25 / 43.75 / 18.75
TCP w/o Speed Command, Bench2Drive1K + Customized	41.60 / 42.11 / 40.98 / 41.70	25.00 / 50.00 / 0.00
TCP-Speed + Virtual2.1k, variant 1	71.65 / 77.33 / 72.63 / 64.99	34.38 / 50.00 / 18.75
TCP-Speed + Virtual2.1k, variant 2	71.21 / 79.02 / 69.16 / 66.40	34.38 / 50.00 / 18.75

These results support three conclusions stated in the benchmark paper. First, speed conditioning substantially improves Speed-Adherence Score across Easy, Medium, and Hard relative to the unconditioned baseline. Second, virtual target-speed supervision matches or slightly exceeds expert supervision in adherence and often yields higher Overtake Scores. Third, virtual-short tends to be more stable than virtual-long, which the paper attributes to less extrapolation noise (Shao et al., 26 Mar 2026).

The benchmark also evaluates standard closed-loop performance on Bench2Drive-220. Here the principal finding is that target-speed conditioning improves Efficiency markedly while keeping Driving Score and Success Rate comparable to the unconditioned baseline; Comfort decreases slightly in some configurations, and the paper describes the change as minor (Shao et al., 26 Mar 2026).

Training setup	Bench2Drive-220 metrics
TCP w/o Speed Command, Bench2Drive1K-only	DS 49.30; SR 20.45; Eff 78.78; Comfort 22.96
TCP-Speed, Bench2Drive1K-only, variant 1	DS 54.15; SR 22.73; Eff 195.48; Comfort 20.92
TCP-Speed, Bench2Drive1K-only, variant 2	DS 51.84; SR 21.36; Eff 195.96; Comfort 22.64
TCP w/o Speed Command, Bench2Drive1K + Customized	DS 55.23; SR 25.45; Eff 77.69; Comfort 27.26
TCP-Speed + Virtual2.1k, variant 1	DS 53.66; SR 22.73; Eff 192.17; Comfort 21.59
TCP-Speed + Virtual2.1k, variant 2	DS 54.80; SR 24.09; Eff 195.63; Comfort 24.14

Overtaking remains the harder control axis. The paper states that executing overtaking commands remains challenging due to the inherent difficulty of interactive behaviors, and that failures often involve collisions, unsafe merges, or missed opportunities under Hard conditions (Shao et al., 26 Mar 2026). This is consistent with the relatively modest Overtake Scores on the hard split even when speed adherence is strong.

6. Relationship to Bench2Drive, closed-loop speed evaluation, and related extensions

Bench2Drive-Speed inherits its closed-loop setting from Bench2Drive, whose official aggregate metrics are Driving Score and Success Rate rather than a standalone speed score. In Bench2Drive, speed-related behavior is assessed implicitly through multiplicative penalties such as Too Slow with factor 0.70, Scenario Timeout with factor 0.70, Route Timeout as immediate shutdown, and Agent Blocked after 180 seconds of no action (Jia et al., 2024). The parent benchmark therefore already penalizes sluggish or stalled behavior, but it does not quantify whether a policy follows a user-specified target speed (Jia et al., 2024).

This distinction is sharpened by the later cross-benchmark study comparing NAVSIM and Bench2Drive. That study reports that Bench2Drive’s closed-loop scoring explicitly penalizes slow progress, and that speed/progress tendencies are the dominant driver of closed-loop success. Across eight methods with fully paired data, NAVSIM’s Ego Progress is the strongest single predictor of Bench2Drive Driving Score, with Spearman correlations to Driving Score of EP $\vec{v}_i$8, DAC $\vec{v}_i$9, TTC $i$0, and NC $i$1; aggregate PDMS reaches Spearman $i$2 but remains non-monotonic because slow yet safe methods can suffer slow-driving penalties and timeouts (Wang et al., 30 Apr 2026). This suggests that Bench2Drive-Speed formalizes a control axis—desired speed adherence—that is highly consequential for closed-loop success but only indirectly represented in the parent benchmark.

A related extension, Bench2Drive-VL, brings closed-loop evaluation to VLM-based driving and includes speed-related reasoning tasks such as Q7 (“What is the current speed limit?”), Q8 (“Does the ego vehicle need to brake? Why?”), and Q50, which outputs a Speed key from the set $i$3 (Jia et al., 1 Apr 2026). Bench2Drive-VL therefore studies speed as a reasoning-and-control problem grounded in visual-language inputs, whereas Bench2Drive-Speed studies speed as a user-conditioned control objective with explicit adherence metrics (Jia et al., 1 Apr 2026).

The three benchmarks are complementary. Bench2Drive establishes multi-ability closed-loop evaluation (Jia et al., 2024); Bench2Drive-Speed adds controllability with explicit desired-speed and overtake/follow commands (Shao et al., 26 Mar 2026); Bench2Drive-VL studies closed-loop speed reasoning in multimodal language-conditioned agents (Jia et al., 1 Apr 2026). The correlation study then provides an external quantitative argument that progress-related behavior is a major determinant of closed-loop success in Bench2Drive (Wang et al., 30 Apr 2026).

7. Limitations, caveats, and research directions

The benchmark paper identifies several limitations. The dataset scope is 2,100 CARLA routes, and real-world generalization and long-tail scenarios require further scaling. Clip durations and total hours are not reported (Shao et al., 26 Mar 2026). Overtaking remains difficult and appears to require better interaction and intent modeling (Shao et al., 26 Mar 2026).

Instruction conflicts are also only partially resolved. The benchmark evaluates speed adherence jointly with safety, rules, and comfort, and the paper is explicit that speed commands are not meant to override safety. When the ego is constrained by a slower leader under the Follow command, penalties can be softened, but the exact softening policy is not fully specified (Shao et al., 26 Mar 2026). Future work, according to the paper, should formalize rule-based overrides and safety constraints when commands conflict with traffic context (Shao et al., 26 Mar 2026).

The re-annotation method is practical but imperfect. Virtual target-speed extrapolation can be noisy, especially with long horizons; improved trend modeling and uncertainty handling are identified as promising directions (Shao et al., 26 Mar 2026). This limitation is reflected in the benchmark’s own comparison between short and long virtual-target configurations, where shorter extrapolation is reported as slightly more stable (Shao et al., 26 Mar 2026).

The benchmark nevertheless establishes a clear experimental template for controllable closed-loop autonomous driving. It provides an explicit command interface, closed-loop scenarios where command compliance can be isolated and measured, and a practical re-annotation pipeline showing that regular driving data can supervise speed control competitively with expert demonstrations (Shao et al., 26 Mar 2026). In that sense, Bench2Drive-Speed turns desired-speed adherence from an implicit stylistic attribute into a measurable benchmark objective embedded within the existing closed-loop evaluation culture of Bench2Drive.