CustomizedSpeedDataset for Speed-Controlled Driving

Updated 5 July 2026

CustomizedSpeedDataset is a dedicated benchmark that integrates explicit target-speed and overtaking commands to transform autonomous driving into a supervised control problem.
It comprises 2,100 annotated CARLA clips balanced across medium and hard difficulties, evaluated using Speed-Adherence and Overtake Scores.
The dataset supports both expert demonstration and virtual target speed re-annotation, enabling scalable supervision for command-conditioned driving policies.

CustomizedSpeedDataset is the dataset component of Bench2Drive-Speed, also referred to as Drive, introduced for desired-speed conditioned autonomous driving. It is designed to make explicit target-speed control and overtake/follow preference a supervised and measurable problem, rather than treating driving solely as trajectory imitation under implicit behavioral priors. The dataset is composed of 2,100 CARLA clips annotated with expert demonstrations and explicit user-facing commands, and it is paired with benchmark metrics such as Speed-Adherence Score and Overtake Score for evaluating whether a policy obeys requested speed and interaction style while retaining standard autonomous-driving performance (Shao et al., 26 Mar 2026).

1. Concept and motivation

CustomizedSpeedDataset was created to address a gap in autonomous-driving datasets and benchmarks: existing resources typically provide ego trajectories, sensor inputs, map information, and traffic annotations, but no explicit target-speed supervision and no principled way to evaluate whether a policy follows a user-specified speed or overtaking preference. Its immediate use case is practical human-facing control, where a user may wish to specify a desired speed or whether the vehicle should overtake or follow (Shao et al., 26 Mar 2026).

A defining distinction is that the dataset treats speed controllability as a first-class objective. Rather than inferring preferences indirectly from demonstrations alone, it introduces explicit target-speed commands and overtake/follow instructions into the supervision and evaluation loop. The paper further studies two supervision regimes: expert demonstration speed, derived from the expert planner’s internal cruising-speed hyperparameter, and virtual target speed, obtained by re-annotating regular driving data from future observed speeds. This makes the dataset both a benchmark artifact and a methodological testbed for how speed-conditioning signals should be constructed (Shao et al., 26 Mar 2026).

A common misconception is that the dataset is merely a standard CARLA driving corpus with an extra scalar label. In fact, its purpose is narrower and more structured: it is intended to train and evaluate whether a closed-loop driving policy can obey explicit longitudinal and interaction commands under controlled scenario design.

2. Dataset composition and collection protocol

CustomizedSpeedDataset contains 2,100 driving scenes / clips collected in CARLA. The dataset is organized along two main axes: difficulty and behavior command. The training composition explicitly includes medium and hard difficulty settings, and both overtake and follow commands are balanced across those settings (Shao et al., 26 Mar 2026).

Difficulty	Overtake	Follow
Medium	570	570
Hard	480	480

This yields 1,050 overtake clips, 1,050 follow clips, and 2,100 total clips. The paper also notes that easy routes are used in evaluation (Shao et al., 26 Mar 2026).

The data is collected in CARLA using a modified expert policy based on PDM / PDM-Lite, itself inspired by PDM-Closed and IDM. The expert has access to privileged simulator information and is extended to follow speed-related commands, handle newly introduced scenarios, and generate demonstrations under controlled settings. Each scene includes visual sensor inputs, ego-state information, bounding boxes or object annotations, an overtake/follow instruction, and a target-speed command. The dataset retains the same general data format as Bench2Drive, but adds the new speed-command channels (Shao et al., 26 Mar 2026).

The collection design is significant because it aligns data generation, command conditioning, and benchmark evaluation within a single simulation framework. This differs from datasets that contain rich motion or perception data but no explicit mechanism for testing command adherence.

3. Speed annotation and virtual target-speed re-annotation

The dataset supports two forms of speed supervision. The first is expert demonstration speed, obtained from the expert planner’s internal cruising-speed hyperparameter. The second is virtual target speed, introduced because expert demonstration supervision is precise but impractical for real-world datasets, as it depends on access to internal planner hyperparameters (Shao et al., 26 Mar 2026).

The re-annotation strategy defines a tendency speed from a short future horizon of speeds. For frame $t$ , with horizon $F$ frames and implementation using $F=40$ , the tendency speed is

$v_t^{\text{tend}} = \begin{cases} \max\limits_{i \in [1, F]} v_{t+i}, & \text{if } v_{t+1} > v_t \quad (\text{acceleration trend}) \ \min\limits_{i \in [1, F]} v_{t+i}, & \text{if } v_{t+1} < v_t \quad (\text{deceleration trend}) \ v_t, & \text{otherwise.} \end{cases}$

The virtual target speed is then extrapolated from the tendency speed:

$v_t^{\text{virt}} = \max \big( v_t^{\text{tend}} + \Delta v_t, 0 \big), \quad \Delta v_t = (v_t^{\text{tend}} - v_{t-1}^{\text{tend}})\cdot \text{FPS}\cdot r, \quad r \sim \mathcal{U}(T_{\min}, T_{\max}),$

with $\text{FPS}=10$ , and with $\Delta v_t$ clipped to $[-\text{MAX\_EXTEND}, \text{MAX\_EXTEND}]$ (Shao et al., 26 Mar 2026).

Two re-annotation settings are defined. Long uses $\text{MAX\_EXTEND}=10.0\,\mathrm{m/s}$ and $T_{\max}=3.0\,\mathrm{s}$ . Short uses $F$ 0 and $F$ 1. The stated intuition is that Long captures longer-horizon trends but may become noisier, whereas Short preserves local detail and is more stable. The methodological importance of this scheme is that it creates target-speed supervision from ordinary driving traces without requiring privileged expert-controller internals (Shao et al., 26 Mar 2026).

4. Scenario structure, route design, and command diversity

The dataset is split by difficulty level and command type, with medium and hard explicitly used for training composition and easy appearing in evaluation. Training routes are balanced across these domains, and target-speed commands vary within a route across segments rather than staying fixed. The paper states that this reduces shortcut learning and forces the model to condition on the speed command (Shao et al., 26 Mar 2026).

The hard subset includes 13 of the most challenging CARLA Leaderboard v2 scenarios, including static obstacle circumvention, dense traffic merging, multi-directional junction negotiation, pedestrian or cyclist yielding, accident avoidance, and construction obstacle avoidance. Weather is uniformly distributed across simulator weather types, and town distribution roughly follows each town’s route availability (Shao et al., 26 Mar 2026).

For evaluation, the benchmark defines 48 cases: 16 scenarios for each of the three difficulty levels—easy, medium, and hard—with each case containing 4 routes and being paired with 4 speed-command sets. This route design is intended to prevent memorization of a fixed route or traffic configuration (Shao et al., 26 Mar 2026).

A specialized OvertakeRoute scenario is introduced to operationalize the interaction command. A slow lead vehicle is placed ahead of the ego vehicle; under overtake, the ego must pass it, and under follow, the ego must stay behind it. Scenario parameters include front vehicle speed, initial distance, behavior type, and oncoming-vehicle frequency in the two-way variant. Scenarios activate only when the ego enters a trigger region, and failure to activate is counted as failure, preventing models from avoiding the difficult portion of the route (Shao et al., 26 Mar 2026).

5. Use in policy training and benchmark metrics

CustomizedSpeedDataset is used to train speed-conditioned policies from visual inputs, ego state, target-speed command, and overtake/follow command. The principal baseline is TCP, adapted so that target speed and overtaking instruction are concatenated into the input. The paper compares multiple training configurations, including Expert2.1k, Virtual2.1k, Bench2Drive1K, Bench2Drive1K + Virtual2.1k, Bench2Drive1K + Customized, and a reference model without speed-command input. Evaluation is performed in closed loop on the 48 Drive routes and on standard Bench2Drive routes to test whether speed conditioning harms normal autonomous-driving capabilities (Shao et al., 26 Mar 2026).

The central command-following metric is Speed-Adherence Score. Given a route $F$ 2 of length $F$ 3, a closed-loop trajectory $F$ 4, route projection at arc-length $F$ 5, actual speed $F$ 6, and target speed $F$ 7, the weighting and scoring terms are

$F$ 8

$F$ 9

$F=40$ 0

and the route-level score is

$F=40$ 1

An implementation detail is that in Follow scenarios, if the ego is constrained by a slower lead vehicle, the penalty is softened when $F=40$ 2, so the agent is not overly punished for being safely blocked (Shao et al., 26 Mar 2026).

The second command metric is Overtake Score. For each route with explicit overtake/follow commands, a scenario is successful only if it is properly triggered and the commanded behavior is executed correctly. Each scenario receives a binary score of 100 or 0, and the route score is the success ratio. If a scenario fails to activate, it is counted as failure. In addition to these command-specific metrics, the benchmark continues to report standard autonomous-driving measures such as Driving Score, Success Rate, efficiency, comfort, and multi-ability metrics (Shao et al., 26 Mar 2026).

The paper’s main empirical conclusion is that re-annotated virtual target speed can work nearly as well as expert speed labels. Models trained with Virtual2.1k achieve comparable Speed-Adherence Scores to those trained with Expert2.1k, and this behavior is reported across multiple difficulty levels. The Short virtual annotation is often slightly better or more stable than Long. At the same time, speed-conditioned models preserve Driving Score and Success Rate roughly comparably to non-speed-conditioned baselines, with only slight comfort changes and often improved efficiency. By contrast, overtaking remains difficult, especially in Hard scenarios, and the paper concludes that target-speed following is much easier than consistent overtaking behavior (Shao et al., 26 Mar 2026).

The comparison between expert and virtual supervision is nuanced. Expert demonstrations can yield slightly better traditional driving scores and success rates because they reflect complex context-aware planning, whereas virtual target speed can produce better or comparable command adherence and sometimes better overtake or efficiency behavior because it is derived directly from observed trajectories and is easier for the model to fit. This establishes a tradeoff between planner-derived supervision quality and re-annotation scalability (Shao et al., 26 Mar 2026).

Within the broader literature, CustomizedSpeedDataset occupies a distinct niche. 100-DrivingStyle is a naturalistic benchmark with explicit five-level driving-style labels from self-evaluation and expert evaluation, focused on human-centered driving-style analysis rather than commanded speed control (Zhang et al., 2024). PDB preserves driver identity and multimodal physiological context under controlled route and vehicle conditions, emphasizing personalization in driving behavior rather than explicit desired-speed conditioning (Wei et al., 9 Mar 2025). DistPre addresses detector-specific traffic-speed prediction in large freeway networks through automatic LSTM customization and sharing, which is a network-level forecasting problem rather than an ego-policy control benchmark (Lee et al., 2020). ForeSpeed targets forensic vehicle speed estimation from CCTV footage with known-speed passes and geometry restoration, prioritizing measurement uncertainty rather than policy conditioning (Iuliani et al., 22 Dec 2025). RACECAR provides high-speed autonomous racing data for localization, perception, and mapping at up to 170 mph, but it does not frame user-specified target speed and overtake/follow compliance as the primary supervised task (Kulkarni et al., 2023). SZUR-Acoustic pairs urban roadside acoustic recordings with three speed classes for traffic monitoring and speed-range classification, again focusing on inference from observations rather than command-conditioned closed-loop control (Fan et al., 26 Jun 2025).

Taken together, these comparisons indicate that CustomizedSpeedDataset is best understood not as a generic speed dataset, but as a benchmark resource for controllable autonomous driving. Its central research contribution is to formalize desired speed and overtaking preference as explicit inputs, trainable supervision targets, and quantitative closed-loop evaluation criteria.