---
title: CustomizedSpeedDataset for Speed-Controlled Driving
url: https://www.emergentmind.com/topics/customizedspeeddataset
type: topic
---

# CustomizedSpeedDataset for Speed-Controlled Driving

CustomizedSpeedDataset is the dataset component of Bench2Drive-Speed, also referred to as Drive, introduced for desired-speed conditioned autonomous driving. It is designed to make explicit target-speed control and overtake/follow preference a supervised and measurable problem, rather than treating driving solely as trajectory imitation under implicit behavioral priors. The dataset is composed of 2,100 CARLA clips annotated with expert demonstrations and explicit user-facing commands, and it is paired with benchmark metrics such as Speed-Adherence Score and Overtake Score for evaluating whether a policy obeys requested speed and interaction style while retaining standard autonomous-driving performance [2603.25672].

## 1. Concept and motivation

CustomizedSpeedDataset was created to address a gap in autonomous-driving datasets and benchmarks: existing resources typically provide ego trajectories, sensor inputs, map information, and traffic annotations, but no explicit target-speed supervision and no principled way to evaluate whether a policy follows a user-specified speed or overtaking preference. Its immediate use case is practical human-facing control, where a user may wish to specify a desired speed or whether the vehicle should overtake or follow [2603.25672].

A defining distinction is that the dataset treats speed controllability as a first-class objective. Rather than inferring preferences indirectly from demonstrations alone, it introduces explicit target-speed commands and overtake/follow instructions into the supervision and evaluation loop. The paper further studies two supervision regimes: expert demonstration speed, derived from the expert planner’s internal cruising-speed hyperparameter, and virtual target speed, obtained by re-annotating regular driving data from future observed speeds. This makes the dataset both a benchmark artifact and a methodological testbed for how speed-conditioning signals should be constructed [2603.25672].

A common misconception is that the dataset is merely a standard CARLA driving corpus with an extra scalar label. In fact, its purpose is narrower and more structured: it is intended to train and evaluate whether a closed-loop driving policy can obey explicit longitudinal and interaction commands under controlled scenario design.

## 2. Dataset composition and collection protocol

CustomizedSpeedDataset contains **2,100 driving scenes / clips** collected in CARLA. The dataset is organized along two main axes: difficulty and behavior command. The training composition explicitly includes medium and hard difficulty settings, and both overtake and follow commands are balanced across those settings [2603.25672].

| Difficulty | Overtake | Follow |
|---|---:|---:|
| Medium | 570 | 570 |
| Hard | 480 | 480 |

This yields **1,050 overtake** clips, **1,050 follow** clips, and **2,100 total** clips. The paper also notes that easy routes are used in evaluation [2603.25672].

The data is collected in CARLA using a modified expert policy based on **PDM / PDM-Lite**, itself inspired by **PDM-Closed** and **IDM**. The expert has access to privileged simulator information and is extended to follow speed-related commands, handle newly introduced scenarios, and generate demonstrations under controlled settings. Each scene includes visual sensor inputs, ego-state information, bounding boxes or object annotations, an overtake/follow instruction, and a target-speed command. The dataset retains the same general data format as Bench2Drive, but adds the new speed-command channels [2603.25672].

The collection design is significant because it aligns data generation, command conditioning, and benchmark evaluation within a single simulation framework. This differs from datasets that contain rich motion or perception data but no explicit mechanism for testing command adherence.

## 3. Speed annotation and virtual target-speed re-annotation

The dataset supports two forms of speed supervision. The first is **expert demonstration speed**, obtained from the expert planner’s internal cruising-speed hyperparameter. The second is **virtual target speed**, introduced because expert demonstration supervision is precise but impractical for real-world datasets, as it depends on access to internal planner hyperparameters [2603.25672].

The re-annotation strategy defines a **tendency speed** from a short future horizon of speeds. For frame \(t\), with horizon \(F\) frames and implementation using \(F=40\), the tendency speed is

$$
v_t^{\text{tend}} =
\begin{cases}
\max\limits_{i \in [1, F]} v_{t+i}, & \text{if } v_{t+1} > v_t \quad (\text{acceleration trend}) \\
\min\limits_{i \in [1, F]} v_{t+i}, & \text{if } v_{t+1} < v_t \quad (\text{deceleration trend}) \\
v_t, & \text{otherwise.}
\end{cases}
$$

The **virtual target speed** is then extrapolated from the tendency speed:

$$
v_t^{\text{virt}} = \max \big( v_t^{\text{tend}} + \Delta v_t, 0 \big), \quad
\Delta v_t = (v_t^{\text{tend}} - v_{t-1}^{\text{tend}})\cdot \text{FPS}\cdot r, \quad
r \sim \mathcal{U}(T_{\min}, T_{\max}),
$$

with \(\text{FPS}=10\), and with \(\Delta v_t\) clipped to \([-\text{MAX\_EXTEND}, \text{MAX\_EXTEND}]\) [2603.25672].

Two re-annotation settings are defined. **Long** uses \(\text{MAX\_EXTEND}=10.0\,\mathrm{m/s}\) and \(T_{\max}=3.0\,\mathrm{s}\). **Short** uses \(\text{MAX\_EXTEND}=3.0\,\mathrm{m/s}\) and \(T_{\max}=1.5\,\mathrm{s}\). The stated intuition is that Long captures longer-horizon trends but may become noisier, whereas Short preserves local detail and is more stable. The methodological importance of this scheme is that it creates target-speed supervision from ordinary driving traces without requiring privileged expert-controller internals [2603.25672].

## 4. Scenario structure, route design, and command diversity

The dataset is split by difficulty level and command type, with medium and hard explicitly used for training composition and easy appearing in evaluation. Training routes are balanced across these domains, and target-speed commands vary within a route across segments rather than staying fixed. The paper states that this reduces shortcut learning and forces the model to condition on the speed command [2603.25672].

The hard subset includes **13 of the most challenging CARLA Leaderboard v2 scenarios**, including static obstacle circumvention, dense traffic merging, multi-directional junction negotiation, pedestrian or cyclist yielding, accident avoidance, and construction obstacle avoidance. Weather is uniformly distributed across simulator weather types, and town distribution roughly follows each town’s route availability [2603.25672].

For evaluation, the benchmark defines **48 cases**: **16 scenarios** for each of the three difficulty levels—easy, medium, and hard—with each case containing **4 routes** and being paired with **4 speed-command sets**. This route design is intended to prevent memorization of a fixed route or traffic configuration [2603.25672].

A specialized **OvertakeRoute** scenario is introduced to operationalize the interaction command. A slow lead vehicle is placed ahead of the ego vehicle; under **overtake**, the ego must pass it, and under **follow**, the ego must stay behind it. Scenario parameters include front vehicle speed, initial distance, behavior type, and oncoming-vehicle frequency in the two-way variant. Scenarios activate only when the ego enters a trigger region, and failure to activate is counted as failure, preventing models from avoiding the difficult portion of the route [2603.25672].

## 5. Use in policy training and benchmark metrics

CustomizedSpeedDataset is used to train speed-conditioned policies from visual inputs, ego state, target-speed command, and overtake/follow command. The principal baseline is **TCP**, adapted so that target speed and overtaking instruction are concatenated into the input. The paper compares multiple training configurations, including **Expert2.1k**, **Virtual2.1k**, **Bench2Drive1K**, **Bench2Drive1K + Virtual2.1k**, **Bench2Drive1K + Customized**, and a reference model without speed-command input. Evaluation is performed in closed loop on the 48 Drive routes and on standard Bench2Drive routes to test whether speed conditioning harms normal autonomous-driving capabilities [2603.25672].

The central command-following metric is **Speed-Adherence Score**. Given a route \(\mathcal{R}\) of length \(L\), a closed-loop trajectory \(\{(x_i, y_i, \mathbf{v}_i)\}_{i=1}^N\), route projection at arc-length \(s_i\), actual speed \(v_i^{\text{actual}} = \|\mathbf{v}_i\|_2\), and target speed \(v_i^{\text{target}} = v^{\text{target}}(s_i)\), the weighting and scoring terms are

$$
w_i = \|(x_i,y_i)-(x_{i-1},y_{i-1})\|_2,
$$

$$
e_i = \frac{|v_i^{\text{actual}} - v_i^{\text{target}}|}{\max(v_i^{\text{target}}, \epsilon)},
$$

$$
\text{score}_i = \exp(-\alpha e_i),
$$

and the route-level score is

$$
\text{Score}_{\text{speed}} =
\frac{\sum_{i=2}^{N} w_i \cdot \text{score}_i}
{\sum_{i=2}^{N} w_i}.
$$

An implementation detail is that in **Follow** scenarios, if the ego is constrained by a slower lead vehicle, the penalty is softened when \(v^{\text{lead}} \le v_i^{\text{actual}} < v_i^{\text{target}}\), so the agent is not overly punished for being safely blocked [2603.25672].

The second command metric is **Overtake Score**. For each route with explicit overtake/follow commands, a scenario is successful only if it is properly triggered and the commanded behavior is executed correctly. Each scenario receives a binary score of **100 or 0**, and the route score is the success ratio. If a scenario fails to activate, it is counted as failure. In addition to these command-specific metrics, the benchmark continues to report standard autonomous-driving measures such as **Driving Score**, **Success Rate**, **efficiency**, **comfort**, and **multi-ability metrics** [2603.25672].

## 6. Empirical findings and position within speed-related dataset research

The paper’s main empirical conclusion is that **re-annotated virtual target speed can work nearly as well as expert speed labels**. Models trained with **Virtual2.1k** achieve comparable Speed-Adherence Scores to those trained with **Expert2.1k**, and this behavior is reported across multiple difficulty levels. The **Short** virtual annotation is often slightly better or more stable than **Long**. At the same time, speed-conditioned models preserve **Driving Score** and **Success Rate** roughly comparably to non-speed-conditioned baselines, with only slight comfort changes and often improved efficiency. By contrast, **overtaking** remains difficult, especially in **Hard** scenarios, and the paper concludes that target-speed following is much easier than consistent overtaking behavior [2603.25672].

The comparison between expert and virtual supervision is nuanced. Expert demonstrations can yield slightly better traditional driving scores and success rates because they reflect complex context-aware planning, whereas virtual target speed can produce better or comparable command adherence and sometimes better overtake or efficiency behavior because it is derived directly from observed trajectories and is easier for the model to fit. This establishes a tradeoff between planner-derived supervision quality and re-annotation scalability [2603.25672].

Within the broader literature, CustomizedSpeedDataset occupies a distinct niche. **100-DrivingStyle** is a naturalistic benchmark with explicit five-level driving-style labels from self-evaluation and expert evaluation, focused on human-centered driving-style analysis rather than commanded speed control [2406.07894]. **PDB** preserves driver identity and multimodal physiological context under controlled route and vehicle conditions, emphasizing personalization in driving behavior rather than explicit desired-speed conditioning [2503.06477]. **DistPre** addresses detector-specific traffic-speed prediction in large freeway networks through automatic LSTM customization and sharing, which is a network-level forecasting problem rather than an ego-policy control benchmark [2005.04788]. **ForeSpeed** targets forensic vehicle speed estimation from CCTV footage with known-speed passes and geometry restoration, prioritizing measurement uncertainty rather than policy conditioning [2512.19364]. **RACECAR** provides high-speed autonomous racing data for localization, perception, and mapping at up to 170 mph, but it does not frame user-specified target speed and overtake/follow compliance as the primary supervised task [2306.03252]. **SZUR-Acoustic** pairs urban roadside acoustic recordings with three speed classes for traffic monitoring and speed-range classification, again focusing on inference from observations rather than command-conditioned closed-loop control [2506.21269].

Taken together, these comparisons indicate that CustomizedSpeedDataset is best understood not as a generic speed dataset, but as a benchmark resource for controllable autonomous driving. Its central research contribution is to formalize desired speed and overtaking preference as explicit inputs, trainable supervision targets, and quantitative closed-loop evaluation criteria.

Source: https://www.emergentmind.com/topics/customizedspeeddataset