highD: German Highway Vehicle Trajectories

• highD is a dataset of naturalistic vehicle trajectories on German highways designed for scenario-based testing of highly automated driving systems.
• It employs aerial observations with a high-resolution camera and advanced post-processing to achieve sub-decimeter accuracy in vehicle tracking.
• The dataset includes detailed infrastructure annotations, maneuver classifications, and safety metrics, enabling diverse analyses from traffic flow to trajectory prediction.

highD is a large-scale collection of naturalistic vehicle trajectories on German highways, created specifically to support scenario-based testing and validation of highly automated driving systems. It combines aerial observation, post-processed trajectories, road infrastructure annotations, maneuver categories, surrounding-vehicle relations, and safety surrogates such as DHW, THW, and TTC. In practice, highD functions both as a dataset and as a scenario source: it has been used for traffic-flow analysis, criticality analysis, scenario extraction, driver-behavior modeling, trajectory prediction, lane-change intention prediction, and autonomous-vehicle decision-making on highways (Krajewski et al., 2018, Kruber et al., 2019).

1. Origin, scope, and design objectives

The original motivation for highD is scenario-based safety validation. The dataset paper frames this need in terms of naturalistic behavior, static scenario description, dynamic scenario description, effort effectiveness, and flexibility, and proposes aerial measurement as a way to satisfy these requirements for highway traffic (Krajewski et al., 2018). Later work continues to treat highD as the straight-highway benchmark within a broader family of drone datasets, often contrasting it with ramp-rich datasets such as exiD (Shi et al., 30 Dec 2025).

The core reported characteristics are as follows (Krajewski et al., 2018):

Characteristic	Reported value
Recordings	60
Locations	6 German highway locations around Cologne
Recording duration	16.5 hours
Observed road segment per recording	about 420 m
Vehicles	110,000
Total driven distance	45,000 km
Complete lane changes	5,600
Median vehicle visibility	13.6 s
Highway geometry	2 or 3 lanes per direction
Video format	4096 × 2160 at 25 fps

The dataset is also positioned against NGSIM. The comparison emphasizes that highD uses a single high-resolution aerial camera per recording, modern detection and tracking, and post-processing that removes false positives and smooths motion, whereas NGSIM is described as suffering from shorter duration, fewer locations, lower data variety, and substantial trajectory errors (Krajewski et al., 2018). This positioning helps explain why highD became a preferred highway corpus for downstream research.

2. Measurement principle and annotation pipeline

highD was recorded with a consumer quadcopter DJI Phantom 4 Pro Plus equipped with a 4K camera. The drone hovered next to the highway to minimize perspective distortion, yielding a bird’s-eye view with a pixel size on the road surface of about $10 \times 10$ cm. The recordings were performed under sunny and windless conditions, between 8 AM and 5 PM, and the videos were stabilized using OpenCV by estimating transformations from each frame’s background to the background in the first frame; the first frame was also rotated so that lane markings became horizontal (Krajewski et al., 2018).

The data-extraction pipeline combines automated and manual annotation. Static infrastructure elements such as lane markings, traffic signs, and speed limits were annotated manually. Dynamic object extraction was based on U-Net semantic segmentation, followed by conversion of vehicle pixel clusters into bounding boxes, distance-based matching across frames, and Rauch–Tung–Striebel smoothing with a constant acceleration model to refine position, speed, and acceleration in both longitudinal and lateral directions (Krajewski et al., 2018).

The reported detection quality is a central technical feature of highD. The dataset paper reports approximately $99\%$ vehicle detection rate, about $2\%$ false positive detections, and mean midpoint position error below $3$ cm longitudinally and laterally relative to manual labels (Krajewski et al., 2018). Later papers often summarize the practical consequence more coarsely as very high or sub-decimeter trajectory accuracy, which is why highD is repeatedly selected for tasks that depend on precise spacing, relative speed, and time-gap estimation (Liu et al., 1 Jul 2025, Shi et al., 30 Dec 2025).

Because the view is aerial, the dataset avoids many occlusion issues common in roadside sensing. This directly matters for scenario reconstruction: vehicle-following states, cut-ins, lane changes, and interaction neighborhoods can be measured from the same source without the camera-boundary artifacts that complicated earlier freeway datasets (Krajewski et al., 2018).

3. Data model, maneuver semantics, and scenario derivation

For each recording, highD provides four files: an aerial image of the highway area, a CSV file with site and infrastructure information, a CSV file summarizing each vehicle track, and a CSV file with detailed per-frame trajectory data. The site file includes the location of the recording site, driving lanes, traffic signs, and speed limits. The track summary file includes vehicle dimensions, vehicle class, driving direction, and mean speed. The detailed trajectory file includes speeds, accelerations, lane positions, and surrounding vehicle descriptions for every frame (Krajewski et al., 2018).

A distinctive property of highD is that it is not limited to raw trajectories. The original dataset defines four maneuver categories: free driving, vehicle following, critical maneuver, and lane change. It also derives surrounding-traffic parameters and safety metrics, including DHW, THW, and TTC, for vehicles on the ego lane and adjacent lanes, and provides scripts for threshold-based extraction of these maneuvers (Krajewski et al., 2018). This design made highD immediately usable for scenario mining rather than only for supervised learning.

The lane-change analysis in the dataset paper illustrates this scenario-oriented logic. Lane changes are parameterized with a quadratic polynomial for longitudinal motion and a 5th-degree polynomial for lateral motion, while surrounding vehicles are summarized through minima of $\mathrm{DHW}_{\min}$, $\mathrm{THW}_{\min}$, and $\mathrm{TTC}_{\min}$ for the preceding vehicle on the initial lane, the preceding vehicle on the target lane, and the tailing vehicle on the target lane. From the 5,600 parameterized lane changes, 850 cut-in scenarios from the right-hand side were extracted for distributional analysis (Krajewski et al., 2018).

Later work operationalized highD further into derivative scenario sets. ConScenD transforms highD trajectories into concrete ALKS test scenarios under UNECE R157 by extracting event-centric cases such as lead-vehicle braking and cut-ins, filtering them, and exporting them to OpenSCENARIO and OpenDRIVE for CARLA and esmini. That work reports more than 340 extracted scenarios overall, with 136 cut-in scenarios and 38 brake scenarios emphasized after stricter filtering (Tenbrock et al., 2021).

highD has also been used to extract similar traffic scenes rather than only maneuver templates. One method represents the surrounding vehicles of a scene as a context set of 4D points

$p_i = [x_i,\lambda y_i,v_{x,i},\lambda v_{y,i}],$

with $\lambda = 10.0$, and compares scenes using the symmetric Hausdorff distance. In a case study on highD, 250 similar scenes were extracted from a hand-picked example; among these, 26 drivers changed lanes within 3 seconds, 62 slowed down by more than $1\%$ of their initial speed, and 162 neither slowed down nor changed lanes within 3 seconds, exposing both tactical and operational variability (Siebinga et al., 2022).

4. Traffic statistics, criticality measures, and interpretive cautions

highD supports both macroscopic and microscopic analysis from the same trajectory source. A comprehensive statistical study defines the standard traffic variables

$99\%$0

and also uses an area-density-style estimate

$99\%$1

Using these variables, the study reports that the density–flow relation is approximately linear in free flow up to roughly 10–15 vehicles/km/lane, with correlation $99\%$2 in that range; average velocity drops substantially around 40 vehicles/km/lane; stop-and-go appears around 30–40 vehicles/km/lane; and jammed traffic was not really observed in the one-minute aggregation windows (Kruber et al., 2019).

The same analysis reports flow-dependent lane usage and lane-change activity. On 3-lane motorways, the right lane dominates at low flow, while the middle and left lanes gain share as flow rises. The number of lane changes per lane, hour, and kilometer follows a roughly triangular dependence on both flow rate and density, increasing to a peak and then declining as traffic becomes more constrained (Kruber et al., 2019).

At the microscopic level, the velocity histogram has two prominent peaks around 90 km/h and 120 km/h, and the maximum recorded mean velocity is 246 km/h. The overall longitudinal and lateral acceleration samples are summarized by logistic fits,

$99\%$3

and strong active braking is reported as rare: about $99\%$4 of vehicles show braking stronger than $99\%$5, and $99\%$6 if the threshold is relaxed to $99\%$7 (Kruber et al., 2019).

For criticality analysis, the paper studies TTC, THW, and the combined risk-perception measure

$99\%$8

$99\%$9

with benchmark values $2\%$0 and $2\%$1 from Kondoh (2008). A striking empirical result is that many low TTC or THW values are not imminent collisions, but ordinary highway maneuvers such as lane changes, cut-ins, congestion-induced oscillations, or overtaking sequences. About $2\%$2 of vehicles go below THW $2\%$3 s at least once, and about $2\%$4 go below $2\%$5 s at least once, yet these values frequently occur in non-emergency contexts. Likewise, the benchmark scenario set $2\%$6 contains 3323 scenarios, about $2\%$7 of tracks meeting the four-second tailgating requirement, but only about $2\%$8 show negative longitudinal acceleration and only $2\%$9 show active braking stronger than $3$0 (Kruber et al., 2019).

This is one of the main interpretive lessons of highD-based safety analysis: low TTC or THW thresholds alone produce many false positives. Multiple later studies retain this caution by combining spacing metrics with maneuver context, traffic state, interaction structure, or learned reward models rather than treating a single threshold as a sufficient definition of risk (Kruber et al., 2019, Liu et al., 1 Jul 2025).

5. Role as a benchmark for learning, prediction, and control

highD has become a standard highway benchmark for trajectory prediction. A graph-spectral approach, GFTNNv2, evaluates on highD after balancing the maneuver distribution into 9000 highway scenarios with equal numbers of lane keeping, lane change to the right, and lane change to the left. In that setup, the model observes 3 seconds of motion from $3$1 vehicles, predicts 5 seconds of future motion for the central target vehicle, and reports its best highD performance at GFTNNv2-80 with ADE $3$2 m and FDE $3$3 m, corresponding to a reported $3$4 improvement in ADE and $3$5 improvement in FDE over the best competing baseline (Neumeier et al., 2023).

Other prediction paradigms use highD to probe different inductive biases. POVL studies variable-length observation and reports that prediction can begin from as little as 2 time steps $3$6 s of history; on highD, it reports RMSE $3$7 m at 5 s, compared with 1.18 m for MHA-LSTM and 1.76 m for constant velocity (Mozaffari et al., 2023). A conditioned diffusion model, cVMD, uses highD with $3$8, $3$9, $\mathrm{DHW}_{\min}$0 s, and $\mathrm{DHW}_{\min}$1 s, and reports ADE $\mathrm{DHW}_{\min}$2 m and FDE $\mathrm{DHW}_{\min}$3 m with uncertainty-adaptive guidance while emphasizing guaranteed drivable trajectories and latent-space uncertainty quantification (Neumeier et al., 2024). A physics-aware xLSTM model, X-TRACK, evaluates on highD as its primary highway benchmark and reports ADE $\mathrm{DHW}_{\min}$4 m and FDE $\mathrm{DHW}_{\min}$5 m, with RMSE values $\mathrm{DHW}_{\min}$6 and $\mathrm{DHW}_{\min}$7 m across 1–5 s (Chugh et al., 31 Oct 2025). A risk-aware predictor, RHP, treats highD as the highway-corridor dataset and reports 5 s RMSE $\mathrm{DHW}_{\min}$8, a $\mathrm{DHW}_{\min}$9 reduction relative to a best baseline with 5 s RMSE $\mathrm{THW}_{\min}$0 (Ning et al., 30 May 2026).

Lane-change intention prediction studies use highD differently: not as a continuous regression target, but as a highly imbalanced three-class benchmark. A Temporal Physics-Informed AI framework uses a strict location-based split with locations 0, 1, 2, and 3 for training and locations 4 and 5 for testing, frames the task as No-LC, Left-LC, and Right-LC, and reports highD macro-F1 values of $\mathrm{THW}_{\min}$1, $\mathrm{THW}_{\min}$2, and $\mathrm{THW}_{\min}$3 at horizons $\mathrm{THW}_{\min}$4 s, respectively (Shi et al., 30 Dec 2025). A related physics-informed three-class formulation reports LightGBM results on highD of accuracy $\mathrm{THW}_{\min}$5 and macro-F1 $\mathrm{THW}_{\min}$6 at 1 s, with performance degrading as the prediction horizon increases and the minority lane-change classes becoming the dominant difficulty (Shi et al., 22 Sep 2025).

Behavior modeling studies exploit highD’s interaction precision rather than only its size. One car-following study extracts ego–lead–following configurations under criteria such as LV within 100 m, both vehicles above 10 km/h, and episode length of at least 10 s, then separates tailgated events with time headway $\mathrm{THW}_{\min}$7 s from gapped events with time headway $\mathrm{THW}_{\min}$8 s. From 1,024 tailgated and 465 gapped events, Dynamic Time Warping plus manual screening yields 81 matched pairs, and the analysis finds that mean THW is significantly smaller in tailgated events while mean DRAC is also significantly different. The recovered AIRL reward maps further indicate that tailgating pressure changes spacing choice and control strategy without removing sensitivity to lead-vehicle speed and spacing (Liu et al., 1 Jul 2025).

Interactive decision-making work uses highD as a human-demonstration source. A game-theoretic AV decision model validated on highD and exiD reports a highD-only similarity rate of $\mathrm{THW}_{\min}$9 in merging scenarios, while a highway prediction-and-control framework uses highD recording 01, containing 1047 vehicles over 900 s with 0.04 s sampling time, to demonstrate an interaction-aware IMM-KF plus scenario-based MPC stack (Huang et al., 2024, Zhang et al., 2023). These uses illustrate that highD is not only a supervised-learning benchmark; it is also a substrate for extracting human-like strategies, scenario probabilities, and control test cases.

6. Limitations, biases, and boundary conditions

highD is highway-focused and therefore does not cover urban driving scenes. The original dataset paper also notes that no height information is captured from the aerial view, and that recordings were limited to daytime, sunny, and windless conditions because of legal and operational constraints (Krajewski et al., 2018). These restrictions matter when highD is interpreted as a proxy for an Operational Design Domain rather than as a highway-only corpus.

A second limitation is distributional bias. The traffic-statistics study reports that highD appears to have an unusually high proportion of trucks relative to a nearby Cologne metering point, a bias that affects velocities, headways, and lane usage and should therefore be considered in traffic modeling and simulation calibration (Kruber et al., 2019). Several learning papers identify a different but related bias: raw highD is heavily imbalanced toward lane keeping, so balanced subsets, resampling, weighting, or threshold calibration are routinely introduced before trajectory-prediction or intention-prediction experiments (Neumeier et al., 2023, Shi et al., 30 Dec 2025).

A third limitation concerns regulatory transfer. ConScenD observes that highD mainly contains higher-speed traffic, typically 100–130 km/h, जबकि ALKS is regulated for operation up to 60 km/h; accordingly, that work applies a 70 km/h filter on initial ego speed when constructing ALKS-relevant concrete scenarios (Tenbrock et al., 2021). This does not invalidate highD for scenario mining, but it means that direct regulatory reuse often requires filtering or reinterpretation.

Finally, highD-based criticality analysis shows that metric definitions alone are not enough. Low TTC, low THW, or even RP-triggered cases frequently correspond to lane changes, cut-ins, overtaking, or congestion rather than imminent collisions (Kruber et al., 2019). A plausible implication is that highD is strongest when used with context-aware interpretation: lane structure, surrounding-vehicle configuration, maneuver state, and traffic regime are usually required to turn precise trajectories into reliable safety judgments.