L4 Motion Forecasting Dataset
- The L4 Motion Forecasting Dataset is a proprietary benchmark that uses Bosch test-vehicle recordings to analyze multi-agent trajectory prediction.
- It maintains Argoverse 2 compatibility while enriching map semantics and agent features with data from Stuttgart and Sunnyvale.
- Empirical results show that while richer features yield limited improvements, scenario diversity and transfer training enhance model generalization.
The L4 Motion Forecasting dataset is a proprietary motion-forecasting benchmark built from Bosch test-vehicle recordings to study multi-agent trajectory prediction under realistic conditions. It was designed to mirror the Argoverse 2 Motion Forecasting format so that existing AV2 models, notably QCNet, can be used without changes, while adding richer agent and map features and geographic diversity through data from Germany and the United States (Demmler et al., 7 Jul 2025).
1. Identity and design objective
The dataset is explicitly aimed at Level-4 autonomy development. Its stated design goals are threefold: to remain AV2-compatible, to test whether richer agent and map features improve prediction, and to enable analysis of cross-country generalization and “driving culture” effects. The name “L4” signals that it comes from full-stack test vehicles equipped with production-grade sensors and HD maps, rather than academic or toy setups. The underlying recordings cover Stuttgart, Germany, and Sunnyvale, United States, and include urban, rural, and highway environments, including German Autobahn segments with speeds up to approximately (Demmler et al., 7 Jul 2025).
This positioning gives the dataset a dual role. On one hand, it is a controlled comparison point against Argoverse 2, because it preserves the same forecasting format and evaluation protocol. On the other hand, it deliberately expands the operating regime beyond city-centric public benchmarks by including high-speed rural and highway traffic, roundabouts, and stronger geographic heterogeneity. A plausible implication is that the dataset was intended less as a format innovation than as an intervention on scenario distribution, map semantics, and transfer conditions.
The dataset is treated as a proprietary research dataset. No explicit statement about public release, access URLs, or licensing terms is given, and external researchers cannot directly download it according to the information reported in the paper. The practical value of the dataset therefore lies not only in the benchmark itself, but also in the empirical evidence it provides about feature sufficiency, domain transfer, and geographic diversity.
2. Corpus, segmentation, and split construction
The reported corpus contains approximately scenarios of $11$ seconds each, corresponding to $274$ hours of scenario data derived from approximately $400$ hours of raw driving, over approximately of HD-mapped roadway at (Demmler et al., 7 Jul 2025).
| Characteristic | Reported value |
|---|---|
| Scenarios | |
| Scenario duration | |
| Sampling rate | |
| Time steps per scenario | 0 |
| Forecast horizon | 1 |
| Total scenario time | 2 |
| Raw driving time | 3 |
| HD-mapped road network | 4 |
| Average tracks per scenario | 5 |
| Average track length | 6 |
Scenario generation follows AV2-style 7-second windows. Continuous driving logs are cut into scenarios based on temporal discontinuities greater than 8, and only scenarios where the focal track is visible for the full 9 are retained. The train, validation, and test partition is $11$0. Continuous recording files are not split across sets, which is intended to ensure temporal independence between splits. The splits are balanced with respect to geography, environment, and object type diversity.
The geographic composition is asymmetric in structure rather than in format. Stuttgart contains a mix of urban streets, rural roads, and German highways, and is described as more balanced between urban and rural driving. Sunnyvale is described as predominantly highway and higher-speed suburban corridors. This difference is central to the later cross-country experiments: the two subsets are not merely geographic replicas, but expose distinct scenario priors.
3. Agent, map, and feature representation
Raw recordings classify objects into $11$1 fine-grained hierarchical classes, which are collapsed into $11$2 AV2-compatible categories for forecasting. The paper lists vehicle, pedestrian, cyclist, and “other/remaining types,” and notes that bus and truck are combined as “large vehicle” in the raw hierarchy, which leads to more “bus” labels in L4 relative to AV2. Agents are represented at $11$3 by center coordinates $11$4, heading $11$5, a 2D bounding box aligned to heading, and derived velocities, accelerations, and yaw rates (Demmler et al., 7 Jul 2025).
Within each scenario, tracks are organized into focal, scored, unscored, and track fragments. The focal track is the primary agent used as the main target in single-agent benchmarks; scored tracks are additional dynamic agents whose futures are predicted in multi-agent evaluation; unscored tracks include typically static objects and the ego vehicle itself; track fragments are only partially observed within the $11$6-second window. This organization preserves direct interoperability with AV2-style evaluation.
Map content is encoded in the same vectorized structure as AV2. For each scenario, map elements within a $11$7 radius around the ego trajectory are included. Standard AV2-like elements are lane segments, lane boundaries, and pedestrian crossings. L4 adds explicit shoulder lanes, parking lanes, maximum speed limit per lane segment, real stop lines for stop signs and signalized intersections, virtual stop or yield lines such as “yield to oncoming traffic when turning left” or roundabout entries, and stop/yield annotations on lanes. No traffic signal state is provided in the AV2-compatible feature set; the paper states “Traffic Signal States: ✗” for both L4 and AV2, in contrast to Waymo.
A central technical feature of the dataset is the inclusion of richer per-agent attributes than public AV2-style baselines. These include Frenet-frame variables $11$8 and $11$9, longitudinal and lateral velocities and accelerations $274$0, associated variances, distances to crosswalks, stop lines, traffic lights and other regulatory elements, explicit width and length, yaw rate $274$1, and variances in position and velocity estimates. Agents therefore admit feature tensors of the form
$274$2
with larger $274$3 than in AV2. The dataset was constructed precisely to test whether such enriched state and map channels materially improve forecasting accuracy.
4. Structural relation to Argoverse 2 and evaluation protocol
The dataset is structurally isomorphic to Argoverse 2: the same scenario length of $274$4, the same sampling rate of $274$5, the same forecast horizon of $274$6, the same focal and scored track definitions, the same $274$7 map radius, the same five object categories, and the same evaluation metrics. The paper states that the same code and evaluation scripts for AV2 can therefore be used on L4 (Demmler et al., 7 Jul 2025).
Evaluation uses the standard AV2 single-agent metrics computed in a multi-agent way, including $274$8, $274$9, $400$0, $400$1, $400$2, $400$3, and $400$4. Metric aggregation over scenes $400$5 and agents $400$6 is reported as
$400$7
This is important because the paper evaluates predictions for all focal and scored agents per scene rather than restricting analysis to one target track.
The relation to AV2 is not only formal but comparative. AV2 has $400$8 scenarios and $400$9 hours, whereas L4 has 0 scenarios and 1 hours, but L4 is reported to be more challenging because it contains a wider speed profile, including high-speed Autobahn segments where small angular errors produce large endpoint errors, and traffic structures such as roundabouts that are rare in AV2. The paper also notes that L4 contains some particularly dense scenes with up to 2 scored agents, not present in AV2.
This structural compatibility also makes the dataset suitable for direct cross-dataset experiments. In the reported QCNet setup, an AV2-only model performs best on AV2 and degrades substantially on L4, while an L4-only model remains comparatively more robust when tested on AV2. The two-stage AV23L4 fine-tuned model nearly matches L4-only performance on L4 and substantially improves over L4-only on AV2, indicating that pretraining on a large public benchmark and fine-tuning on a more complex target domain is effective.
5. Empirical findings: feature sufficiency, transfer, and geography
A common assumption is that richer engineered map and agent features necessarily improve motion forecasting. The paper’s central finding is that, for QCNet on this dataset, performance is essentially identical across feature configurations, including Baseline, Enhanced Lanes, Enhanced Stoplines, Enhanced Agent, Full Integration, and Focused Integration (Demmler et al., 7 Jul 2025).
The reported numbers are narrowly clustered. Baseline yields 4, 5, 6, and 7. Full Integration remains 8, 9, 0, and 1. Enhanced Agent and Focused Integration slightly shift 2 to 3, but the paper states that differences are within noise 4–5, with no consistent improvement. The stated interpretation is that modern transformer-based prediction models such as QCNet can infer most useful implicit information from simple features such as positions, headings, and basic map geometry, and that public datasets with limited feature sets are already sufficient for strong performance.
Cross-dataset transfer experiments reinforce a second claim: scenario diversity may matter more than raw feature richness. When testing on L4, the AV2-only model has 6, compared with 7 for the L4-only model. When testing on AV2, the L4-only model obtains 8, while the AV2-only model reaches 9. The AV20L4 fine-tuned model achieves 1 on L4 and 2 on AV2. The interpretation offered in the paper is that training on the more diverse and difficult dataset yields models with stronger generalization, while pretraining on a large public corpus remains beneficial.
The geography split provides the most explicit evidence about “driving culture” effects. Three models are compared: L4-full, STG-only, and SVL-only. L4-full yields the best performance on all three test sets: overall L4, Stuttgart, and Sunnyvale. STG-only generalizes relatively well to Sunnyvale, while SVL-only degrades severely on Stuttgart and on the overall L4 test set. The paper attributes this to the larger and more diverse Stuttgart subset and concludes that adding even a relatively small amount of data from another country improves performance even on the dominant region. A plausible implication is that geographic diversity acts as a regularizer on behavioral priors, not only as a coverage extension.
6. Position within the broader L4 forecasting literature
The L4 Motion Forecasting dataset belongs to the scenario-based branch of autonomous-driving forecasting, where models consume processed agent histories and HD maps rather than raw sensor streams. Recent surveys identify Argoverse 1, Argoverse 2, Waymo Open Motion, INTERACTION, and nuScenes as the standard public datasets for this regime, and place Argoverse 2 and Waymo Open Motion at the center of current L4-style evaluation practice (Shi et al., 10 Feb 2025). Within that ecosystem, L4 is unusual not because it changes the benchmark API, but because it keeps the AV2 interface while altering the scenario distribution through higher speeds, additional map semantics, and cross-country coverage.
Its design is also consistent with trends in cross-dataset learning. JointMotion reports that self-supervised pretraining on Waymo Open Motion improves downstream performance and enables transfer learning between WOMD and Argoverse 2 Motion Forecasting, emphasizing that large-scale motion–map corpora can support reusable priors across benchmarks (Wagner et al., 2024). The L4 results are aligned with that observation: AV2 pretraining plus L4 fine-tuning outperforms either isolated domain in cross-dataset tests. This suggests that format compatibility is strategically valuable because it lowers the barrier to transfer experiments.
At the same time, the dataset inherits the limitations of clip-based scenario benchmarks. RealMotion argues that conventional motion forecasting benchmarks process each scene independently and ignore the situational and contextual relationships between successive driving scenes, then introduces a data reorganization strategy that turns Argoverse scenes into sequences of overlapping sub-scenes for continuous driving (Song et al., 2024). This suggests that an eventual public successor to L4 could benefit from releasing longer temporal logs or sequence-structured evaluation, especially because the current dataset is already derived from continuous recording files.
The dataset also occupies a different point in the design space from perception-based and sensor-centric benchmarks. Valeo4Cast studies the Argoverse 2 Sensor end-to-end forecasting challenge, where systems must jointly detect, track, and forecast from sensor data rather than from curated tracks (Xu et al., 2024). 4DLidarOpen, by contrast, is an open multi-modal dataset centered on 4D FMCW LiDAR with point-wise radial velocity and benchmarks for motion forecasting with planning (Qian et al., 18 May 2026). Relative to such datasets, L4 is not a raw-sensor benchmark; it is a map-centric, AV2-compatible forecasting corpus intended to isolate questions of feature design, transfer, and distribution shift.
Historically, it also follows a line of industrial-scale map-and-trajectory datasets created for downstream forecasting and planning rather than raw perception. The Lyft Level 5 dataset provided 3 hours of perception-output scenes, a high-definition semantic map with 4 labelled elements, and a high-definition aerial view for motion forecasting, planning, and simulation (Houston et al., 2020). L4 narrows that tradition toward AV2 interoperability and contemporary cross-domain analysis. Its most consequential result is therefore methodological rather than merely curatorial: under a strong modern architecture, richer hand-engineered features do not measurably improve forecasting, whereas scenario diversity, geographic breadth, and transfer setup do.