DeepAccident Dataset: V2X Accident Benchmark

• DeepAccident is a simulation-based dataset offering high-fidelity accident scenarios and multi-agent sensor data for V2X research.
• It features 12 distinct accident types across various urban and road settings, with detailed annotations including collision timing and sensor fusion data.
• The dataset supports end-to-end motion prediction, 3D detection, and cooperative V2X perception, with baselines like V2XFormer and advanced fusion strategies.

DeepAccident Dataset

The DeepAccident dataset is a V2X-centric, simulation-based benchmark designed for end-to-end motion and accident prediction, V2X cooperative perception, 3D detection, and explainable evaluation of autonomous driving safety. Generated with the CARLA simulator, DeepAccident emphasizes high-fidelity accident scenarios that replicate real-world complexity, supporting the development, evaluation, and analysis of vehicle-to-everything research and safety-critical motion prediction models in both academic and industrial contexts.

1. Dataset Structure and Generation

DeepAccident comprises 285,000 annotated samples and 57,000 annotated V2X frames, significantly surpassing the scale of previous simulator-derived benchmarks (e.g., nuScenes). The dataset is partitioned into train (203K), validation (41K), and test (41K) splits, covering a broad variety of normal and accident events. Each scenario is recorded simultaneously from four vehicle agents (each equipped with synchronized multi-view cameras and LiDAR) and one road infrastructure unit, resulting in rich multi-view, multi-agent sensor data.

Accident scenarios in DeepAccident are derived from a knowledge-based simulation process (Wang et al., 2023), where two vehicles are controlled to create explicit collision events. This is achieved by manipulating their trajectories—via perturbation of initial positions and speed controls—so that the vehicles reach their intersection point concurrently. Rule-based controllers ensure behavior diversity and allow for timing variability, enabling naturalistic secondary collisions and interaction with surrounding simulated agents.

2. Accident Types and Scene Diversity

Twelve accident types are included, modeled after empirically observed pre-crash configurations sourced from NHTSA reports. These encompass typical urban intersection hazards—red-light running, unprotected left-turn collisions, unsignalized conflicts, and complex maneuvers. Each accident is annotated with precise collision timing, involved agent IDs, spatial coordinates, and the full multi-agent view.

DeepAccident’s scenarios span dense urban settings, residential roads, intersections, and open lanes, with varying weather (day/night, rain, fog, clear) and traffic conditions. This breadth provides a stringent testbed for models tackling both perception and risk assessment under occlusion, changing visibility, and dynamic multi-agent interactions.

3. Task Schemes and Evaluation Metrics

A core differentiator of DeepAccident is its explicit support for end-to-end tasks:

• Motion and Accident Prediction: The input is multi-view video/point-clouds; the output is a set of future ego-centered object trajectories. Post-processing determines accidents by checking if two predicted object polygons approach within a safety threshold (default: 1 m).
• 3D Detection and Tracking: Standard mAP and mIOU evaluation for object localization and segmentation across agents.
• Cooperative V2X Perception: Fusion of vehicle and infrastructure viewpoints allows scene elements occluded in a single perspective to be recovered in the global BEV.

A specialized metric—Accident Prediction Accuracy (APA)—is defined as follows:

$$\text{APA} = \frac{1}{|\mathcal{D}|} \sum_{d \in \mathcal{D}} \frac{|\mathrm{TP}|_d}{|\mathrm{TP}|_d + \frac{1}{2}|\mathrm{FP}|_d + \frac{1}{2}|\mathrm{FN}|_d}$$

where $\mathcal{D}$ is a set of position thresholds (e.g., $\{5, 10, 15\}$ meters), and $|\mathrm{TP}|_d$, $|\mathrm{FP}|_d$, $|\mathrm{FN}|_d$ are counts of true positives, false positives, and false negatives at threshold $d$. Additional true positive errors are reported for each accident: positional, temporal, and agent ID mismatches.

4. Baseline Algorithms and Data Fusion Strategies

DeepAccident establishes a baseline with a V2XFormer model, based on the BEVerse architecture with a SwinTransformer backbone (Wang et al., 2023). In the baseline pipeline:

• Each agent (vehicle/infrastructure) encodes its sensor data into a BEV feature map.
• Features are wrapped to the ego-agent’s coordinate frame for aggregation.
• Advanced V2X fusion modules, including CoBEVT and V2X-ViT, are compared, with CoBEVT demonstrating the highest improvements in APA and mIOU.
• The fused BEV representation enables joint object detection, tracking, and motion prediction, and supports post-hoc accident prediction via trajectory analysis.

Recent work has further extended DeepAccident’s utility:

• The UniE2EV2X framework uses deformable attention-based spatial and temporal fusion, improving prediction reliability by selectively focusing on dynamically relevant regions in BEV space (Li et al., 7 May 2024).
• Intention-based and risk-aware models have reported at least 26.5% improvement in average displacement error (ADE) and final displacement error (FDE) over baselines for accident scenarios (Wei et al., 24 Sep 2024).
• DeepAccident-360, a panoramic reformatting of the dataset, enables BEV semantic mapping from a single omni-directional input, supporting the evaluation of panoramic-to-BEV models such as OneBEV (Wei et al., 20 Sep 2024).

5. Research Applications and Comparative Analysis

DeepAccident is architected for V2X research, multi-agent sensor fusion, and as an explainable benchmark for evaluating and comparing perception and prediction modules. Its multi-agent, multi-view design enables:

• Robustness testing under occlusion and partial observability, as complementary vehicle and infrastructure sensors capture events otherwise hidden from a single viewpoint.
• Detailed quantitative assessment of accident anticipation systems in both perception and decision layers.
• Study of domain transfer and sim2real generalization, as experiments mixing real-world nuScenes and DeepAccident data have demonstrated improved adaptation to challenging safety-critical situations.

Compared to other accident datasets such as DADA-2000 (real-world, driver-attention centric, with fixation/saccade data and dense crash annotation) (Fang et al., 2019), DeepAccident offers synthetic control, reproducibility, and scalability in labeling. In contrast to DADA-seg and V-TIDB, which emphasize pixel-wise segmentation and multi-label context, DeepAccident’s unique value lies in its V2X coverage, high-density temporal and spatial annotation, and cooperative perception framework.

6. Technical and Methodological Significance

DeepAccident’s design allows methodologically rigorous evaluation of a wide spectrum of algorithms:

• Motion Forecasting and Risk Assessment: Detailed agent trajectories and event logs enable ADE/FDE-based comparison for both normal and hazardous traffic.
• V2X Sensor Fusion: Comprehensive annotation supports the benchmarking of multi-agent fusion architectures, including attention-based, transformer, and deformable fusion schemes.
• Out-of-Distribution Evaluation: The diversity and simulation flexibility facilitate the testing of model robustness under rare and previously unseen accident scenarios, as documented in studies on risk-aware trajectory prediction (Wei et al., 24 Sep 2024).

Additionally, the dataset enables the controlled paper of accident emergence, timing, and spatial context, facilitating research into early hazard warning, risk-aware motion planning, cooperative decision-making, and domain-adaptive perception systems. The open use of the CARLA simulation platform, auto-generated ground truth, and structured access to all sensor and state variables further broaden its methodological utility.

7. Future Directions and Impact

DeepAccident is positioned as a cornerstone for V2X autonomous driving safety research, supporting innovations in real-time safety filtering (Yang et al., 29 Oct 2024, Tabbara et al., 2 Dec 2024), accident description/analysis (Li et al., 20 Feb 2025), BEV semantic mapping (Wei et al., 20 Sep 2024), and multi-modal model evaluation (Skender et al., 23 Sep 2025). Its extensive scale, high annotation fidelity, scenario richness, and multi-agent sensor coverage collectively make it a reference platform for benchmarking predictive safety and cooperative autonomy in simulation. As advanced research increasingly depends on large-scale, explainable, and reproducible datasets to drive algorithmic development and policy deployment, DeepAccident addresses critical gaps in current benchmarking for accident anticipation, cooperative perception, and safe planning under partial observability.