CrashSight: Infrastructure Crash Benchmark

• CrashSight is a phase-aware, infrastructure-centric benchmark that decomposes real-world crash videos into pre-crash, collision, aftermath, and cause phases.
• It evaluates vision-language models via structured multiple-choice questions focusing on causal reasoning, temporal progression, and fine-grained crash scene analysis.
• Empirical results reveal that fine-tuning models significantly improves crash understanding performance, though challenges remain in object identification and temporal integration.

CrashSight is a large-scale vision-language benchmark for roadway crash understanding using real-world roadside camera data, introduced to evaluate infrastructure-assisted perception in cooperative autonomous driving from fixed surveillance viewpoints rather than the ego-vehicle perspective (Gan et al., 9 Apr 2026). Its core contribution is a phase-aware formulation of crash-scene understanding in which each clip is decomposed into pre-crash context, collision dynamics, aftermath, and potential causes, and then probed through structured multiple-choice video question answering. In the supplied literature, the same name is also used for several crash-oriented system blueprints, including automatic emergency reporting, crash-risk prediction, incident-report reasoning, and software crash localization; nevertheless, the benchmark published as "CrashSight: A Phase-Aware, Infrastructure-Centric Video Benchmark for Traffic Crash Scene Understanding and Reasoning" is the primary research instantiation of the term (Gan et al., 9 Apr 2026).

1. Benchmark rationale and research positioning

CrashSight was motivated by a gap between the needs of cooperative autonomous driving and the evaluation regimes commonly used for vision-LLMs. Cooperative autonomous driving depends on situational awareness from both vehicle-side and infrastructure-side sensors, whereas existing benchmarks predominantly adopt an ego-vehicle viewpoint. This leaves crash understanding from fixed roadside cameras underrepresented, even though roadside cameras support V2X cooperative perception, post-incident analysis, and traffic management (Gan et al., 9 Apr 2026).

The benchmark is explicitly infrastructure-centric. Its source material consists of real-world crash recordings from roadside cameras in the TAD corpus, typically captured from above or from the side, with oblique surveillance viewpoints common. This is consequential because roadside imagery differs systematically from dashcam video: the camera is stationary, scene scale varies sharply across the frame, actors are frequently small or occluded, and causal interpretation often depends on integrating sparse temporal cues rather than following a dominant ego trajectory. Existing traffic QA and surveillance datasets do not target this problem in the same way. Ego or mixed-view traffic QA resources such as SUTD-TrafficQA, VRU-Accident, and TAU-106K remain vehicle-centric or general; surveillance anomaly datasets such as UCF-Crime, Avenue, and ShanghaiTech seldom focus on traffic crashes or lack structured phase-aware semantics; and infrastructure-side video QA such as TUMTraffic-VideoQA targets general traffic scenes rather than crash understanding (Gan et al., 9 Apr 2026).

A common misconception is to treat CrashSight as simply another traffic VQA benchmark. Its design is narrower and more safety-critical. The benchmark targets crash-specific semantics from fixed roadside viewpoints, provides phase-aware temporal annotations that preserve a crash’s narrative structure, and standardizes VQA tasks around causal attribution, temporal progression, and post-crash outcomes. This suggests that CrashSight is less a generic scene-understanding resource than a controlled probe of whether contemporary VLMs can ground, sequence, and explain rare but operationally important events under surveillance-style visual conditions.

2. Corpus composition and phase-aware annotation

The dataset comprises 250 annotated surveillance clips, each approximately 5–70 seconds long, sourced from real-world roadside crash recordings in the TAD corpus (Gan et al., 9 Apr 2026). Clip selection followed three criteria: a visible collision or near-miss involving at least one road user, sufficient pre-crash context to enable causal reasoning, and observable post-crash aftermath. The split is fixed at 60/20/20 train/validation/test with clip-level disjointness, and all reported results are on the held-out test split.

For model evaluation, a uniform preprocessing scheme samples 4 frames per clip at 1 FPS, with frames resized within pixel bounds of $128\times 28^2$ to $256\times 28^2$, or approximately 100K–200K pixels per frame. This low-frame, bounded-resolution regime is not incidental; it mirrors the computational constraints under which many general-purpose VLMs are currently evaluated, while also exposing the brittleness of such models on long, sparse, safety-critical video.

The benchmark’s distinctive design choice is phase decomposition. Each clip is partitioned into four parts that preserve the crash narrative: Phase 1, Traffic Scenario, covering the pre-crash road environment and movements; Phase 2, Crash Content, covering collision dynamics, vehicle trajectories, and impact configuration; Phase 3, Aftermath, covering the post-crash scene state; and Potential Causes, covering expert causal analysis based on visible evidence (Gan et al., 9 Apr 2026). Phase references are not merely descriptive metadata. They are carried into VQA generation through an evidence field grounded in phase descriptions and a phase_reference field indicating the supportive phase or phases.

Annotation proceeds in three stages. First, InternVL3-80B generates phase-delimited draft captions using structured templates, entity identifiers such as “V1 white sedan” or “P1 motorcyclist,” and directional references. Second, three trained annotators refine entity precision, spatial relations, phase boundaries, and causal specificity; approximately 90% of drafts require substantial correction. Third, a quality-verification stage enforces internal consistency between phase boundaries and events, standardizes terminology, and resolves ambiguities (Gan et al., 9 Apr 2026). The heavy correction burden is itself informative: even a strong VLM was insufficient as a direct annotator for this domain, which underscores the gap between broad visual description capability and crash-specific factual precision.

CrashSight is accessible through its project page, which provides the full dataset and code. The paper does not explicitly state a license in text and directs readers to the project page for license, terms of use, and any usage restrictions. Because the videos are sourced from the TAD surveillance corpus, the paper also notes practical challenges in releasing surveillance-based crash data and emphasizes expert verification (Gan et al., 9 Apr 2026).

3. Taxonomy, task design, and evaluation protocol

CrashSight contains 13,016 multiple-choice question-answer pairs, each with one correct answer and three counterfactual distractors; answer positions A/B/C/D are uniformly shuffled, with a ground-truth distribution across options of approximately 23.3–27.8% (Gan et al., 9 Apr 2026). The taxonomy is two-tiered and spans seven categories.

Tier 1 covers Crash Understanding, meaning phase-local visual grounding. Scene Identification addresses road type, lighting, weather, and traffic context and is answerable from Phase 1, with 3–4 questions per video. Involved Parties addresses vehicle types, VRU types, distinguishing features, and counts, drawing on Phases 1 and 2, again with 3–4 questions. Post-Crash Outcome addresses VRU status, vehicle condition, and emergency response indicators from Phase 3, also with 3–4 questions.

Tier 2 covers Crash Reasoning, meaning cross-phase temporal and causal integration. Crash Mechanics addresses collision type, impact dynamics, and pre-collision maneuvers and requires Phase 2 detail, with 3–4 questions. Fault Determination addresses primary cause and traffic violations by synthesizing Phases 1–2 and expert causal analysis, with 2 questions. Temporal Sequence addresses event ordering, pre- versus post-comparisons, and phase identification, with 2–3 questions; it is explicitly unanswerable from any single frame (Gan et al., 9 Apr 2026).

The final category is Robustness, which probes hallucination resistance through four probe types: cannot-determine, absent-entity, temporal-trap, and false-premise. Each video receives 5 such questions, and correct performance requires recognition of the limits of visual evidence rather than aggressive answer completion. This category is methodologically important because it tests not only whether a model can answer crash questions, but whether it can refuse unsupported inferences such as phone usage or BAC when such details are visually unobservable.

Question generation is also staged. InternVL3-80B, run through vLLM, produces phase-grounded QA using structured prompts that require evidence and phase_reference. Human verification then checks evidence correctness, distractor plausibility, and phase consistency, and removes questions demanding unobservable details such as precise speed. Robustness augmentation adds the hallucination-probing questions, and paraphrase augmentation increases linguistic diversity while preserving answers and labels (Gan et al., 9 Apr 2026).

The task itself is multiple-choice video question answering over the sampled 4 frames at 1 FPS. Models receive frames and question text in an instruction-following prompt and output a single answer letter. Decoding is deterministic, and scoring uses exact letter match. The main metric is overall accuracy,

$$Acc=\frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[\hat{y}_i=y_i],$$

with category-specific accuracies reported alongside the overall average (Gan et al., 9 Apr 2026). The paper does not define a temporal-consistency metric, and phase-wise accuracy is likewise not defined.

Eight VLM configurations are benchmarked. The zero-shot open-source models are LLaVA-OneVision-0.5B, LLaVA-NeXT-Video-7B, Qwen2.5-VL-3B, Qwen2.5-VL-7B, InternVL3-2B, and InternVL3-8B. Domain-adapted variants fine-tune Qwen2.5-VL-3B and Qwen2.5-VL-7B on the CrashSight train split. The fine-tuning regime uses QLoRA with 4-bit NF4 quantization and bfloat16 compute; LoRA uses $r=16$, $\alpha=32$, and dropout 0.05, applied to Q/K/V projections, MLP layers, and the vision-language projector. Optimization uses AdamW-8bit with learning rate $2\times 10^{-4}$, a cosine schedule with warmup ratio 0.03, per-device batch size 1, gradient accumulation 8, 2 epochs, maximum sequence length 8192, a single NVIDIA A100 80GB, and approximately 4 hours per variant (Gan et al., 9 Apr 2026).

4. Empirical performance and failure structure

CrashSight’s main empirical result is that state-of-the-art VLMs remain substantially below human performance in infrastructure-side crash understanding. Overall average accuracies are 41.5% for LLaVA-OneVision-0.5B, 58.6% for LLaVA-NeXT-Video-7B, 58.6% for Qwen2.5-VL-3B, 62.9% for Qwen2.5-VL-7B, 64.2% for InternVL3-2B, 68.7% for InternVL3-8B, 74.7% for Qwen2.5-VL_3B_FT, and 76.4% for Qwen2.5-VL_7B_FT, while human experts reach 94.7% (Gan et al., 9 Apr 2026).

Several category-level observations are especially salient. InternVL3-8B achieves 85.7% on Temporal Sequence, the highest TS score among all models. Fine-tuned Qwen2.5-VL_7B_FT reaches 97.8% on Robustness, 83.9% on Fault Determination, and 84.3% on Post-Crash Outcome. Involved Parties remains the hardest category, where the best model reaches only 63.2% compared with human performance of 94.7% (Gan et al., 9 Apr 2026). This indicates that failure is not confined to abstract reasoning; fine-grained perception under oblique surveillance geometry is at least as limiting.

Fine-tuning yields large gains. Average improvement over the vanilla Qwen counterparts is +16.1 points for the 3B model and +13.5 points for the 7B model. The largest gains appear in Robustness at +26.1 and +24.5, and in Crash Mechanics at +17.3 and +13.3. A fine-tuned small model, Qwen2.5-VL_3B_FT at 74.7%, exceeds the much larger zero-shot InternVL3-8B at 68.7% (Gan et al., 9 Apr 2026). This suggests that domain adaptation on safety-critical crash semantics currently outweighs brute parameter scaling.

Architecture also matters. InternVL3 shows stronger zero-shot reasoning, especially on temporal ordering, even at smaller sizes; InternVL3-2B exceeds Qwen2.5-VL-7B overall. Human category accuracies are tightly clustered—95.1 for SI, 94.7 for IP, 93.8 for CM, 94.5 for FD, 95.1 for PCO, 94.8 for TS, and 99.2 for Robustness—indicating that the benchmark is not intrinsically ambiguous at the level of expert interpretation (Gan et al., 9 Apr 2026).

The failure analysis sharpens this picture. Comparing vanilla and fine-tuned Qwen2.5-VL-7B, total errors fall from 728 to 464, a 36.3% reduction. Three failure modes are eliminated or nearly eliminated: treating video as a static image, which falls from 4.1% to 0.0%; refusals on answerable questions; and robustness hallucinations, which fall from 6.5% to 0.9% (Gan et al., 9 Apr 2026). Yet persistent errors remain concentrated in visually demanding categories. In the fine-tuned model, Involved Parties accounts for 45.3% of its errors, Scene Identification for 21.8%, and Crash Mechanics for 20.9%.

On 1,962 test samples in the transition analysis, 1,117 are correct for both models, 381 are corrected by fine-tuning, 117 are regressions, and 347 remain persistently incorrect (Gan et al., 9 Apr 2026). The major residual failure modes are confusion over primary violation when visual evidence is ambiguous, missed short pre-crash interactions due to sparse frame sampling, misidentification of vehicle subtypes or VRUs under oblique angles and occlusions, and misunderstanding of collision type or impact dynamics because of limited spatial resolution. The paper attributes these errors to insufficient visual token budget, possible degradation from 4-bit NF4 QLoRA in subtle surveillance discrimination, and the fact that the visual encoder remains frozen during LoRA adaptation.

5. Extended usages of the name

The supplied literature uses the name CrashSight more broadly than the benchmark alone. In traffic-safety systems, one line of usage denotes a real-time crash detection and automatic emergency reporting architecture inspired by "Instant Accident Reporting and Crowdsensed Road Condition Analytics for Smart Cities" (Yousefpour et al., 2017). In that formulation, CrashSight combines a vehicle-side Accident Detection Module with a cloud-based Accident Reporting and Analysis Module. The edge device uses an Arduino 101 for motion sensing and an Arduino Uno with a SIM808 module for GPS and GSM communication, declares a crash when the g-force on any axis exceeds 12 G, stores data to an SD card, and triggers cloud-side emergency notification, reverse geocoding, Twilio-based calling, and pothole analytics (Yousefpour et al., 2017).

A second usage treats CrashSight as a crash-risk prediction system built around roadway complexity. Adapting the two-stage framework of "The Context of Crash Occurrence: A Complexity-Infused Approach Integrating Semantic, Contextual, and Kinematic Features," CrashSight can fuse semantic, driving, and contextual features through a 32-neuron continuous-output encoder whose latent representation improves downstream Random Forest crash-likelihood prediction from 87.98% to 90.15% when the complexity-infused feature vector $z$ is added (Wang et al., 2024). In this line of work, CrashSight is predictive rather than diagnostic: it scores Low, Medium, or High crash likelihood over spatial segments before a crash occurs.

A third usage centers on post hoc reasoning over incident reports. Following "CRASH: Cognitive Reasoning Agent for Safety Hazards in Autonomous Driving," CrashSight is cast as a constrained LLM pipeline over NHTSA Standing General Order crash reports, using a unified four-column representation and a compact output schema with fields such as AV_Failed, Cause, System, and Late (Silva et al., 16 Mar 2026). On 2,168 curated incidents representing more than 80 million miles driven, the underlying study reports that system-related failures dominate and that expert validation yields 86% accuracy on AV responsibility and 84% on late AI detection (Silva et al., 16 Mar 2026).

A fourth group of usages concerns anticipation and spatial risk mapping. These include scenario-wise spatio-temporal attention guidance built from 162,104 FARS fatal crashes (Li et al., 2021), accident anticipation with object-aware, context-aware, and temporal-focus modules (Liao et al., 2024), multimodal six-hour regional risk prediction with CrashFormer (Monsefi et al., 2024), and a real-time ensemble system for hierarchical severity classification that reports mAP 0.893, 92.4% accuracy, and 89.7% hotspot-identification precision (Sivakoti, 9 Feb 2025). Another surveillance-oriented formulation implements a five-module CCTV pipeline with YOLOv3, MOSSE tracking, collision estimation, ViF plus SVM, and GPS/GSM alerting, reporting 91% accuracy and 94% recall for collision estimation alone, and 93% accuracy with 78% recall when ViF verification is added (Essam et al., 2022).

The term also appears in software-crash diagnostics. In one interpretation, CrashSight maps onto industrial crash root-cause localization using only crashdumps and source repositories, as in AutoCrashFL, which reports top-1 localization accuracy of 30% on 454 SAP HANA crashes, compared with 17% and 11% for stack-based baselines (Kang et al., 26 Oct 2025). Related software-crash localization frameworks include stack-expansion with Ochiai and heuristics on Firefox crash reports, locating 63.9% of crashing faults by examining only 5% of functions (Gong et al., 2014), multi-task sequence labeling over Windows Error Reporting stacks reaching about 0.90 average top-1 accuracy across four Microsoft applications (Shetty et al., 2021), and continuous contrast set mining for crash triage at Facebook scale, which achieves a 40x speedup over discretization-based baselines on iOS OOM data (Qian et al., 2019).

6. Significance and future directions

CrashSight’s principal significance lies in standardizing evaluation for infrastructure-assisted crash understanding. By combining fixed roadside video, phase-aware annotation, evidence-grounded MCQ design, and robustness probes, it turns crash-scene interpretation into a reproducible benchmark rather than an anecdotal capability claim (Gan et al., 9 Apr 2026). It also exposes a specific failure boundary in contemporary VLMs: scene description is comparatively strong, but temporal integration, causal attribution, fine-grained party identification, and calibrated abstention remain fragile in safety-critical settings.

The benchmark’s future directions are correspondingly concrete. The authors recommend improving visual token utilization through adaptive or event-driven frame sampling around pre-crash and impact moments; increasing decisive-scene resolution; integrating explicit object tracking and 3D- or motion-aware representations for oblique surveillance viewpoints; supervising not only answers but also evidence grounding and reasoning consistency; and moving beyond multiple-choice evaluation toward multi-turn QA, dense phase-grounded captioning, and structured explanations for causal attribution and outcomes (Gan et al., 9 Apr 2026). They also identify benchmark-level extensions: scaling the number and diversity of videos, expanding scene sources, improving annotation efficiency through stronger VLMs or agentic workflows while preserving human-in-the-loop verification, and addressing perception bottlenecks by unfreezing visual encoders, increasing frame count or resolution, and reducing quantization constraints during fine-tuning.

A broader implication is that CrashSight sits at the intersection of benchmark design, infrastructure-side perception, and safety analysis. In the narrower sense of the 2026 benchmark, it measures whether models can interpret crashes from surveillance footage. In the broader sense reflected by adjacent technical narratives, the same name has come to denote a family of crash-centered systems spanning sensing, reporting, reasoning, prediction, and localization. That semantic spread is itself revealing: crash intelligence is no longer treated as a single-task problem, but as a stack of coupled tasks ranging from early risk estimation and event detection to causal explanation and post-event diagnosis.