Real-World Crash Grounding (RCG)

• Real-world Crash Grounding (RCG) is a framework that anchors safety-critical models and evaluations to observable crash evidence, ensuring traceability and credibility.
• It integrates phase-aware video/text analysis, probabilistic scoring, and adversary selection to mirror real-world dynamics in safety systems.
• RCG enhances robustness in applications such as autonomous driving, aircraft collision avoidance, and active crash intervention by grounding decisions in real crash data.

Searching arXiv for the cited papers to ground the article in current sources. Real-world Crash Grounding (RCG) denotes a research orientation in which safety-critical inference, explanation, generation, or control is anchored to real crash evidence rather than to generic priors or purely synthetic objectives. In current literature, the term is used explicitly for crash-grounded scenario generation in autonomous driving (Stoler et al., 14 Jul 2025), while closely related works instantiate the same principle by grounding claims in roadside crash video (Gan et al., 9 Apr 2026), national crash statistics and naturalistic trajectories (Roy et al., 16 Mar 2026), investigator-authored crash narratives (Wang et al., 10 Oct 2025), or replayed real crash cases from the GIDAS Pre-Crash Matrix (Zimmermann et al., 12 Jun 2025). Across these formulations, grounding refers to traceability from outputs to observable crash evidence, real crash distributions, or temporally coherent pre-crash dynamics.

1. Conceptual foundations

In the most explicit formulation, RCG is a safety-critical scenario generation framework that uses a crash-grounded behavior embedding to select adversarial perturbations that are both high-risk and behaviorally realistic (Stoler et al., 14 Jul 2025). The same core idea appears in CrashSight, although that paper does not use the exact term “RCG”: it defines crash grounding as the end-to-end process of anchoring every claim about a crash to observable video evidence and a temporally coherent narrative spanning pre-crash context, collision dynamics, and post-crash aftermath, with expert-validated causal analysis (Gan et al., 9 Apr 2026).

This yields a multi-level view of grounding. At one level, a system must retrieve and align directly observable facts, such as road environment, actor identity, or post-impact state. At another level, it must synthesize across time to reconstruct collision mechanics, temporal ordering, and contributing factors without hallucinating unobservable details. SafeDriver-IQ instantiates a related notion in probabilistic form: outputs are grounded in eight years of NHTSA’s nationally representative CRSS data and validated on Waymo naturalistic scenarios, with the model’s posterior crash probability inverted into a continuous safety score (Roy et al., 16 Mar 2026). The crash-narrative work extends grounding into text, treating Manner of Collision and per-vehicle Crash Type extraction as the mapping of unstructured narratives into structured crash semantics (Wang et al., 10 Oct 2025).

This suggests that RCG is best understood as an umbrella principle rather than a single benchmark or architecture. The common requirement is that safety-relevant outputs remain accountable to the evidentiary substrate from which they are derived.

2. Grounding substrates and task families

Recent RCG-oriented work spans multiple substrates, each imposing distinct observability constraints and different notions of evidentiary sufficiency.

Work	Grounding substrate	Grounded output
CrashSight	250 real-world roadside crash videos from TAD	Phase-aware visual grounding and crash reasoning
SafeDriver-IQ	NHTSA CRSS (2016–2023) + Waymo Open Motion Dataset	Continuous 0–100 safety score
Crash-narrative PLMs	CISS SUMMARY narratives with MANCOLL and CRASHTYPE labels	Per-crash and per-vehicle semantics
V2X braking study	925 GIDAS PCM crashes	Crash avoidance evaluation and failure attribution
RCG scenario generation	WOMD + TADS-traj crash-video trajectories	Crash-grounded adversary selection
Auto-GCAS	DTED-grounded terrain clearance and operational dive states	Ground collision avoidance under safety constraints

CrashSight is infrastructure-centric rather than ego-centric. Its videos come from fixed roadside cameras with overhead or oblique viewpoints, and each clip is segmented into four phases: Traffic Scenario, Crash Content, Aftermath, and Potential Causes. The dataset contains approximately 13,016 multiple-choice QA pairs across 7 categories, with answer positions empirically balanced between 23.3% and 27.8% across A/B/C/D, and uses fixed 60/20/20 train/validation/test splits with clip-level disjointness (Gan et al., 9 Apr 2026).

SafeDriver-IQ operates on tabular and trajectory data rather than visual evidence. It uses 417,335 ACCIDENT records from CRSS across 11 linked tables, filters to 23,194 unique VRU crash records, constructs balanced crash and safe samples, and validates on 500 Waymo scenarios comprising 455 real and 45 synthetic augmentation cases (Roy et al., 16 Mar 2026). By contrast, the crash-narrative study uses the CISS dataset, with SUMMARY text as the input channel and MANCOLL or CRASHTYPE as the structured target semantics (Wang et al., 10 Oct 2025).

The scenario-generation formulation of RCG uses two data sources with complementary roles: WOMD provides large-scale nominal behavior for contrastive pretraining, and TADS-traj provides 144 crash-rich scenarios with 385 accident-involved agent trajectories for fine-tuning toward crash semantics (Stoler et al., 14 Jul 2025). The V2X braking study grounds system evaluation directly in 925 real front-to-side intersection crashes from GIDAS PCM, including trajectories, speeds, road geometry, occlusions, and surface conditions (Zimmermann et al., 12 Jun 2025).

3. Phase-aware semantic grounding in video and text

CrashSight formalizes phase-aware grounding through a two-tier taxonomy. Tier 1 evaluates phase-local visual grounding via Scene Identification, Involved Parties, and Post-Crash Outcome. Tier 2 evaluates cross-phase reasoning via Crash Mechanics, Fault Determination, and Temporal Sequence, while a separate robustness component probes hallucination resistance through cannot-determine, absent-entity, temporal-trap, and false-premise questions (Gan et al., 9 Apr 2026). This design makes the phase reference part of dataset construction rather than merely part of post hoc analysis: each QA has a phase_reference tag during generation and verification.

The benchmark’s construction pipeline further emphasizes evidentiary discipline. Videos range from about 5 to 70 seconds; for benchmarking, 4 frames per clip are uniformly sampled at 1 FPS and resized to approximately 100K–200K pixels per frame. Annotation proceeds through VLM-assisted draft captioning with four explicit phases and entity identifiers, followed by human expert refinement correcting entity precision, spatial relations, phase boundaries, and causal specificity; approximately 90% of drafts require substantial correction (Gan et al., 9 Apr 2026).

Crash-narrative extraction addresses a different but closely related grounding problem: the source is not pixels but unstructured, non-standardized prose written by investigators with varying styles and detail levels. Two tasks are emphasized. MANCOLL is a 7-class per-crash label set with IDs $\{0,1,2,4,5,6,9\}$, while CRASHTYPE is a 98-class per-vehicle label space structured through CRASHCAT $\rightarrow$ CRASHCONF $\rightarrow$ CRASHTYPE. The paper treats CRASHCONF as oracle knowledge and performs CRASHTYPE classification within the corresponding subset, yielding 13 subtasks (Wang et al., 10 Oct 2025).

The narrative work is noteworthy because the target semantics require implicit reasoning rather than lexical matching. Entity reference resolution, vehicle-role asymmetry, and “first harmful event” ordering are central. Domain-adapted compact models perform strongly under these constraints: for MANCOLL, LLaMA3-8B + LoRA reaches 96.1% accuracy on all labels and 97.1% without Unknown; for CRASHTYPE, LLaMA3-8B + LoRA reaches 77.1% for single-vehicle cases, 77.3% for two-vehicle cases, 82.0% for three-vehicle cases, and 84.8% for cases with more than three vehicles (Wang et al., 10 Oct 2025). These results, together with case studies in which the models correct mislabeled “Unknown” or reinterpret loss-of-control coding according to narrative causality, show that grounding can include structured correction of annotation noise when the evidence supports it.

4. Probabilistic and embedding-based formulations

SafeDriver-IQ makes the grounding relation mathematically explicit. Let $p(x)=P(y=1\mid x)$ denote calibrated crash probability for feature vector $x \in \mathbb{R}^{64}$. The raw safety score is defined as

$S_{\text{raw}}(x)=100\cdot (1-p(x)),$

and the calibrated score is

$$S_{\text{cal}}(x)=S_{\text{raw}}(x)\cdot \prod_{k=1}^{K}\alpha_k(x),$$

where $\alpha_k(x)\in(0,1]$ are literature-based penalty factors. Implemented penalties include Ice $(\alpha=0.60)$, Snow $(0.70)$, Wet $\rightarrow$0, Dark (unlit) $\rightarrow$1, High speed $\rightarrow$2, VRU present $\rightarrow$3, and Compound conditions $\rightarrow$4 (Roy et al., 16 Mar 2026). The feature space spans seven domain groups: Temporal, Environmental, Location, VRU-specific, Interaction, Crash/Vehicle, and Metadata.

This formulation is grounded in both crash prevalence and observed compounding. The paper reports ROC-AUC $\rightarrow$5, AP $\rightarrow$6, a safety score distribution with $\rightarrow$7, $\rightarrow$8, and range $\rightarrow$9–$\rightarrow$0, and Waymo validation in which mean crash probabilities order correctly as safe $\rightarrow$1, near-miss $\rightarrow$2, and collision $\rightarrow$3 (Roy et al., 16 Mar 2026). It also reports that 87% of crashes involve at least two factors and that VRU + Urban + Night reaches $\rightarrow$4 baseline, indicating that the grounding target is not merely marginal risk but multi-factor crash structure.

The RCG scenario-generation framework uses a different formalization. It learns a behavior embedding from agent trajectories $\rightarrow$5 and context $\rightarrow$6, with an encoder output $\rightarrow$7 and normalized projection $\rightarrow$8. Pretraining combines reconstruction with instance-level and prototypical contrastive losses, while fine-tuning inserts a LoRA adapter at the bottleneck:

$\rightarrow$9

Unsafe behavior is then represented as a memory bank $p(x)=P(y=1\mid x)$0, and candidate adversaries are scored by their $p(x)=P(y=1\mid x)$1-NN distance to that bank, with lower distance corresponding to more crash-grounded unsafe behavior (Stoler et al., 14 Jul 2025).

This embedding replaces handcrafted scoring objectives in CAT and SEAL. Quantitatively, CAT improves from average $p(x)=P(y=1\mid x)$2 for Success, Crash, and OoR to $p(x)=P(y=1\mid x)$3 under CAT-RCG, while SEAL improves from $p(x)=P(y=1\mid x)$4 to $p(x)=P(y=1\mid x)$5 under SEAL-RCG. Averaged across seven evaluation settings, RCG improves success rate by 9.2% (Stoler et al., 14 Jul 2025). The critical point is that “unsafe” is no longer defined by proximity alone; it is defined by proximity to real accident behavior in embedding space.

5. Intervention-grounded evaluation and control

The V2X-enhanced braking study shows how RCG can function as an evaluation principle for active safety systems. Rather than using synthetic test suites, it evaluates a 2-stage braking cascade on 925 actual GIDAS PCM crashes. Stage 1 is a V2X-triggered partial brake with $p(x)=P(y=1\mid x)$6, jerk $p(x)=P(y=1\mid x)$7, brake application delay $p(x)=P(y=1\mid x)$8, and varied TTC thresholds of $p(x)=P(y=1\mid x)$9, $x \in \mathbb{R}^{64}$0, and $x \in \mathbb{R}^{64}$1. Stage 2 is a sensor-triggered AEB with $x \in \mathbb{R}^{64}$2, the same jerk and delay, TTC threshold $x \in \mathbb{R}^{64}$3, and a no-time-to-evade gate (Zimmermann et al., 12 Jun 2025).

Grounding in real crashes materially changes the conclusions. Under unknown $x \in \mathbb{R}^{64}$4, sensor-only AEB achieves 36.4% avoidance for the 1R/1V sensor set, V2X-only partial braking at TTC $x \in \mathbb{R}^{64}$5 achieves 74.4%, and the 2-stage cascade achieves 88.3%. Detection failures drop from 12.6% of all cases for AEB to 1.8% for V2X-only, while friction becomes the top AEB failure cause at 27.7% (Zimmermann et al., 12 Jun 2025). These numbers are not generic AEB benchmarks; they are consequences of evaluating against real occlusions, low-friction episodes, and late heading changes encoded in PCM.

A distinct but related usage appears in automatic ground collision avoidance for aircraft. The Auto-GCAS paper uses RCG to denote CFIT-like events in operational manned aircraft and proposes an exponential CBF design merged with adaptive sliding manifolds and flight-envelope protection CBFs (Altunkaya et al., 26 Jan 2025). The terrain-clearance barrier is

$x \in \mathbb{R}^{64}$6

with a relative-degree-2 eCBF constraint

$x \in \mathbb{R}^{64}$7

In Monte Carlo over 500 random dives on flat ground with $x \in \mathbb{R}^{64}$8, the reported success rate is 499/500 $x \in \mathbb{R}^{64}$9, and minimum clearance averages approximately $S_{\text{raw}}(x)=100\cdot (1-p(x)),$0, matching the buffer (Altunkaya et al., 26 Jan 2025). Although this aviation usage is domain-specific, it preserves the same underlying commitment: safety claims are tied to physically grounded failure states, explicit safety sets, and real terrain models.

6. Evaluation hierarchies, misconceptions, and open problems

A recurring result across RCG work is that basic semantic competence does not imply crash-grounded reasoning competence. In CrashSight, zero-shot models can describe scenes reasonably well yet still struggle with temporal and causal reasoning in safety-critical settings. InternVL3-8B is the top zero-shot model at 68.7% average accuracy, whereas Qwen2.5-VL-7B-FT reaches the best overall 76.4%; the human expert majority-vote upper bound is 94.7% (Gan et al., 9 Apr 2026). Residual errors after fine-tuning are still dominated by Involved Parties errors at 45.3%, followed by Scene Identification at 21.8% and Crash Mechanics at 20.9%. The stated causes are insufficient visual token budget, only 4 uniformly sampled frames for clips up to 70 seconds, and a frozen visual encoder under QLoRA 4-bit NF4 (Gan et al., 9 Apr 2026).

Another common misconception is to equate “fault” labels with legal judgments. CrashSight states explicitly that Fault Determination labels reflect expert-validated judgments from visible evidence and are not legal determinations (Gan et al., 9 Apr 2026). A similar caution applies to continuous safety scoring: SafeDriver-IQ’s $S_{\text{raw}}(x)=100\cdot (1-p(x)),$1–$S_{\text{raw}}(x)=100\cdot (1-p(x)),$2 outputs are grounded operational indicators derived from classifier posteriors and literature-based calibration, not claims of legal responsibility or absolute safety truth (Roy et al., 16 Mar 2026).

RCG also does not imply that realism and adversariality are interchangeable. The scenario-generation work reports that baseline methods can produce more extreme failures but less realistic low-level kinematics, whereas crash-grounded selection improves plausibility and interactive nuance while still yielding stronger downstream robustness (Stoler et al., 14 Jul 2025). Likewise, the V2X braking study assumes a perfect V2X channel with 100% penetration, no packet loss, and no positioning noise, so its measured gains should be interpreted within those assumptions rather than as unconditional field performance (Zimmermann et al., 12 Jun 2025).

The principal open problems are repeatedly similar across papers. CrashSight identifies broader geographic and scene diversity, stronger verification and refinement mechanisms, adaptive or event-driven frame sampling, better temporal-token utilization, object tracking, spatial grounding, 3D or motion-aware representations, and training objectives that supervise both answer correctness and evidence-grounding consistency (Gan et al., 9 Apr 2026). SafeDriver-IQ identifies synthetic safe-sample bias, limited VRU discriminability, absence of sequence-aware temporal modeling, and subjectivity in rule-based calibration (Roy et al., 16 Mar 2026). The narrative work identifies jurisdictional dependence of taxonomies and persistent difficulty with two-vehicle subject–object asymmetries (Wang et al., 10 Oct 2025). The RCG scenario-generation paper identifies annotation noise from monocular depth and tracking, crash-type coverage bias, and domain shift between mapless video-derived data and structured map-dependent tasks (Stoler et al., 14 Jul 2025).

Taken together, these works indicate that RCG is not a single benchmark category but a methodological standard for safety-critical modeling. Its defining requirement is that prediction, explanation, generation, or intervention remain auditable against real crash evidence, whether that evidence is phase-aware roadside video, national crash records, naturalistic trajectories, free-text investigation narratives, or physically instantiated accident reconstructions.