---
title: 'ScenGE: Safety-Critical Scenario Generation'
url: https://www.emergentmind.com/topics/scenge
type: topic
---

# ScenGE: Safety-Critical Scenario Generation

ScenGE is a framework for generating safety-critical scenarios for autonomous vehicles in simulation from a simple benign prompt. It is designed to produce scenarios that are novel, plausible, executable, severe, portable across simulators, and useful for robustness improvement. Its pipeline has two stages: **Meta-Scenario Generation**, in which a large language model grounded in structured driving knowledge infers a plausible but safety-critical adversarial agent and converts the result into executable Scenic code, and **Complex Scenario Evolution**, in which background vehicles are introduced and selected trajectories are perturbed to amplify the threat by reducing the ego vehicle’s maneuvering space and creating critical occlusions [2508.14527].

## 1. Definition and problem setting

ScenGE addresses a specific problem in autonomous-driving safety evaluation: dangerous failures are often caused by rare, compound, and context-dependent interactions, while real-world logs contain few such events and manual scenario design tends to emphasize predefined threat templates or rule-based cases. The framework therefore targets scenarios that are not merely unsafe, but also plausible under traffic norms and sufficiently structured to be executed directly in simulation. The paper describes its target outputs as scenarios that are **novel**, **plausible**, **executable**, **severe**, **portable**, and **useful for robustness improvement** [2508.14527].

The formal core of the first stage is the **meta-scenario**
$$
\mathcal{S}_{\mathrm{meta}} = \{\mathbf{a}_{\mathrm{ego}}, \mathbf{a}_{\mathrm{adv}} \mid \mathrm{R}, \mathrm{L}\},
$$
where $\mathbf{a}_{\mathrm{ego}}$ is the ego vehicle, $\mathbf{a}_{\mathrm{adv}}$ is the adversarial agent, $\mathrm{R}$ is the road type, and $\mathrm{L}$ is the traffic-light state. The adversarial agent is specified by semantic properties $(\mathrm{c}, \mathrm{p}, \mathrm{b})$, namely type, position, and behavior. The final adversarial scenario is then defined as
$$
\mathcal{S}_{adv} = \mathcal{S}_{meta} \cup \{\mathbf{a}_1, \dots, \mathbf{a}_N\},
$$
where the added agents are background vehicles whose trajectories can be optimized to amplify risk [2508.14527].

A common misunderstanding is to treat ScenGE as a template library for known traffic hazards. The paper argues the opposite: its purpose is to go beyond predefined threat patterns and simplistic rule violations by combining language-based adversarial reasoning with trajectory-level collaborative evolution. Another potential misunderstanding is that its effect comes from producing obviously illegal behavior. The reported safety, functionality, and etiquette results are used to argue that ScenGE often creates high-pressure, socially plausible dilemmas rather than relying only on blatant rule breaking [2508.14527].

## 2. Two-stage architecture

The end-to-end workflow begins with a benign text description, retrieves relevant driving knowledge, uses an LLM to infer a dangerous but plausible adversarial situation, parses the result into structured fields, populates a predefined Scenic template, instantiates the scenario in CARLA, generates baseline background traffic with CARLA Traffic Manager, records trajectories, constructs an adversarial collaborator graph, perturbs selected trajectory segments, and finally replays the complete scenario in closed-loop simulation with the ego AV reacting to the adversarial environment [2508.14527].

The first stage, **Meta-Scenario Generation**, is semantic and programmatic. Its inputs are a benign text prompt $\Phi_{\mathrm{base}}$, a fixed instruction prompt $\Phi_{\mathrm{inst}}$, and a structured driving knowledge base $\mathbb{D}$. Its output is a structured meta-scenario plus executable Scenic code. The second stage, **Complex Scenario Evolution**, is trajectory-based. Its inputs are the executable meta-scenario, the trajectories of background vehicles, and the trajectories of the ego and adversarial agent. Its output is a final adversarial scenario with optimized collaborator trajectories [2508.14527].

This decomposition matters because it separates two forms of adversarial pressure. Meta-Scenario Generation identifies the primary threat by reasoning over road type, traffic-light state, and the behavior of a key adversarial agent. Complex Scenario Evolution then amplifies that threat by coordinating surrounding traffic. This suggests that ScenGE is not simply a language-to-simulation compiler; it is a joint semantic-and-dynamical scenario generator in which high-level adversarial intent is converted into executable code and then into multi-agent trajectory perturbations [2508.14527].

## 3. Meta-Scenario Generation

The structured driving knowledge base is
$$
\mathbb{D} = \{\mathbf{D}_r, \mathbf{D}_l, \mathbf{D}_c\},
$$
with three components: 27 driving regulations segmented from official manuals in the USA, Germany, and China; 100 standardized driver’s-license exam questions and answers; and 14 real-world pre-crash scenarios from the NHTSA Pre-Crash Typology Report. The paper uses these sources to ground the LLM so that generated threats are both plausible and deliberately challenging [2508.14527].

Given the ego vehicle, the benign prompt, the retrieved knowledge, and the instruction prompt, the LLM produces textual descriptions for the adversarial agent category, position, behavior, road type, and traffic-light state:
$$
\langle {\Phi}_\mathrm{c}, {\Phi}_\mathrm{p}, {\Phi}_\mathrm{b}, {\Phi}_\mathrm{R}, {\Phi}_\mathrm{L} \rangle
=
f_{\mathrm{LLM}}\left( \mathbf{a}_{\mathrm{ego}}, \Phi_{\mathrm{base}}, f_R(\mathbb{D}, \Phi_{\mathrm{base}}) \mid \Phi_{\mathrm{inst}} \right).
$$
These descriptions are parsed into structured values and used to instantiate the meta-scenario. The paper states that the LLM is guided with retrieval-augmented grounding, few-shot prompting, slot-based templates, and in-context learning. The main LLM in experiments is **qwq-32b**, with **GPT-4o** used in ablations and reported to perform similarly [2508.14527].

The translation from language to simulator execution is handled through **Scenic**. The generated scenario descriptions are parsed into structured values and populated into a predefined Scenic template, which provides syntactic and physical constraints before instantiation in CARLA. The paper does not reproduce the full Scenic template in the provided text, but its role is explicit: it bridges semantic generation and precise in-simulator control [2508.14527].

## 4. Complex Scenario Evolution

After the meta-scenario is created, ScenGE adds background traffic and models each background vehicle $i$ by a trajectory
$$
\tau_i = \{(x_t, y_t)\}_{t=0}^{T}.
$$
A subset of these vehicles is selected for perturbation. The selection is driven by an **adversarial collaborator graph** whose nodes are agents and whose edges encode directional behavioral relevance. The graph is derived from a frame-wise attention matrix over trajectories,
$$
{\bm{M}_{\mathrm{a}} = \frac{({\tau}_{\mathrm{ego}}, {\tau}_{\mathrm{adv}}) \cdot {({\tau}_1, \dots, {\tau}_N)}^\top}{\sqrt{d}} + {\bm{M}_{\mathrm{m}} + \log {\bm{M}_{\mathrm{d}}},
$$
where ${\bm{M}_{\mathrm{m}}$ is a causal mask and ${\bm{M}_{\mathrm{d}}$ is a temporal-decay matrix emphasizing recent interactions [2508.14527].

The collaborator search process aggregates attention across frames, selects the Top-k most influential vehicles, identifies for each selected vehicle the most relevant keyframe, and extracts a perturbable local trajectory window around that keyframe. In the reported implementation, ScenGE perturbs 4 selected vehicles among 10 background vehicles, and each perturbation spans 60% of the vehicle trajectory centered around the most relevant keyframe [2508.14527].

The perturbation objective is
$$
\{\tilde{\tau}_{i^{\star}}\}_{i^{\star} \in K}
=
\underset{\{\tilde{\tau}_{i^{\star}}\}_{i^{\star} \in K}}{\arg\max}
~\mathcal{L}(\tilde{\tau}_{\mathrm{ego}}, \tilde{\tau}_{\mathrm{adv}}, \{\tilde{\tau}_{i^{\star}}\}_{i^{\star} \in K}),
$$
with loss
$$
\mathcal{L}
=
\lambda_1 \underbrace{\| \tilde{\tau}_{i^{\star}} - \tilde{\tau}_{\text{ego}} \|_2}_{\mathcal{L}_{\text{ego}}}
+
\lambda_2 \underbrace{\|\Delta^2 \tilde{\tau}_{i^{\star}}\|_2^2}_{\mathcal{L}_{\text{smooth}}}
+
\lambda_3 \underbrace{\| (\tilde{\tau}_{i^{\star}} - \tilde{\tau}_{\text{ego}}) \times (\tilde{\tau}_{\text{adv}} - \tilde{\tau}_{\text{ego}}) \|_{\perp}}_{\mathcal{L}_{\text{occ}}},
$$
using $\lambda_1 = 0.3$, $\lambda_2 = 0.2$, and $\lambda_3 = 0.5$. The three terms respectively encourage collaborators to remain close to the ego trajectory, preserve smoothness, and align with the ego–adversary line of sight so as to create critical occlusion [2508.14527].

The paper does not provide a separate closed-form collision-severity function, TTC metric, or detailed dynamics equations in the provided text. Severity is instead operationalized through downstream AV outcomes such as collision rate and overall score. This is important because ScenGE’s optimization target is not a single hand-engineered hazard score, but a combination of ego-space restriction, smoothness, and occlusion that is validated by closed-loop failure statistics [2508.14527].

## 5. Experimental evaluation

The main experiments use CARLA and SafeBench, with 8 base traffic scenarios derived from the NHTSA Pre-Crash Typology Report and 10 routes per scenario. For each route, 10 adversarial scenarios are generated, yielding 800 scenarios per method. The evaluated AV models are PPO, SAC, and TD3, and the framework is also tested on the vision-language AV system **LMDrive**. Baselines are Learning-to-Collide, AdvSim, Carla Scenario Generator, Adversarial Trajectory Optimization, and ChatScene [2508.14527].

The headline result is that ScenGE uncovers more severe collision cases, with a reported **+31.96%** improvement on average over state-of-the-art baselines. In the main table, average collision rate rises to **0.820**, while average overall score drops to **0.499**. The per-scenario collision rates reported for ScenGE are 0.860 for Straight Obstacle, 0.773 for Turning Obstacle, 0.837 for Lane Changing, 0.897 for Vehicle Passing, 0.823 for Red-light Running, 0.747 for Unprotected Left-turn, 0.763 for Right-turn, and 0.863 for Crossing Negotiation [2508.14527].

The paper also reports degradation in safety, functionality, and etiquette dimensions. It highlights a **4.96% drop in RF** and a **29.16% reduction in Comp**, together with increases in ACC, YV, and LI. On LMDrive, performance degrades progressively from benign scenarios to meta-scenarios and then to full adversarial scenarios. The framework is additionally reported to transfer to **MetaDrive**. In adversarial training experiments on a SAC-based ego vehicle, ScenGE-generated scenarios improve robustness, with the retrained model achieving collision rate **0.031** and overall score **0.947** when tested against ScenGE scenarios [2508.14527].

The paper supplements simulator results with two further forms of validation. In closed-road real-vehicle tests, two recreated scenario types yielded untimely AV reaction in **73.3%** of pedestrian-emergence trials and failure to alter trajectory in **60%** of unprotected-left-turn trials. In a human evaluation with 30 licensed drivers and 1,200 total responses, the generated scenarios received average ratings of **4.765/5** for plausibility and **4.934/5** for perceived risk. The reported generation cost is approximately **0.024 GPU hours** per scenario, lower than AdvSim, Adversarial Trajectory Optimization, and Learning-to-Collide [2508.14527].

## 6. Position within related research and unresolved points

ScenGE sits within a broader landscape of structured scenario generation and simulator-oriented environment modeling, but its emphasis is distinct. **SceneGen** models realistic traffic-scene layouts conditioned on ego state and HD map, without focusing on adversarial safety-critical scenario discovery [2101.06541]. **Scenic 3.0** and **GzScenic** provide formal scenario-description and simulator-integration layers, but they are not themselves frameworks for LLM-grounded adversarial scenario reasoning and collaborative trajectory evolution [2307.03325]; [2104.08625]. ScenGE differs by combining semantic adversarial generation, executable Scenic instantiation, and multi-agent trajectory optimization in a single pipeline [2508.14527].

The paper explicitly argues against two misconceptions. First, ScenGE is not restricted to reinforcement-learning AVs; it is also evaluated on LMDrive. Second, it does not simply maximize crude traffic-rule violations; its strongest effects arise from coordinated multi-agent interactions that jointly restrict maneuvering space and induce occlusion. A plausible implication is that its contribution lies as much in scenario composition as in adversarial search [2508.14527].

Some technical boundaries are also explicit. The provided text does not include a dedicated limitations subsection. It also does not provide a full Scenic template, detailed feasibility-projection equations, or a separate formal collision-severity function. Those omissions matter because they locate ScenGE’s contribution primarily in structured generation and collaborative evolution rather than in a new physical vehicle model or a closed-form risk functional [2508.14527].

Source: https://www.emergentmind.com/topics/scenge