ScenGE: Safety-Critical Scenario Generation

• ScenGE is a framework that generates safety-critical autonomous driving scenarios by combining semantic meta-scenario generation with multi-agent trajectory evolution.
• It converts a benign text prompt into executable Scenic code, then perturbs background vehicle trajectories to constrain the ego vehicle’s maneuverability and induce critical occlusions.
• Experimental results in CARLA and real-world tests demonstrate that ScenGE uncovers more severe collision cases and improves robustness when used for adversarial training.

ScenGE is a framework for generating safety-critical scenarios for autonomous vehicles in simulation from a simple benign prompt. It is designed to produce scenarios that are novel, plausible, executable, severe, portable across simulators, and useful for robustness improvement. Its pipeline has two stages: Meta-Scenario Generation, in which a LLM grounded in structured driving knowledge infers a plausible but safety-critical adversarial agent and converts the result into executable Scenic code, and Complex Scenario Evolution, in which background vehicles are introduced and selected trajectories are perturbed to amplify the threat by reducing the ego vehicle’s maneuvering space and creating critical occlusions (Liu et al., 20 Aug 2025).

1. Definition and problem setting

ScenGE addresses a specific problem in autonomous-driving safety evaluation: dangerous failures are often caused by rare, compound, and context-dependent interactions, while real-world logs contain few such events and manual scenario design tends to emphasize predefined threat templates or rule-based cases. The framework therefore targets scenarios that are not merely unsafe, but also plausible under traffic norms and sufficiently structured to be executed directly in simulation. The paper describes its target outputs as scenarios that are novel, plausible, executable, severe, portable, and useful for robustness improvement (Liu et al., 20 Aug 2025).

The formal core of the first stage is the meta-scenario

$$\mathcal{S}_{\mathrm{meta}} = \{\mathbf{a}_{\mathrm{ego}}, \mathbf{a}_{\mathrm{adv}} \mid \mathrm{R}, \mathrm{L}\},$$

where $\mathbf{a}_{\mathrm{ego}}$ is the ego vehicle, $\mathbf{a}_{\mathrm{adv}}$ is the adversarial agent, $\mathrm{R}$ is the road type, and $\mathrm{L}$ is the traffic-light state. The adversarial agent is specified by semantic properties $(\mathrm{c}, \mathrm{p}, \mathrm{b})$, namely type, position, and behavior. The final adversarial scenario is then defined as

$$\mathcal{S}_{adv} = \mathcal{S}_{meta} \cup \{\mathbf{a}_1, \dots, \mathbf{a}_N\},$$

where the added agents are background vehicles whose trajectories can be optimized to amplify risk (Liu et al., 20 Aug 2025).

A common misunderstanding is to treat ScenGE as a template library for known traffic hazards. The paper argues the opposite: its purpose is to go beyond predefined threat patterns and simplistic rule violations by combining language-based adversarial reasoning with trajectory-level collaborative evolution. Another potential misunderstanding is that its effect comes from producing obviously illegal behavior. The reported safety, functionality, and etiquette results are used to argue that ScenGE often creates high-pressure, socially plausible dilemmas rather than relying only on blatant rule breaking (Liu et al., 20 Aug 2025).

2. Two-stage architecture

The end-to-end workflow begins with a benign text description, retrieves relevant driving knowledge, uses an LLM to infer a dangerous but plausible adversarial situation, parses the result into structured fields, populates a predefined Scenic template, instantiates the scenario in CARLA, generates baseline background traffic with CARLA Traffic Manager, records trajectories, constructs an adversarial collaborator graph, perturbs selected trajectory segments, and finally replays the complete scenario in closed-loop simulation with the ego AV reacting to the adversarial environment (Liu et al., 20 Aug 2025).

The first stage, Meta-Scenario Generation, is semantic and programmatic. Its inputs are a benign text prompt $\Phi_{\mathrm{base}}$, a fixed instruction prompt $\Phi_{\mathrm{inst}}$, and a structured driving knowledge base $\mathbb{D}$. Its output is a structured meta-scenario plus executable Scenic code. The second stage, Complex Scenario Evolution, is trajectory-based. Its inputs are the executable meta-scenario, the trajectories of background vehicles, and the trajectories of the ego and adversarial agent. Its output is a final adversarial scenario with optimized collaborator trajectories (Liu et al., 20 Aug 2025).

This decomposition matters because it separates two forms of adversarial pressure. Meta-Scenario Generation identifies the primary threat by reasoning over road type, traffic-light state, and the behavior of a key adversarial agent. Complex Scenario Evolution then amplifies that threat by coordinating surrounding traffic. This suggests that ScenGE is not simply a language-to-simulation compiler; it is a joint semantic-and-dynamical scenario generator in which high-level adversarial intent is converted into executable code and then into multi-agent trajectory perturbations (Liu et al., 20 Aug 2025).

3. Meta-Scenario Generation

The structured driving knowledge base is

$\mathbf{a}_{\mathrm{ego}}$0

with three components: 27 driving regulations segmented from official manuals in the USA, Germany, and China; 100 standardized driver’s-license exam questions and answers; and 14 real-world pre-crash scenarios from the NHTSA Pre-Crash Typology Report. The paper uses these sources to ground the LLM so that generated threats are both plausible and deliberately challenging (Liu et al., 20 Aug 2025).

Given the ego vehicle, the benign prompt, the retrieved knowledge, and the instruction prompt, the LLM produces textual descriptions for the adversarial agent category, position, behavior, road type, and traffic-light state:

$\mathbf{a}_{\mathrm{ego}}$1

These descriptions are parsed into structured values and used to instantiate the meta-scenario. The paper states that the LLM is guided with retrieval-augmented grounding, few-shot prompting, slot-based templates, and in-context learning. The main LLM in experiments is qwq-32b, with GPT-4o used in ablations and reported to perform similarly (Liu et al., 20 Aug 2025).

The translation from language to simulator execution is handled through Scenic. The generated scenario descriptions are parsed into structured values and populated into a predefined Scenic template, which provides syntactic and physical constraints before instantiation in CARLA. The paper does not reproduce the full Scenic template in the provided text, but its role is explicit: it bridges semantic generation and precise in-simulator control (Liu et al., 20 Aug 2025).

4. Complex Scenario Evolution

After the meta-scenario is created, ScenGE adds background traffic and models each background vehicle $\mathbf{a}_{\mathrm{ego}}$2 by a trajectory

$\mathbf{a}_{\mathrm{ego}}$3

A subset of these vehicles is selected for perturbation. The selection is driven by an adversarial collaborator graph whose nodes are agents and whose edges encode directional behavioral relevance. The graph is derived from a frame-wise attention matrix over trajectories,

$\mathbf{a}_{\mathrm{ego}}$4

where $\mathbf{a}_{\mathrm{ego}}$5 is a causal mask and $\mathbf{a}_{\mathrm{ego}}$6 is a temporal-decay matrix emphasizing recent interactions (Liu et al., 20 Aug 2025).

The collaborator search process aggregates attention across frames, selects the Top-k most influential vehicles, identifies for each selected vehicle the most relevant keyframe, and extracts a perturbable local trajectory window around that keyframe. In the reported implementation, ScenGE perturbs 4 selected vehicles among 10 background vehicles, and each perturbation spans 60% of the vehicle trajectory centered around the most relevant keyframe (Liu et al., 20 Aug 2025).

The perturbation objective is

$\mathbf{a}_{\mathrm{ego}}$7

with loss

$\mathbf{a}_{\mathrm{ego}}$8

using $\mathbf{a}_{\mathrm{ego}}$9, $\mathbf{a}_{\mathrm{adv}}$0, and $\mathbf{a}_{\mathrm{adv}}$1. The three terms respectively encourage collaborators to remain close to the ego trajectory, preserve smoothness, and align with the ego–adversary line of sight so as to create critical occlusion (Liu et al., 20 Aug 2025).

The paper does not provide a separate closed-form collision-severity function, TTC metric, or detailed dynamics equations in the provided text. Severity is instead operationalized through downstream AV outcomes such as collision rate and overall score. This is important because ScenGE’s optimization target is not a single hand-engineered hazard score, but a combination of ego-space restriction, smoothness, and occlusion that is validated by closed-loop failure statistics (Liu et al., 20 Aug 2025).

5. Experimental evaluation

The main experiments use CARLA and SafeBench, with 8 base traffic scenarios derived from the NHTSA Pre-Crash Typology Report and 10 routes per scenario. For each route, 10 adversarial scenarios are generated, yielding 800 scenarios per method. The evaluated AV models are PPO, SAC, and TD3, and the framework is also tested on the vision-language AV system LMDrive. Baselines are Learning-to-Collide, AdvSim, Carla Scenario Generator, Adversarial Trajectory Optimization, and ChatScene (Liu et al., 20 Aug 2025).

The headline result is that ScenGE uncovers more severe collision cases, with a reported +31.96% improvement on average over state-of-the-art baselines. In the main table, average collision rate rises to 0.820, while average overall score drops to 0.499. The per-scenario collision rates reported for ScenGE are 0.860 for Straight Obstacle, 0.773 for Turning Obstacle, 0.837 for Lane Changing, 0.897 for Vehicle Passing, 0.823 for Red-light Running, 0.747 for Unprotected Left-turn, 0.763 for Right-turn, and 0.863 for Crossing Negotiation (Liu et al., 20 Aug 2025).

The paper also reports degradation in safety, functionality, and etiquette dimensions. It highlights a 4.96% drop in RF and a 29.16% reduction in Comp, together with increases in ACC, YV, and LI. On LMDrive, performance degrades progressively from benign scenarios to meta-scenarios and then to full adversarial scenarios. The framework is additionally reported to transfer to MetaDrive. In adversarial training experiments on a SAC-based ego vehicle, ScenGE-generated scenarios improve robustness, with the retrained model achieving collision rate 0.031 and overall score 0.947 when tested against ScenGE scenarios (Liu et al., 20 Aug 2025).

The paper supplements simulator results with two further forms of validation. In closed-road real-vehicle tests, two recreated scenario types yielded untimely AV reaction in 73.3% of pedestrian-emergence trials and failure to alter trajectory in 60% of unprotected-left-turn trials. In a human evaluation with 30 licensed drivers and 1,200 total responses, the generated scenarios received average ratings of 4.765/5 for plausibility and 4.934/5 for perceived risk. The reported generation cost is approximately 0.024 GPU hours per scenario, lower than AdvSim, Adversarial Trajectory Optimization, and Learning-to-Collide (Liu et al., 20 Aug 2025).

6. Position within related research and unresolved points

ScenGE sits within a broader landscape of structured scenario generation and simulator-oriented environment modeling, but its emphasis is distinct. SceneGen models realistic traffic-scene layouts conditioned on ego state and HD map, without focusing on adversarial safety-critical scenario discovery (Tan et al., 2021). Scenic 3.0 and GzScenic provide formal scenario-description and simulator-integration layers, but they are not themselves frameworks for LLM-grounded adversarial scenario reasoning and collaborative trajectory evolution (Vin et al., 2023); (Afzal et al., 2021). ScenGE differs by combining semantic adversarial generation, executable Scenic instantiation, and multi-agent trajectory optimization in a single pipeline (Liu et al., 20 Aug 2025).

The paper explicitly argues against two misconceptions. First, ScenGE is not restricted to reinforcement-learning AVs; it is also evaluated on LMDrive. Second, it does not simply maximize crude traffic-rule violations; its strongest effects arise from coordinated multi-agent interactions that jointly restrict maneuvering space and induce occlusion. A plausible implication is that its contribution lies as much in scenario composition as in adversarial search (Liu et al., 20 Aug 2025).

Some technical boundaries are also explicit. The provided text does not include a dedicated limitations subsection. It also does not provide a full Scenic template, detailed feasibility-projection equations, or a separate formal collision-severity function. Those omissions matter because they locate ScenGE’s contribution primarily in structured generation and collaborative evolution rather than in a new physical vehicle model or a closed-form risk functional (Liu et al., 20 Aug 2025).