HASS: Hard-case Augmented Synthetic Scenarios

• HASS is a targeted augmentation strategy that generates synthetic, high-risk scenarios to emphasize rare and critical cases in model testing.
• It balances realism and controllability by selectively modifying uncertain regions while preserving essential structures using methods like inpainting and retrieval.
• Applications span autonomous driving, semantic segmentation, portfolio optimization, and self-supervised vision, driving improvements in safety and performance.

to=arxiv_search.search เดิมพันฟรี 日本一本道ीകം so user gets best తాజా papers on HASS exact term and related scenario generation. to=arxiv_search.search 一级a做爰片 天天彩 query="\"Hard-case Augmented Synthetic Scenarios\" OR HASS synthetic scenarios" max_results=10 sort_by="relevance" sort_order="descending" to=arxiv_search.search ฝ่ายขายข่าวitle="Hard-case Augmented Synthetic Scenarios" max_results=10 Hard-case Augmented Synthetic Scenarios (HASS) denotes a family of augmentation strategies that concentrate synthetic data generation, scenario construction, or estimator correction on rare, difficult, safety-critical, or otherwise high-uncertainty cases rather than on ordinary samples. In the cited literature, the term is used most explicitly for a CARLA-generated autonomous-driving dataset containing 13 high-risk edge-case categories (Xiao et al., 6 Aug 2025), but closely related formulations also appear in uncertainty-guided semantic-segmentation augmentation, synthetic hard-negative construction for self-supervised vision, stress-scenario generation for portfolio optimization, and hard-case augmentation of synthetic-control estimators (Röhrich et al., 30 Jun 2026, Giakoumoglou et al., 2 Sep 2025, Choudhary et al., 8 Oct 2025, Ben-Michael et al., 2018). A common motivation is that human-authored scenarios, uniform simulation, or log replay tend to underrepresent the rare but critical events that are most informative for testing safety, robustness, generalization, or tail-risk behavior (Ding, 2023, Shenoy et al., 2020).

1. Scope, terminology, and problem setting

The central premise of HASS is that ordinary data collection disproportionately captures nominal behavior, whereas evaluation and training often fail precisely in the low-probability regions of the data-generating process. In autonomous systems, critical scenarios are described as “rare but important” for testing under risky conditions and unpredictable perturbations, and the desired scenario set should cover “all cases in the real world, especially rare but critical events with extremely low probability” (Ding, 2023). In autonomous driving, rare high-risk scenarios, long-tailed events, and complex interactions are similarly identified as a major bottleneck for real-world data collection and model improvement (Xiao et al., 6 Aug 2025).

The nomenclature is not uniform across the literature. One manuscript on critical scenario generation discusses the underlying problem—scenario diversity, realism, and effectiveness—without defining or describing any HASS framework (Ding, 2023). The AGENTS-LLM excerpt introduces an LLM-agent framework for augmenting real-world traffic scenarios using natural language descriptions, but the supplied excerpt explicitly notes that the Introduction does not contain the formal HASS definitions, algorithms, metrics, or experimental details (Yao et al., 18 Jul 2025). This suggests that HASS is better understood as a recurring methodological pattern than as a single canonical framework.

Across domains, the “hard case” itself is defined in domain-specific ways. In autonomous driving it may mean rare, safety-critical interactions or edge-case categories (Xiao et al., 6 Aug 2025). In semantic segmentation it is defined by predictive entropy over labeled semantic regions, followed by a preserve mask that selects the most uncertain classes until their union covers at least a fraction $\tau \cdot H \cdot W$ of the image (Röhrich et al., 30 Jun 2026). In self-supervised learning it corresponds to negatives with high cosine similarity to the query embedding, selected from a queue and then transformed into synthetic hard negatives (Giakoumoglou et al., 2 Sep 2025). In portfolio optimization it refers to stress-scenario trajectories from the tail of the market-return distribution (Choudhary et al., 8 Oct 2025). In synthetic control, a “hard-case” arises when the treated unit cannot be well approximated by a convex combination of donors, producing poor pre-treatment fit (1811.14551).

2. Autonomous-driving roots: critical scenarios, simulation, and controllable generation

The problem setting most directly associated with HASS is trustworthy autonomy. The critical-scenario literature emphasizes that simulations or digital twins are widely used because they offer low cost and high efficiency, but scenario design remains difficult: human design is time-consuming and bounded by expert experience, while log replay is realistic yet dominated by redundant ordinary cases (Ding, 2023). That diagnosis motivates targeted generation of rare but consequential scenes.

A concrete implementation appears in the dynamic-scenario modeling platform built on Scenic and CARLA. The platform comprises four tightly integrated modules: Scenario Definition with Scenic, Dynamic Behavior Modeling in the simulator loop, Sensor Simulation through the CARLA sensors API, and Data Labeling and Export (Shenoy et al., 2020). Its formal scenario model writes

$$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$

and the joint path distribution is

$$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$

To enrich hard cases, the platform proposes either importance sampling or the cross-entropy method. With a risk indicator such as

$$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$

the objective is to concentrate sampling near the conditional distribution of risky rollouts (Shenoy et al., 2020).

Retrieval-augmented and graph-based scenario synthesis provide two additional formulations. RealGen uses a template/tag database, a behavior encoder, a retrieval module, and an in-context combiner-decoder pipeline. Retrieval uses a similarity score defined as the negative Wasserstein-2 distance, $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$, followed by top-$K$ neighbor selection; the combiner then merges retrieved behaviors, initial poses, and map encodings via multi-head cross-attention before decoding a full multi-agent trajectory (Ding et al., 2023). RealGen reports quantitative metrics including $mADE \approx 1.54$ m, $mFDE \approx 1.21$ m, collision rate $0.05$, and off-road rate $0.04$, and qualitative case studies show crash-scenario generation from a handful of crash templates (Ding et al., 2023).

CC-SGG represents each frame as a directed heterogeneous scene graph whose nodes include Ego vehicle, Road, Lane, Pavement, Shoulder, Car, Bicycle, Pedestrian, Traffic-light, and Object, with cross-edges such as isIn, distance, and relativePosition (Drayson et al., 2023). The model treats corner-case generation as multi-relational link prediction on an extended graph, using HGNN-based graph attention with edge features and a triple-embedding classifier trained by binary cross-entropy. The reported test metrics are Accuracy $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$0, Precision $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$1, Recall $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$2, F1 $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$3, and AUC $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$4; when converted back into OpenSCENARIO and executed in CARLA, the resulting scenarios induce Scenario Collision Rates above $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$5 across five baseline autonomous-driving controllers (Drayson et al., 2023).

3. HASS as a purpose-built simulated driving dataset

The most explicit formal definition of HASS appears in RoboTron-Sim, where HASS is described as “a purpose-built simulated dataset designed to augment underrepresented, high-risk driving events in end-to-end autonomous driving” (Xiao et al., 6 Aug 2025). Let $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$6 denote the distribution of real-world driving samples and $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$7 the subset of simulated scenarios focusing on rare, safety-critical conditions. The dataset is defined as

$$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$8

where $$x_0 \sim p(x_0;\theta), \qquad u_t \sim \pi(u_t \mid x_t;\theta), \qquad x_{t+1}=f(x_t,u_t)+w_t,$$9 is the set of 13 high-risk scenario categories and $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$0 is the CARLA-generated driving distribution (Xiao et al., 6 Aug 2025).

The stated objectives are threefold: rebalancing the long-tailed distribution in $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$1 by oversampling critical edge cases, providing systematic environmental diversity and interaction complexity, and enabling robust sim-to-real transfer by furnishing both rare scenarios and balanced routine maneuvers (Xiao et al., 6 Aug 2025).

Category name	Semantic description	Relative frequency
Jaywalking Pedestrians	Pedestrian crossing unpredictably between lanes	12.5%
Sudden Vehicle Cut-ins	Fast lane-change of another vehicle immediately in front of ego	11.8%
Near-Collision Events	Ego narrowly avoids collision with static or dynamic obstacle	10.4%
Abrupt Pedestrian Appearance	Pedestrian emerges suddenly at sharp turns	9.2%
Red-light Running Vehicle	Other agent runs a red traffic light	8.7%
Pedestrian in Roadway	Pedestrian steps into roadway despite oncoming traffic	7.9%
Opposing Lane Encroachment	Oncoming vehicle drifts into ego lane	8.4%
Lane Invasion	Vehicle crosses lane markings into ego lane	7.3%
Parked Vehicle Activation	Parked vehicle doors open or vehicle starts moving unexpectedly	6.5%
Temporary Parking Ahead	Ego encounters a newly parked vehicle partially blocking lane	6.1%
Roadwork Ahead	Cones, barriers or workers obstruct lane unexpectedly	5.8%
Vehicle Stall in Lane	Other vehicle halts suddenly in the middle of ego lane	4.5%
U-Turn Intrusion	Vehicle makes an unexpected U-turn ahead of ego	1.9%

HASS contains 47 553 simulated samples. Coverage over environmental conditions is explicitly balanced through

$$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$2

with empirical values $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$3, $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$4, $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$5, $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$6, $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$7, and $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$8 (Xiao et al., 6 Aug 2025). Each scenario has 5 frames at 10 Hz, six $$p(x_{0:T},u_{0:T-1};\theta)=p(x_0;\theta)\prod_{t=0}^{T-1}\pi(u_t\mid x_t;\theta)\delta(x_{t+1}-f(x_t,u_t)).$$9 video streams, and annotations including 3D bounding boxes, semantic segmentation masks, traffic signal states, pedestrian intents, and ground-truth trajectories and speeds (Xiao et al., 6 Aug 2025).

Scenario synthesis uses a three-stage CARLA-based pipeline: scenario taxonomy and scene layout, Think2Drive-driven RL agent behavior scripting, and domain randomization with sensor-realistic multimodality (Xiao et al., 6 Aug 2025). Hardness is quantified either as a time-averaged collision indicator,

$$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$0

or as a scenario-level failure rate $$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$1 (Xiao et al., 6 Aug 2025). The dataset is integrated with Scenario-aware Prompt Engineering (SPE) and an Image-to-Ego Encoder (I2E), and the resulting RoboTron-Sim system is reported to improve driving performance in challenging scenarios by around 50% and to reduce collision rates in H2D scenarios by over 50% (Xiao et al., 6 Aug 2025).

4. Uncertainty-guided preservation and regeneration in semantic segmentation

In dense prediction, HASS is formulated as uncertainty-guided synthetic context augmentation. The key idea is to preserve the hard pixels and regenerate only the complementary context. Let $$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$2 produce per-pixel softmax probabilities $$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$3. Predictive entropy at pixel $$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$4 is

$$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$5

and classwise uncertainty is aggregated through the ground-truth class mask $$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$6 as

$$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$7

Classes are sorted by descending $$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$8, then greedily added until their union covers at least $$L(x_{0:T}) = 1_{\{\text{minimum-time-to-collision} \le \tau_x\}},$$9 pixels; the result is the hard-case preserve mask $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$0 (Röhrich et al., 30 Jun 2026).

The generative component uses a pre-trained latent diffusion inpainter $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$1. Given original image $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$2 and preserve mask $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$3, the inpaint mask is $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$4, a forward pass produces

$S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$5

and the final synthetic image is obtained by bit-exact paste-back,

$S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$6

The synthetic label $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$7 keeps the original labels only on preserved pixels and sets all generated pixels to ignore. Fine-tuning then uses ignore-masked cross-entropy only over the original pixels (Röhrich et al., 30 Jun 2026).

This formulation is explicitly designed to avoid label-pixel mismatch. Existing synthetic augmentation methods are criticized for augmenting all foreground objects or entire backgrounds, wasting capacity on uninformative pixels; by contrast, the method “strictly preserves label validity” and computes loss only over the original uncertain regions (Röhrich et al., 30 Jun 2026). No external guardrails such as ControlNet or edge-models are required (Röhrich et al., 30 Jun 2026).

Empirical validation is reported on Cityscapes, UAVID, and BDD100K. On 10% Cityscapes, the one-shot results are Real only: 69.60% mIoU, Simple BG aug: 70.29%, Instance aug: 70.71%, and HASS it1: 71.67%, corresponding to +2.07 over real and +0.96 over the best baseline (Röhrich et al., 30 Jun 2026). Iterative gains on Cityscapes 10% reach 72.09% at it2 and 72.24% at it3, for +2.64 total (Röhrich et al., 30 Jun 2026). Rare-class improvements for Cityscapes 10% it1 are truck +4.10, bus +11.21, and train +1.08 IoU; on UAVID, real 60.31 improves to HASS it3 63.99, with rare-category gains including moving_car +8.75, static_car +9.70, and human +4.60 (Röhrich et al., 30 Jun 2026). The best preserve-area fraction is $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$8, diffusion costs approximately 6 s/image on an A100, and iterative returns diminish after approximately 3 rounds (Röhrich et al., 30 Jun 2026).

5. Generalizations beyond scene simulation

HASS-like constructions also appear outside driving. In robust portfolio optimization, HASS is defined as “stress-scenario trajectories deliberately drawn from the tail of the market-return distribution,” generated by a conditional DDPM and injected into a PPO training loop (Choudhary et al., 8 Oct 2025). The diffusion model conditions on a stress intensity variable $S(z_b^q,z_b^j)=-W_2(z_b^q,z_b^j)$9, with forward kernel

$K$0

reverse kernel

$K$1

and noise-prediction objective

$K$2

The reported stress evaluation covers the 2007–09 Financial Crisis, the 2020 COVID-19 crash, the unseen 2025 Tariff Crisis, and 1000 synthetic HASS sequences at $K$3 (Choudhary et al., 8 Oct 2025). Reported headline results include Sharpe $K$4 and MaxDD $K$5 without HASS augmentation, versus Sharpe $K$6 and MaxDD $K$7 with HASS (DARL), with $K$8 improved by approximately 12% in the Tariff Crisis test (Choudhary et al., 8 Oct 2025).

In self-supervised vision, Syn2Co instantiates HASS through both synthetic images and synthetic hard negatives (Giakoumoglou et al., 2 Sep 2025). A diffusion model produces roughly 130 K synthetic images distributed across the same 100 classes as ImageNet-100, and at training time real and synthetic images are mixed with real fraction $K$9 (Giakoumoglou et al., 2 Sep 2025). Hard negatives are selected from a momentum queue using cosine similarity,

$mADE \approx 1.54$0

then transformed by an overview function $mADE \approx 1.54$1 implementing six variants: interpolation, extrapolation, perturbation, noise jittering, mixing, and adversarial (Giakoumoglou et al., 2 Sep 2025). These synthetic negatives are added to the InfoNCE denominator. Reported ImageNet-100 linear-probe results are 82.12% top-1 for DeiT-S and 83.70% for Swin-T; synthetic negatives alone already push Swin to 84.04% (Giakoumoglou et al., 2 Sep 2025).

A conceptually different extension appears in causal inference through the Augmented Synthetic Control Method. Here “hard-case” denotes settings where standard SCM cannot achieve good pre-treatment fit because the treated unit lies outside or on the boundary of the convex hull of the donor paths (1811.14551). The augmented estimator combines SCM weights with a ridge outcome model, yielding a de-biased counterfactual

$mADE \approx 1.54$2

and an equivalent weighting form that allows negative weights for controlled extrapolation (1811.14551). Simulations with $mADE \approx 1.54$3, $mADE \approx 1.54$4, and one treated unit are reported to reduce absolute bias relative to SCM by 40–90% across calibrated data-generating processes (1811.14551). This use of HASS does not involve synthetic scenes in the simulator sense, but it preserves the central idea of augmentation targeted at failure regimes.

6. Recurrent design pattern, misconceptions, and open issues

Despite domain heterogeneity, a recurrent HASS pattern is visible. First, a hard region is identified through a task-specific criterion: risk indicators and minimum time-to-collision in driving simulation, predictive entropy in segmentation, high cosine-similarity negatives in contrastive learning, tail stress intensity in finance, or poor pre-treatment balance in synthetic control (Shenoy et al., 2020, Röhrich et al., 30 Jun 2026, Giakoumoglou et al., 2 Sep 2025, Choudhary et al., 8 Oct 2025, 1811.14551). Second, the augmentation mechanism is targeted rather than uniform: CARLA scenario synthesis, retrieval-and-composition from tagged behaviors, learned scene-graph perturbation, diffusion inpainting of complementary context, queue-based hard-negative synthesis, or ridge-based bias correction (Xiao et al., 6 Aug 2025, Ding et al., 2023, Drayson et al., 2023, Röhrich et al., 30 Jun 2026, Giakoumoglou et al., 2 Sep 2025, 1811.14551). Third, training or evaluation is explicitly focused on the hard cases through scenario hardness metrics, ignore-masked losses, added synthetic negatives in InfoNCE, $mADE \approx 1.54$5 regularization, or cross-validated extrapolation penalties (Xiao et al., 6 Aug 2025, Röhrich et al., 30 Jun 2026, Giakoumoglou et al., 2 Sep 2025, Choudhary et al., 8 Oct 2025, 1811.14551).

A common misconception is to equate HASS with unconstrained generation from scratch. Several formulations instead preserve critical structure and modify only selected components. RealGen synthesizes new scenarios by combining behaviors from multiple retrieved examples (Ding et al., 2023). CC-SGG learns to minimally manipulate scene graphs and then imports them back into simulation (Drayson et al., 2023). The segmentation variant preserves the uncertain semantic regions bit-exactly and regenerates only the complementary visual context (Röhrich et al., 30 Jun 2026). AGENTS-LLM frames the problem explicitly as augmenting original scenarios from the test set rather than generating novel scenarios from scratch, partly to avoid distributional shift (Yao et al., 18 Jul 2025).

Another recurrent issue is realism versus controllability. The critical-scenario literature notes the limitations of human design and log replay, but also implies that realism and diversity must be balanced carefully (Ding, 2023). The AGENTS-LLM abstract states that generating novel scenarios from scratch can introduce a distributional shift from the original training scenes, undermining evaluation validity for learning-based planners (Yao et al., 18 Jul 2025). In segmentation, the central failure mode is label-pixel mismatch, addressed by paste-back and ignore masking (Röhrich et al., 30 Jun 2026). In finance, synthetic stress scenarios require validation by risk teams through visual inspection, statistical tests, and scenario interpretability (Choudhary et al., 8 Oct 2025). These tensions indicate that HASS is not simply a matter of adding more synthetic data; the augmented samples must remain aligned with the evaluation target, the supervision mechanism, and the operational notion of hardness.

The present literature therefore portrays HASS less as a single fixed algorithm than as an organizing principle for long-tail augmentation. Its unifying concern is selective concentration of model capacity, testing effort, or estimator correction on the parts of the problem distribution where ordinary sampling is weakest and failures are most consequential (Xiao et al., 6 Aug 2025, Röhrich et al., 30 Jun 2026, 1811.14551).