A Causal Probabilistic Framework for Perception-Informed Closed-Loop Simulation of Autonomous Driving

Abstract: Software-in-the-loop (SIL) simulation is a cornerstone for the validation of modern automotive safety functions. However, many current frameworks utilize ideal sensing, which bypasses the functional insufficiencies of perception algorithms, leading to over-optimistic safety assessments. This paper proposes a perception-informed SIL testing methodology that bridges the gap between ground-truth simulation and real-world perception behavior. We present a framework for incorporating causal probabilistic models into standardized, scenario-based simulation toolchains, applicable to both Advanced Driver Assistance Systems (ADAS) and Autonomous Driving Systems (ADS). Our approach enables the systematic injection of realistic perception errors, such as loss of detection, sizing inaccuracies, and positioning offsets, derived from physical triggering conditions like fog, rain, and object-merging scenarios. By evaluating these ``faults'' within a standardized simulation environment, we demonstrate that perception-informed testing reveals latent operational risks that ideal SIL environments fail to capture, providing a scalable pathway for SOTIF (ISO 21448) validation.

• The paper proposes a causal Bayesian Network framework that links environmental and sensor factors to specific camera perception failures.
• It details how object-level SIL simulations integrate environmental triggers to induce errors in detection, sizing, and positioning.
• Experimental results show that subtle perception errors can cause significant control challenges, bridging the gap between ideal and realistic simulations.

Causal Probabilistic Framework for Perception-Informed Closed-Loop Simulation of Autonomous Driving

Motivation and Problem Context

The validation of ADAS and ADS functionalities demands simulation environments that faithfully represent real-world perception errors. Standard object-level SIL frameworks currently rely on "ideal sensing," yielding optimistic safety metrics by omitting the impact of environmental and physical degradations on perception algorithms. Full-stack raw sensor simulations, although realistic, are computationally prohibitive for large-scale scenario evaluations required by regulatory standards such as SOTIF (ISO 21448) and EuroNCAP. This paper proposes a scalable alternative: a causal, perception-informed object-level SIL simulation using Bayesian Networks (BNs) to systematically inject perception failures conditioned on environmental triggers.

Causal Probabilistic Modeling of Perception Failures

The framework centers on BNs to causally relate Operational Design Domain (ODD) factors (e.g., fog, rain, contrast, distance, illumination) to three main classes of camera perception insufficiencies: detection losses, object sizing inaccuracies, and positioning errors. This model is constructed from system specifications and Failure Mode and Effects Analysis (FMEA), ensuring coverage of both external (atmospheric, scenario-driven) and internal (sensor, digitization) stressors.

Contrast degradation is modeled as a causal chain from physical decay (e.g., fog-induced scattering) and digitalization noise to detection failure, calibrated with threshold metrics such as CNR. The interaction of sharpness-reducing effects and pixel-level blurring translates into measurable localization and sizing errors. Object merging failures—typically arising in dense traffic or cut-in/cut-out maneuvers—are captured through overlap (IoU) statistics and non-maximum suppression (NMS) algorithmic parameters, directly reflecting algorithmic susceptibility to spatial ambiguity.

Figure 1: Causal chain of contrast degradation and detection failure.

Figure 2: Causal chain of sharpness degradation and localization errors.

Figure 3: Causal chain of object merging and localization failures.

SIL Architecture and Failure Injection

The BN-driven causal model is embedded in an esmini-based SIL architecture. Ground truth object data and environmental parameters from OpenSCENARIO/OpenDRIVE inputs are mediated through OSI messages. The BN performs real-time inference at each simulation step, injecting statistically-realistic failures—such as missed detections, offset bounding boxes, and merged objects—into the data stream prior to perception fusion and control logic processing.

This integration ensures that control decisions and vehicle behavior in simulation directly reflect the consequences of plausible perception failures, rather than artifact-free sensor information.

Figure 4: Integration architecture for failure injection in the SIL toolchain.

Experimental Results

Scenario: Stationary Vehicle in Darkness (CCRs)

The model injects detection misses in low-illumination settings as CNR falls below empirically validated thresholds. Gaps in perceived object height and width result, representing stochastic non-detections.

Figure 5: CCRs with the model - target height and width.

Despite these dropouts, ego vehicle deceleration profiles largely match the ideal case, indicating ADAS control logic’s ability to absorb short-duration perception lapses. This suggests sufficient temporal filtering or inertia for certain classes of failures.

Scenario: Moving Vehicle—Fog and Rain (CCRm)

Atmospheric scattering and lens spray induce both detection and localization errors via the BN model. The ego vehicle’s estimated lead object dimensions and longitudinal distances show heightened variance and frequent jitter compared to the ground truth.

Figure 6: CCRm with the model - target height and width.

Control logic remains robust in terms of velocity, but high-frequency noise propagates to observed acceleration, indicating possible impacts on comfort or longer-term actuation wear.

Scenario: Cut-Out Object Merging

Overlapping vehicles during cut-out maneuvers probabilistically merge into a single detection based on the underlying scene geometry and IoU thresholds. This results in erroneous width readings and shifts in perceived object distance, occasionally prompting abrupt, unnecessary braking events by the ADAS logic.

Figure 7: Cut-out with the model - target widths.

The model demonstrates that geometric ambiguities in perception can propagate as upstream trigger events for hazardous or overly conservative control actions.

Implications, Limitations, and Future Work

This approach closes the fidelity gap between ideal object-level and computationally intensive raw sensor simulation. By rooting perception error injection in a causal, physics-informed model, the toolchain enables scalable, standardized, and regulatory-aligned safety validation, fully compatible with SOTIF’s emphasis on functional insufficiencies as opposed to only random failures.

Strong empirical findings include:

• The ADAS system exhibits robust velocity and moderate acceleration stability to stochastic perception dropouts in longitudinal tracking.
• Object merging failures can result in sharp, potentially unsafe, control interventions.
• The realism of perception errors in simulation uncovers risk classes absent from ideal environments.

Further development is needed to expand causal modeling coverage (e.g., glare, lens occlusion), calibrate BN parameters against real-world perception stack data, and align injected failure probabilities with sensor-specific performance envelopes. This will strengthen both predictive validity and regulatory confidence in virtual ADAS/ADS certification processes.

Conclusion

This paper establishes a scalable, theoretically grounded framework for injecting perception-informed failures into closed-loop object-level SIL simulations. By causally linking environmental and scenario triggers to camera perception insufficiencies, the method enables realistic, high-throughput, and standards-compliant safety validation for ADAS and ADS. The results expose risk mechanisms not visible in traditional simulations, providing actionable insight for the advancement of autonomous vehicle verification practices (2606.07186).