Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques (1605.04965v2)

Abstract: Automated vehicles (AVs) must be evaluated thoroughly before their release and deployment. A widely-used evaluation approach is the Naturalistic-Field Operational Test (N-FOT), which tests prototype vehicles directly on the public roads. Due to the low exposure to safety-critical scenarios, N-FOTs are time-consuming and expensive to conduct. In this paper, we propose an accelerated evaluation approach for AVs. The results can be used to generate motions of the primary other vehicles to accelerate the verification of AVs in simulations and controlled experiments. Frontal collision due to unsafe cut-ins is the target crash type of this paper. Human-controlled vehicles making unsafe lane changes are modeled as the primary disturbance to AVs based on data collected by the University of Michigan Safety Pilot Model Deployment Program. The cut-in scenarios are generated based on skewed statistics of collected human driver behaviors, which generate risky testing scenarios while preserving the statistical information so that the safety benefits of AVs in non-accelerated cases can be accurately estimated. The Cross Entropy method is used to recursively search for the optimal skewing parameters. The frequencies of occurrence of conflicts, crashes and injuries are estimated for a modeled automated vehicle, and the achieved accelerated rate is around 2,000 to 20,000. In other words, in the accelerated simulations, driving for 1,000 miles will expose the AV with challenging scenarios that will take about 2 to 20 million miles of real-world driving to encounter. This technique thus has the potential to reduce greatly the development and validation time for AVs.

• The paper introduces an accelerated evaluation approach using importance sampling to simulate millions of real-world miles in lane change scenarios.
• It employs cross entropy techniques to design skewed probability distributions that amplify critical unsafe lane changes.
• The methodology achieves acceleration rates between 2,000 and 20,000, offering a scalable framework for automated vehicle safety assessments.

Accelerated Evaluation of Automated Vehicles in Lane Change Scenarios

The paper "Accelerated Evaluation of Automated Vehicles: Safety in Lane Change Scenarios Based on Importance Sampling Techniques" by Ding Zhao et al. offers an advanced methodology for the evaluation of Automated Vehicles (AVs) under lane change scenarios, utilizing Importance Sampling (IS) techniques to address the challenges associated with conventional evaluation methods like Naturalistic Field Operational Tests (N-FOTs).

Methodology and Contributions

The research addresses significant limitations in current AV testing methods, particularly the time-consuming and costly nature of N-FOTs due to the infrequent occurrence of safety-critical scenarios on public roads. The authors propose an accelerated evaluation approach leveraging Importance Sampling to generate critical lane-change scenarios that maintain statistical representativeness while significantly reducing the required testing time.

To achieve this, human-controlled vehicles making unsafe lane changes, which could potentially lead to frontal collisions, are modeled as primary disturbances to AVs based on data from the University of Michigan Safety Pilot Model Deployment Program. Using the Cross Entropy method, skewed probabilistic distributions are engineered to amplify these critical scenarios. This technique enables AVs to be exposed to scenarios that would ordinarily take millions of miles to encounter in real-world driving, within substantially fewer miles in simulation.

Numerical Results and Implications

The paper presents significant findings where the accelerated evaluation shows an achieved accelerated rate of approximately 2,000 to 20,000. This indicates that AVs can be evaluated under challenging conditions sufficient to equate to 2 to 20 million miles of real-world driving by simulating just 1,000 miles. Thus, the approach can drastically shorten the timeline required for AV development and validation without compromising the safety assessment's accuracy. This enhancement in testing efficiency has profound implications for the scalability of AV deployment and their interaction with human-driven vehicles.

Theoretical and Practical Implications

Theoretically, the research contributes a robust methodological advancement in AV safety evaluation by integrating stochastic modeling with IS and Cross Entropy methods. The ability to effectively simulate rare but critical driving scenarios forms a strong basis for guided advancements in contingency scenario planning in AV systems.

Practically, the model provides an objective framework that can be incorporated into various test platforms beyond simulations, potentially extending to hardware-in-the-loop tests and controlled track environments. This framework offers automotive manufacturers and policymakers a structured strategy for assessing AV performance and safety benefits across a spectrum of likely real-world operational contexts.

Future Directions

Continued research could investigate broader vehicle behavior scenarios such as pedestrian avoidance and different traffic conditions, applying alternative IS distribution families to further increase evaluation efficacy. Additionally, collecting more diverse datasets from varying geographic and traffic environments could refine human-controlled vehicle models, enhancing the robustness and reliability of the evaluation system.

In conclusion, this paper makes a critical contribution to AV safety research, providing a scientifically sound and practically scalable solution to the challenges of evaluating AVs in realistic yet rare driving conditions. This work is a significant step towards the safe and expedited integration of AVs into the transportation ecosystem.