Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective (2202.02215v6)

Published 4 Feb 2022 in cs.RO

Abstract: Autonomous driving systems have witnessed a significant development during the past years thanks to the advance in machine learning-enabled sensing and decision-making algorithms. One critical challenge for their massive deployment in the real world is their safety evaluation. Most existing driving systems are still trained and evaluated on naturalistic scenarios collected from daily life or heuristically-generated adversarial ones. However, the large population of cars, in general, leads to an extremely low collision rate, indicating that the safety-critical scenarios are rare in the collected real-world data. Thus, methods to artificially generate scenarios become crucial to measure the risk and reduce the cost. In this survey, we focus on the algorithms of safety-critical scenario generation in autonomous driving. We first provide a comprehensive taxonomy of existing algorithms by dividing them into three categories: data-driven generation, adversarial generation, and knowledge-based generation. Then, we discuss useful tools for scenario generation, including simulation platforms and packages. Finally, we extend our discussion to five main challenges of current works -- fidelity, efficiency, diversity, transferability, controllability -- and research opportunities lighted up by these challenges.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. I. Masi, Y. Wu, T. Hassner, and P. Natarajan, “Deep face recognition: A survey,” in 2018 31st SIBGRAPI conference on graphics, patterns and images (SIBGRAPI).   IEEE, 2018, pp. 471–478.
  2. Z. Saquib, N. Salam, R. P. Nair, N. Pandey, and A. Joshi, “A survey on automatic speaker recognition systems,” Signal Processing and Multimedia, pp. 134–145, 2010.
  3. Z. Zhong, Y. Tang, Y. Zhou, V. d. O. Neves, Y. Liu, and B. Ray, “A survey on scenario-based testing for automated driving systems in high-fidelity simulation,” arXiv preprint arXiv:2112.00964, 2021.
  4. S. Riedmaier, T. Ponn, D. Ludwig, B. Schick, and F. Diermeyer, “Survey on scenario-based safety assessment of automated vehicles,” IEEE access, vol. 8, pp. 87 456–87 477, 2020.
  5. “California Department of Motor Vehicle Disengagement Report,” https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/disengagement-reports/, 2022, [Online].
  6. E. Thorn, S. C. Kimmel, M. Chaka, B. A. Hamilton et al., “A framework for automated driving system testable cases and scenarios,” United States. Department of Transportation. National Highway Traffic Safety Administration, Tech. Rep., 2018.
  7. W. G. Najm, J. D. Smith, M. Yanagisawa et al., “Pre-crash scenario typology for crash avoidance research,” United States. National Highway Traffic Safety Administration, Tech. Rep., 2007.
  8. S. C. Schnelle, M. K. Salaani, S. J. Rao, F. S. Barickman, D. Elsasser et al., “Review of simulation frameworks and standards related to driving scenarios,” United States. Department of Transportation. National Highway Traffic Safety Administration, Tech. Rep., 2019.
  9. J. M. Scanlon, K. D. Kusano, T. Daniel, C. Alderson, A. Ogle, and T. Victor, “Waymo simulated driving behavior in reconstructed fatal crashes within an autonomous vehicle operating domain,” 2021.
  10. K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song, “Robust physical-world attacks on deep learning visual classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1625–1634.
  11. W. Ding, B. Chen, M. Xu, and D. Zhao, “Learning to collide: An adaptive safety-critical scenarios generating method,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 2243–2250.
  12. G. Harshvardhan, M. K. Gourisaria, M. Pandey, and S. S. Rautaray, “A comprehensive survey and analysis of generative models in machine learning,” Computer Science Review, vol. 38, p. 100285, 2020.
  13. I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014.
  14. T. Menzel, G. Bagschik, and M. Maurer, “Scenarios for development, test and validation of automated vehicles,” in 2018 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2018, pp. 1821–1827.
  15. R. van der Made, M. Tideman, U. Lages, R. Katz, and M. Spencer, “Automated generation of virtual driving scenarios from test drive data,” in 24th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration, no. 15-0268, 2015.
  16. J. Fang, D. Zhou, F. Yan, T. Zhao, F. Zhang, Y. Ma, L. Wang, and R. Yang, “Augmented lidar simulator for autonomous driving,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 1931–1938, 2020.
  17. S. Manivasagam, S. Wang, K. Wong, W. Zeng, M. Sazanovich, S. Tan, B. Yang, W.-C. Ma, and R. Urtasun, “Lidarsim: Realistic lidar simulation by leveraging the real world,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 167–11 176.
  18. M. Arief, P. Glynn, and D. Zhao, “An accelerated approach to safely and efficiently test pre-production autonomous vehicles on public streets,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC).   IEEE, 2018, pp. 2006–2011.
  19. C. Knies and F. Diermeyer, “Data-driven test scenario generation for cooperative maneuver planning on highways,” Applied Sciences, vol. 10, no. 22, p. 8154, 2020.
  20. F. Kruber, J. Wurst, and M. Botsch, “An unsupervised random forest clustering technique for automatic traffic scenario categorization,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC).   IEEE, 2018, pp. 2811–2818.
  21. F. Kruber, J. Wurst, E. S. Morales, S. Chakraborty, and M. Botsch, “Unsupervised and supervised learning with the random forest algorithm for traffic scenario clustering and classification,” in 2019 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2019, pp. 2463–2470.
  22. W. Wang and D. Zhao, “Extracting traffic primitives directly from naturalistically logged data for self-driving applications,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 1223–1229, 2018.
  23. Z. Yang, Y. Chai, D. Anguelov, Y. Zhou, P. Sun, D. Erhan, S. Rafferty, and H. Kretzschmar, “Surfelgan: Synthesizing realistic sensor data for autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 118–11 127.
  24. Y. Chen, F. Rong, S. Duggal, S. Wang, X. Yan, S. Manivasagam, S. Xue, E. Yumer, and R. Urtasun, “Geosim: Realistic video simulation via geometry-aware composition for self-driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 7230–7240.
  25. S. Ehrhardt, O. Groth, A. Monszpart, M. Engelcke, I. Posner, N. Mitra, and A. Vedaldi, “Relate: Physically plausible multi-object scene synthesis using structured latent spaces,” arXiv preprint arXiv:2007.01272, 2020.
  26. W. Li, C. Pan, R. Zhang, J. Ren, Y. Ma, J. Fang, F. Yan, Q. Geng, X. Huang, H. Gong et al., “Aads: Augmented autonomous driving simulation using data-driven algorithms,” Science robotics, vol. 4, no. 28, 2019.
  27. A. Xiao, J. Huang, D. Guan, F. Zhan, and S. Lu, “Synlidar: Learning from synthetic lidar sequential point cloud for semantic segmentation,” arXiv preprint arXiv:2107.05399, 2021.
  28. A. Kar, A. Prakash, M.-Y. Liu, E. Cameracci, J. Yuan, M. Rusiniak, D. Acuna, A. Torralba, and S. Fidler, “Meta-sim: Learning to generate synthetic datasets,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 4551–4560.
  29. J. Devaranjan, A. Kar, and S. Fidler, “Meta-sim2: Unsupervised learning of scene structure for synthetic data generation,” in European Conference on Computer Vision.   Springer, 2020, pp. 715–733.
  30. A. Savkin, R. Ellouze, N. Navab, and F. Tombari, “Unsupervised traffic scene generation with synthetic 3d scene graphs,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 1229–1235.
  31. W. Ding, W. Wang, and D. Zhao, “A new multi-vehicle trajectory generator to simulate vehicle-to-vehicle encounters,” arXiv preprint arXiv:1809.05680, 2018.
  32. M. Håkansson and J. Wall, “Driving scenario generation using generative adversarial networks,” Master Thesis, 2021.
  33. W. Ding, M. Xu, and D. Zhao, “Cmts: A conditional multiple trajectory synthesizer for generating safety-critical driving scenarios,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 4314–4321.
  34. Z. Huang, M. Arief, H. Lam, and D. Zhao, “Synthesis of different autonomous vehicles test approaches,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC).   IEEE, 2018, pp. 2000–2005.
  35. Y. Guo, V. V. Kalidindi, M. Arief, W. Wang, J. Zhu, H. Peng, and D. Zhao, “Modeling multi-vehicle interaction scenarios using gaussian random field,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC).   IEEE, 2019, pp. 3974–3980.
  36. S. Tan, K. Wong, S. Wang, S. Manivasagam, M. Ren, and R. Urtasun, “Scenegen: Learning to generate realistic traffic scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 892–901.
  37. M. Wen, J. Park, and K. Cho, “A scenario generation pipeline for autonomous vehicle simulators,” Human-centric Computing and Information Sciences, vol. 10, pp. 1–15, 2020.
  38. T. A. Wheeler, M. J. Kochenderfer, and P. Robbel, “Initial scene configurations for highway traffic propagation,” in 2015 IEEE 18th International Conference on Intelligent Transportation Systems.   IEEE, 2015, pp. 279–284.
  39. T. A. Wheeler and M. J. Kochenderfer, “Factor graph scene distributions for automotive safety analysis,” in 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).   IEEE, 2016, pp. 1035–1040.
  40. ——, “Critical factor graph situation clusters for accelerated automotive safety validation,” in 2019 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2019, pp. 2133–2139.
  41. B. Wulfe, S. Chintakindi, S.-C. T. Choi, R. Hartong-Redden, A. Kodali, and M. J. Kochenderfer, “Real-time prediction of intermediate-horizon automotive collision risk,” arXiv preprint arXiv:1802.01532, 2018.
  42. M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang et al., “End to end learning for self-driving cars,” arXiv preprint arXiv:1604.07316, 2016.
  43. J. Ho and S. Ermon, “Generative adversarial imitation learning,” Advances in neural information processing systems, vol. 29, pp. 4565–4573, 2016.
  44. J. Chen, B. Yuan, and M. Tomizuka, “Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2019, pp. 2884–2890.
  45. L. Jain, V. Chandrasekaran, U. Jang, W. Wu, A. Lee, A. Yan, S. Chen, S. Jha, and S. A. Seshia, “Analyzing and improving neural networks by generating semantic counterexamples through differentiable rendering,” arXiv preprint arXiv:1910.00727, 2019.
  46. J. Tu, M. Ren, S. Manivasagam, M. Liang, B. Yang, R. Du, F. Cheng, and R. Urtasun, “Physically realizable adversarial examples for lidar object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13 716–13 725.
  47. M. Abdelfattah, K. Yuan, Z. J. Wang, and R. Ward, “Towards universal physical attacks on cascaded camera-lidar 3d object detection models,” arXiv preprint arXiv:2101.10747, 2021.
  48. Y. Cao, C. Xiao, D. Yang, J. Fang, R. Yang, M. Liu, and B. Li, “Adversarial objects against lidar-based autonomous driving systems,” arXiv preprint arXiv:1907.05418, 2019.
  49. A. Prakash, S. Boochoon, M. Brophy, D. Acuna, E. Cameracci, G. State, O. Shapira, and S. Birchfield, “Structured domain randomization: Bridging the reality gap by context-aware synthetic data,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 7249–7255.
  50. R. Khirodkar, D. Yoo, and K. M. Kitani, “Vadra: Visual adversarial domain randomization and augmentation,” arXiv preprint arXiv:1812.00491, 2018.
  51. Y. Zhu, C. Miao, F. Hajiaghajani, M. Huai, L. Su, and C. Qiao, “Adversarial attacks against lidar semantic segmentation in autonomous driving,” in Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, 2021, pp. 329–342.
  52. N. Ruiz, S. Schulter, and M. Chandraker, “Learning to simulate,” arXiv preprint arXiv:1810.02513, 2018.
  53. M. O’Kelly, A. Sinha, H. Namkoong, J. Duchi, and R. Tedrake, “Scalable end-to-end autonomous vehicle testing via rare-event simulation,” arXiv preprint arXiv:1811.00145, 2018.
  54. Y. Abeysirigoonawardena, F. Shkurti, and G. Dudek, “Generating adversarial driving scenarios in high-fidelity simulators,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 8271–8277.
  55. W. Ding, B. Chen, B. Li, K. J. Eun, and D. Zhao, “Multimodal safety-critical scenarios generation for decision-making algorithms evaluation,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1551–1558, 2021.
  56. Z. Ghodsi, S. K. S. Hari, I. Frosio, T. Tsai, A. Troccoli, S. W. Keckler, S. Garg, and A. Anandkumar, “Generating and characterizing scenarios for safety testing of autonomous vehicles,” arXiv preprint arXiv:2103.07403, 2021.
  57. J. Wang, A. Pun, J. Tu, S. Manivasagam, A. Sadat, S. Casas, M. Ren, and R. Urtasun, “Advsim: Generating safety-critical scenarios for self-driving vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9909–9918.
  58. A. Nonnengart, M. Klusch, and C. Müller, “Crisgen: Constraint-based generation of critical scenarios for autonomous vehicles,” in International Symposium on Formal Methods.   Springer, 2019, pp. 233–248.
  59. S. Feng, X. Yan, H. Sun, Y. Feng, and H. X. Liu, “Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment,” Nature communications, vol. 12, no. 1, pp. 1–14, 2021.
  60. H. Sun, S. Feng, X. Yan, and H. X. Liu, “Corner case generation and analysis for safety assessment of autonomous vehicles,” arXiv preprint arXiv:2102.03483, 2021.
  61. M. Koren, S. Alsaif, R. Lee, and M. J. Kochenderfer, “Adaptive stress testing for autonomous vehicles,” in 2018 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2018, pp. 1–7.
  62. M. Koren and M. J. Kochenderfer, “Efficient autonomy validation in simulation with adaptive stress testing,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC).   IEEE, 2019, pp. 4178–4183.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Wenhao Ding (43 papers)
  2. Chejian Xu (18 papers)
  3. Mansur Arief (21 papers)
  4. Haohong Lin (14 papers)
  5. Bo Li (1107 papers)
  6. Ding Zhao (172 papers)
Citations (103)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now