Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DeepQTest: Testing Autonomous Driving Systems with Reinforcement Learning and Real-world Weather Data (2310.05170v1)

Published 8 Oct 2023 in cs.SE, cs.AI, cs.LG, and cs.RO

Abstract: Autonomous driving systems (ADSs) are capable of sensing the environment and making driving decisions autonomously. These systems are safety-critical, and testing them is one of the important approaches to ensure their safety. However, due to the inherent complexity of ADSs and the high dimensionality of their operating environment, the number of possible test scenarios for ADSs is infinite. Besides, the operating environment of ADSs is dynamic, continuously evolving, and full of uncertainties, which requires a testing approach adaptive to the environment. In addition, existing ADS testing techniques have limited effectiveness in ensuring the realism of test scenarios, especially the realism of weather conditions and their changes over time. Recently, reinforcement learning (RL) has demonstrated great potential in addressing challenging problems, especially those requiring constant adaptations to dynamic environments. To this end, we present DeepQTest, a novel ADS testing approach that uses RL to learn environment configurations with a high chance of revealing abnormal ADS behaviors. Specifically, DeepQTest employs Deep Q-Learning and adopts three safety and comfort measures to construct the reward functions. To ensure the realism of generated scenarios, DeepQTest defines a set of realistic constraints and introduces real-world weather conditions into the simulated environment. We employed three comparison baselines, i.e., random, greedy, and a state-of-the-art RL-based approach DeepCOllision, for evaluating DeepQTest on an industrial-scale ADS. Evaluation results show that DeepQTest demonstrated significantly better effectiveness in terms of generating scenarios leading to collisions and ensuring scenario realism compared with the baselines. In addition, among the three reward functions implemented in DeepQTest, Time-To-Collision is recommended as the best design according to our study.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (98)
  1. Testing vision-based control systems using learnable evolutionary algorithms. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE), pages 1016–1026. IEEE, 2018.
  2. Testing autonomous cars for feature interaction failures using many-objective search. In 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 143–154. IEEE, 2018.
  3. Data freshness and energy-efficient uav navigation optimization: A deep reinforcement learning approach. IEEE Transactions on Intelligent Transportation Systems, 2020.
  4. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623–2631, 2019.
  5. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In 2011 33rd International Conference on Software Engineering (ICSE), pages 1–10. IEEE, 2011.
  6. Self-driving like a human driver instead of a robocar: Personalized comfortable driving experience for autonomous vehicles. arXiv preprint arXiv:2001.03908, 2020.
  7. A collision warning system for rear-end collision: a driving simulator study. Procedia-social and behavioral sciences, 20:676–686, 2011.
  8. Testing advanced driver assistance systems using multi-objective search and neural networks. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, pages 63–74, 2016.
  9. Benchmarking image sensors under adverse weather conditions for autonomous driving. In 2018 IEEE Intelligent Vehicles Symposium (IV), pages 1773–1779. IEEE, 2018.
  10. Paul E. Black. Ratcliff/obershelp pattern recognition. In Dictionary of Algorithms and Data Structures [online]. Available from: https://www.nist.gov/dads/HTML/ratcliffObershelp.html, 2021.
  11. Evolutionary functional testing. Computers & Operations Research, 35(10):3144–3160, 2008.
  12. Generating avoidable collision scenarios for testing autonomous driving systems. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), pages 375–386. IEEE, 2020.
  13. Handbook of methods of applied statistics. 1967.
  14. Adversarial evaluation of autonomous vehicles in lane-change scenarios. IEEE Transactions on Intelligent Transportation Systems, 2021.
  15. Model-free deep reinforcement learning for urban autonomous driving. In 2019 IEEE intelligent transportation systems conference (ITSC), pages 2765–2771. IEEE, 2019.
  16. Conditional dqn-based motion planning with fuzzy logic for autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 2020.
  17. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 1907–1915, 2017.
  18. On offline evaluation of vision-based driving models. In Proceedings of the European Conference on Computer Vision (ECCV), pages 236–251, 2018.
  19. End-to-end driving via conditional imitation learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 4693–4700. IEEE, 2018.
  20. Adaptive stress testing with reward augmentation for autonomous vehicle validatio. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pages 163–168. IEEE, 2019.
  21. Krzysztof Czarnecki. Operational world model ontology for automated driving systems–part 1: Road structure. Waterloo Intelligent Systems Engineering Lab (WISE) Report, University of Waterloo, 2018.
  22. A survey of deep learning and its applications: a new paradigm to machine learning. Archives of Computational Methods in Engineering, 27(4):1071–1092, 2020.
  23. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, 6(2):182–197, 2002.
  24. Carla: An open urban driving simulator. In Conference on robot learning, pages 1–16. PMLR, 2017.
  25. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679, 2015.
  26. Challenges of real-world reinforcement learning. arXiv preprint arXiv:1904.12901, 2019.
  27. Baidu apollo em motion planner. arXiv preprint arXiv:1807.08048, 2018.
  28. Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems, 22(3):1341–1360, 2020.
  29. Quantitative analysis of the risk to road networks exposed to slow-moving landslides: a case study in the campania region (southern italy). Landslides, 18:303–319, 2021.
  30. Enabling safe autonomous driving in real-world city traffic using multiple criteria decision making. IEEE Intelligent Transportation Systems Magazine, 3(1):4–17, 2011.
  31. Software release planning: an evolutionary and iterative approach. Information and software technology, 46(4):243–253, 2004.
  32. Dlfuzz: Differential fuzzing testing of deep learning systems. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 739–743, 2018.
  33. Is it safe to drive? an overview of factors, metrics, and datasets for driveability assessment in autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 21(8):3135–3151, 2019.
  34. Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5108–5115. IEEE, 2017.
  35. Many-objective reinforcement learning for online testing of dnn-enabled systems. arXiv preprint arXiv:2210.15432, 2022.
  36. Comparing offline and online testing of deep neural networks: An autonomous car case study. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), pages 85–95. IEEE, 2020.
  37. Is neuron coverage a meaningful measure for testing deep neural networks? In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 851–862, 2020.
  38. Search based software engineering: Techniques, taxonomy, tutorial. In Empirical software engineering and verification, pages 1–59. Springer, 2010.
  39. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  40. Deepmutation++: A mutation testing framework for deep learning systems. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 1158–1161. IEEE, 2019.
  41. Trade-off between jerk and time headway as an indicator of driving style. PloS one, 12(10):e0185856, 2017.
  42. Yaochu Jin. Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation, 1(2):61–70, 2011.
  43. Autoware on board: Enabling autonomous vehicles with embedded systems. In 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS), pages 287–296. IEEE, 2018.
  44. Will the pedestrian cross? a study on pedestrian path prediction. IEEE Transactions on Intelligent Transportation Systems, 15(2):494–506, 2013.
  45. Guiding deep learning system testing using surprise adequacy. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pages 1039–1049. IEEE, 2019.
  46. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 2021.
  47. Deep reinforcement learning for autonomous driving: A survey. arXiv preprint arXiv:2002.00444, 2020.
  48. Robust statistical methods for empirical software engineering. Empirical Software Engineering, 22(2):579–630, 2017.
  49. Mykel J. Kochenderfer and Kyle H. Wray Tim A. Wheeler. Algorithms for Decision Making. MIT press, 2022.
  50. A template-based information extraction from web sites with unstable markup. In Semantic Web Evaluation Challenge: SemWebEval 2014 at ESWC 2014, Anissaras, Crete, Greece, May 25-29, 2014, Revised Selected Papers, pages 89–94. Springer, 2014.
  51. Multi-objective optimization using genetic algorithms: A tutorial. Reliability engineering & system safety, 91(9):992–1007, 2006.
  52. Traffic signal timing manual. Technical report, United States. Federal Highway Administration, 2008.
  53. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097–1105, 2012.
  54. Vehicle detection from 3d lidar using fully convolutional network. arXiv preprint arXiv:1608.07916, 2016.
  55. Av-fuzzer: Finding safety violations in autonomous driving systems. In 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), pages 25–36. IEEE, 2020.
  56. Tune: A research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118, 2018.
  57. Multiobjective reinforcement learning: A comprehensive overview. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 45(3):385–398, 2014.
  58. Learning configurations of operating environment of autonomous vehicles to maximize their collisions. IEEE Transactions on Software Engineering, 49(1):384–402, 2022.
  59. Deepscenario: An open driving scenario dataset for autonomous driving system testing. In 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), pages 52–56, 2023.
  60. Deepmutation: Mutation testing of deep learning systems. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE), pages 100–111. IEEE, 2018.
  61. Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:1801.09454, 2018.
  62. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, pages 50–60, 1947.
  63. Human-level control through deep reinforcement learning. nature, 518(7540):529–533, 2015.
  64. Tactical decision making for lane changing with deep reinforcement learning. 2017.
  65. American Society of Addiction Medicine. Asam openscenario, 2022.
  66. OMG. Object constraint language v2.0. Object Management Group Adopted Specification (formal/06-05-01), 2006.
  67. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32:8026–8037, 2019.
  68. Deepxplore: Automated whitebox testing of deep learning systems. In proceedings of the 26th Symposium on Operating Systems Principles, pages 1–18, 2017.
  69. A first look at the integration of machine learning models in complex autonomous driving systems: a case study on apollo. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 1240–1250, 2020.
  70. Optimal sensor data fusion architecture for object detection in adverse weather conditions. In 2018 21st International Conference on Information Fusion (FUSION), pages 1–8. IEEE, 2018.
  71. Improved semantic stixels via multimodal sensor fusion. In German Conference on Pattern Recognition, pages 447–458. Springer, 2018.
  72. Geoscenario: An open dsl for autonomous driving scenario representation. In 2019 IEEE Intelligent Vehicles Symposium (IV), pages 287–294. IEEE, 2019.
  73. A new safety distance calculation for rear-end collision avoidance. IEEE Transactions on Intelligent Transportation Systems, 22(3):1742–1747, 2020.
  74. Rest apis: a large-scale analysis of compliance with principles and best practices. In International conference on web engineering, pages 21–39. Springer, 2016.
  75. Lgsvl simulator: A high fidelity simulator for autonomous driving. In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pages 1–6. IEEE, 2020.
  76. End-to-end control of kart agent with deep reinforcement learning. In 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1688–1693. IEEE, 2018.
  77. Deep reinforcement learning framework for autonomous driving. Electronic Imaging, 2017(19):70–76, 2017.
  78. Multimodal neural networks: Rgb-d for semantic segmentation and object detection. In Scandinavian conference on image analysis, pages 98–109. Springer, 2017.
  79. Advanced driver assistance systems-past, present and future. In 2011 Seventh International Computer Engineering Conference (ICENCO’2011), pages 72–82. IEEE, 2011.
  80. Understanding, modeling and predicting weather and climate extremes: Challenges and opportunities. Weather and climate extremes, 18:65–74, 2017.
  81. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  82. Jonathan Kelly Steven-Waslander. Motion planning for self-driving cars. Coursera&University of Toronto, https://www.coursera.org/lecture/motion-planning-self-driving-cars/lesson-3-time-to-collision-pS9zl, 2021.
  83. Mind the gap! a study on the transferability of virtual vs physical-world testing of autonomous driving systems. IEEE Transactions on Software Engineering, 2022.
  84. Reinforcement learning: An introduction. MIT press, 2018.
  85. Baidu Apollo team. Apollo: Open source autonomous driving, 2017.
  86. Deeptest: Automated testing of deep-neural-network-driven autonomous cars. In Proceedings of the 40th international conference on software engineering, pages 303–314, 2018.
  87. Defining and substantiating the terms scene, situation, and scenario for automated driving. In 2015 IEEE 18th International Conference on Intelligent Transportation Systems, pages 982–988. IEEE, 2015.
  88. Empirical evaluation methods for multiobjective reinforcement learning algorithms. 2011.
  89. Harsha Vardhan. Hd maps: New age maps powering autonomous vehicles. Geospatial world, 22, 2017.
  90. Katja Vogel. A comparison of headway and time to collision as safety indicators. Accident analysis & prevention, 35(3):427–433, 2003.
  91. Q-learning. Machine learning, 8(3-4):279–292, 1992.
  92. Deephunter: A coverage-guided fuzz testing framework for deep neural networks. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 146–157, 2019.
  93. Deeproad: Gan-based metamorphic testing and input validation framework for autonomous driving systems. In 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 132–142. IEEE, 2018.
  94. Zijun Zhang. Improved adam optimizer for deep neural networks. In 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), pages 1–2. IEEE, 2018.
  95. Deepbillboard: Systematic physical-world testing of autonomous driving systems. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pages 347–358, 2020.
  96. Morvan Zhou. pytorch tutorials. https://github.com/MorvanZhou/PyTorch-Tutorial, 2021.
  97. Automated evaluation of semantic segmentation robustness for autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 21(5):1951–1963, 2019.
  98. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transportation Research Part C: Emerging Technologies, 117:102662, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Chengjie Lu (7 papers)
  2. Tao Yue (37 papers)
  3. Man Zhang (38 papers)
  4. Shaukat Ali (51 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now