A machine learning environment for evaluating autonomous driving software

Abstract: Autonomous vehicles need safe development and testing environments. Many traffic scenarios are such that they cannot be tested in the real world. We see hybrid photorealistic simulation as a viable tool for developing AI (artificial intelligence) software for autonomous driving. We present a machine learning environment for detecting autonomous vehicle corner case behavior. Our environment is based on connecting the CARLA simulation software to TensorFlow machine learning framework and custom AI client software. The AI client software receives data from a simulated world via virtual sensors and transforms the data into information using machine learning models. The AI clients control vehicles in the simulated world. Our environment monitors the state assumed by the vehicle AIs to the ground truth state derived from the simulation model. Our system can search for corner cases where the vehicle AI is unable to correctly understand the situation. In our paper, we present the overall hybrid simulator architecture and compare different configurations. We present performance measurements from real setups, and outline the main parameters affecting the hybrid simulator performance.