A Survey of Deep Learning Applications to Autonomous Vehicle Control (1912.10773v1)

Abstract: Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment. However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalising previously learned rules to new scenarios. For these reasons, the use of deep learning for vehicle control is becoming increasingly popular. Although important advancements have been achieved in this field, these works have not been fully summarised. This paper surveys a wide range of research works reported in the literature which aim to control a vehicle through deep learning methods. Although there exists overlap between control and perception, the focus of this paper is on vehicle control, rather than the wider perception problem which includes tasks such as semantic segmentation and object detection. The paper identifies the strengths and limitations of available deep learning methods through comparative analysis and discusses the research challenges in terms of computation, architecture selection, goal specification, generalisation, verification and validation, as well as safety. Overall, this survey brings timely and topical information to a rapidly evolving field relevant to intelligent transportation systems.

• The paper distinguishes its focus on vehicle control by contrasting perceptual tasks with dynamic control using deep learning.
• The paper employs diverse learning strategies, including supervised methods for steering prediction and reinforcement learning for adaptive control.
• The paper highlights challenges like safety, robustness, and generalization, calling for further research into interpretable and adaptive systems.

Deep Learning Applications to Autonomous Vehicle Control: A Survey Overview

The paper "A Survey of Deep Learning Applications to Autonomous Vehicle Control" provides a comprehensive examination of the integration of deep learning techniques in the field of autonomous vehicular control. The authors methodically explore the utilization of deep neural networks in managing both lateral and longitudinal vehicle control, employing approaches such as supervised learning, reinforcement learning, and combinations of both.

Key Contributions and Methodological Insights

1. Focus on Vehicle Control: The manuscript distinguishes between the broader perception tasks such as semantic segmentation and object detection, and its core focus, which lies in the direct control aspects of autonomous vehicles. This is a crucial scope definition as it addresses the control of vehicular dynamics, an important subset of autonomous driving research.
2. Learning Strategies Explored: The survey introduces a diverse range of learning strategies used in autonomous vehicle control:
   • Supervised Learning: Utilized primarily for tasks where expert demonstration is feasible. It is highlighted in works like the NVIDIA PilotNet, where a CNN was trained to predict steering angles based on video data from expert drivers.
   • Reinforcement Learning (RL): Utilized for more complex environments where the vehicle learns through interaction, such as optimizing the balance between safety and performance in Adaptive Cruise Control (ACC) systems. Various studies are cited showcasing the strengths of RL in dynamic vehicle control scenarios.
   • Hybrid Methods: Combinations of RL and supervised learning are shown to improve learning efficiency and policy performance, exemplifying the hybrid approach's success in leveraging the advantages of both learning paradigms.
3. Challenges in Deep Learning for Control: The paper acknowledges the computational demands of training deep networks, the difficulty in defining robust reward functions for RL, challenges in neural architecture selection, and the critical aspect of ensuring safety and generalization across diverse driving conditions.
4. Full Vehicle Control: While the paper initially segregates lateral and longitudinal control methods, it expands into comprehensive control systems encompassing both. This section discusses more sophisticated approaches that address full vehicular autonomy with integrated learning frameworks.

Implications and Future Research Directions

The survey underscores that despite significant advances, there remain substantial challenges in migrating deep learning-based vehicle control from research to practical, real-world applications. Key areas needing further exploration include:

• Safety and Verification: The opacity of deep learning models necessitates innovative methods to ensure functional safety, such as interpretable models or virtual safety nets
• Generalization: Leveraging extensive and diverse datasets to train models capable of generalizing across unforeseen driving environments and conditions.
• Adaptive Systems: Developing real-time adaptive systems that maintain performance and safety even when encountering new driving contexts or sensor anomalies.

Conclusion

The paper provides a timely synthesis of current methodologies and innovations in autonomous vehicle control using deep learning, offering insights into the present state and trajectory of this rapidly evolving domain. Although deep learning models for autonomous vehicles demonstrate potential, the transition towards robust, commercially viable solutions will require ongoing research, particularly in the realms of safety, interpretability, and real-world validation.