SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments (2204.01922v3)

Abstract: Designing a safe and human-like decision-making system for an autonomous vehicle is a challenging task. Generative imitation learning is one possible approach for automating policy-building by leveraging both real-world and simulated decisions. Previous work that applies generative imitation learning to autonomous driving policies focuses on learning a low-level controller for simple settings. However, to scale to complex settings, many autonomous driving systems combine fixed, safe, optimization-based low-level controllers with high-level decision-making logic that selects the appropriate task and associated controller. In this paper, we attempt to bridge this gap in complexity by employing Safety-Aware Hierarchical Adversarial Imitation Learning (SHAIL), a method for learning a high-level policy that selects from a set of low-level controller instances in a way that imitates low-level driving data on-policy. We introduce an urban roundabout simulator that controls non-ego vehicles using real data from the Interaction dataset. We then demonstrate empirically that even with simple controller options, our approach can produce better behavior than previous approaches in driver imitation that have difficulty scaling to complex environments. Our implementation is available at https://github.com/sisl/InteractionImitation.

An Analytical Overview of SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving

The paper "SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments" presents a novel approach to the development of decision-making systems for autonomous vehicles, specifically focusing on urban driving environments characterized by complexity and variability. The paper introduces SHAIL, an innovative method leveraging hierarchical adversarial imitation learning to enhance decision-making processes, ensuring that autonomous vehicles operate safely and effectively mimic human-like driving behavior.

Hierarchical Imitation Learning in Autonomous Vehicles

The research underscores the transition from traditional reinforcement learning approaches, which depend on predefined reward functions, to generative imitation learning methods that use expert demonstrations for training. This approach bypasses the need for explicit reward functions, which are often challenging to specify accurately, thus avoiding potential risks associated with reward misspecification. The paper emphasizes the adoption of a hierarchical framework, wherein high-level policies determine the selection of low-level controllers responsible for specific driving tasks. Such a hierarchical paradigm is particularly beneficial as it combines the robustness of optimization-based low-level control with adaptive high-level decision-making strategies.

SHAIL's Innovative Methodology

The SHAIL methodology is a critical advancement in imitation learning, proposing a structure where state and action occupancy measures are expanded to hierarchically include options and initiation states. This reformulation enables the model to match expert distributions more precisely and enhances the reliability of autonomous decisions by using a learned safety-aware layer to predict the feasibility and safety of different driving options. The model employs a binary safety variable that informs the high-level policy, which determines controller options based on predicted safety measures. This nuanced level of control ensures that operational decisions are not only imitative of human driving behavior but are executed within the bounds of safety and situational appropriateness.

Empirical Validation in Simulated Urban Environments

The paper further validates SHAIL by implementing a simulator using real-world data from urban driving scenarios, specifically targeting roundabouts—a challenging driving environment due to their complexity and dynamic interaction demands. The experimental results present a compelling case for SHAIL's superiority over baseline models such as behavior cloning and traditional GAIL (Generative Adversarial Imitation Learning) techniques. SHAIL demonstrated improved success rates and safer navigation metrics, attributed to its hierarchical structure that allows sophisticated reasoning over different driving scenarios and adapts dynamically to environmental changes.

Implications and Future Prospects

The implications of SHAIL are profound for the development of autonomous driving systems. By coupling hierarchical structures with safety-aware decision-making layers, the method allows for more scalable and adaptive autonomous systems, potentially leading to broader applications in various driving environments beyond the urban scope. This approach opens avenues for more generalizable models that can seamlessly adapt to new and unseen conditions, a critical requirement for autonomous driving systems aiming for widespread deployment.

Looking forward, the extension of SHAIL could involve more complex low-level controller designs and enriched safety prediction mechanisms. Additionally, integrating SHAIL within a broader holistic vehicle system could result in even more robust autonomous driving solutions. Exploring cross-setting applications will also provide insights into SHAIL’s effectiveness and adaptability in varying global traffic conditions.

In summary, this paper contributes significantly to the field of autonomous vehicle research by presenting a method that not only improves decision-making in complex environments but also ensures that these decisions prioritize safety without compromising on mimicking expert behavior. The hierarchical adversarial structure of SHAIL offers a promising direction for future advancements in the quest for reliable and human-like autonomous driving systems.