Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence (2103.10290v2)

Abstract: Decision and control are core functionalities of high-level automated vehicles. Current mainstream methods, such as functionality decomposition and end-to-end reinforcement learning (RL), either suffer high time complexity or poor interpretability and adaptability on real-world autonomous driving tasks. In this paper, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically. First, the static path planning generates several candidate paths only considering static traffic elements. Then, the dynamic optimal tracking is designed to track the optimal path while considering the dynamic obstacles. To that end, we formulate a constrained optimal control problem (OCP) for each candidate path, optimize them separately and follow the one with the best tracking performance. To unload the heavy online computation, we propose a model-based reinforcement learning (RL) algorithm that can be served as an approximate constrained OCP solver. Specifically, the OCPs for all paths are considered together to construct a single complete RL problem and then solved offline in the form of value and policy networks, for real-time online path selecting and tracking respectively. We verify our framework in both simulations and the real world. Results show that compared with baseline methods IDC has an order of magnitude higher online computing efficiency, as well as better driving performance including traffic efficiency and safety. In addition, it yields great interpretability and adaptability among different driving tasks. The effectiveness of the proposed method is also demonstrated in real road tests with complicated traffic conditions.

• The paper introduces an integrated framework that combines static path planning with dynamic optimal tracking to enhance driving decisions.
• It employs model-based reinforcement learning and a novel Generalized Exterior Point Method to efficiently solve constrained control problems.
• Simulations and real-world tests demonstrate up to an order of magnitude improvement in computational efficiency, ensuring safe and adaptable driving performance.

Integrated Decision and Control Framework for Automated Vehicles

This paper introduces an innovative approach to decision-making and control in automated vehicles through an Integrated Decision and Control (IDC) framework, addressing the perennial issues of computational inefficiency and interpretability in autonomous driving systems. The framework proposes an alternative to the traditional decomposed and end-to-end approaches by integrating static path planning with dynamic optimal tracking, thereby enhancing computational efficiency and ensuring adaptability.

Framework Highlights

At the core of the IDC framework is its hierarchical structure that separates static and dynamic components. The static path planning layer is responsible for generating candidate paths based solely on static traffic information such as road topology and traffic signals, efficiently pre-computed or generated in real-time. Each path is associated with an expected velocity derived from established traffic norms, serving as the starting point for subsequent control layers.

The dynamic optimal tracking layer employs a Model-Based Reinforcement Learning (MBRL) approach to address the Constrained Optimal Control Problem (OCP) for path selection and tracking in consideration of dynamic obstacles. A significant contribution of this work is the development of the Generalized Exterior Point Method (GEP), which efficiently solves the OCP by transforming it into an unconstrained optimization problem through the use of penalty functions. The trained neural networks, which approximate optimal control policies and value functions, allow for rapid online decision-making, thus drastically reducing computational expenses traditionally associated with solving OCPs in real-time.

Key Numerical and Practical Results

The results presented in the paper are a testament to the method's high computational efficiency and superior driving performance. The simulations and real-world tests demonstrate that the framework achieves an order of magnitude higher computational efficiency compared to baseline methods. The findings highlight improvements in safety metrics and driving compliance, such as reduced incidences of collisions and adherence to traffic signals, akin to the performance of traditional methods but achieved with significantly reduced computational cost.

The simulation in complex traffic scenarios, such as multi-lane intersections in dense traffic conditions, verifies the framework's applicability across diverse driving tasks, validating its generality and robustness to variations in traffic dynamics. Moreover, experimental evaluations on real-world roads further accentuate the method's adaptability and practical utility.

Theoretical and Practical Implications

The IDC framework presents several theoretical and practical implications. Theoretically, it bridges the gap between model-based control and RL by incorporating model knowledge into the RL paradigm, improving learning efficiency and interpretability. This approach provides a template for enhancing policy and value function learning repurposed to emulate real-life driving scenarios in constrained environments.

Practically, the improved computational efficiency allows for real-time deployment in industrial-grade vehicle systems with limited computational resources. The framework's capability to handle a wide range of tasks and scenarios indicates potential for scaling to complex urban and highway environments, making it viable for real-world autonomous driving applications.

Future Directions in AI Development

Looking forward, further exploration into combining model-based and data-driven techniques can expand capabilities in adaptive traffic management systems beyond current limitations. Enhancements in the robustness to noise and disturbance, as explored through the robustness experiments, are crucial for deploying such systems in less controlled or unpredictable environments. Furthermore, the methodology can inspire novel applications in other domains requiring efficient decision-making under constraints, such as robotics and smart infrastructure management.

In conclusion, the paper not only introduces a comprehensive solution to decision and control in automated vehicles but also provides a foundation for future explorations in AI and machine learning towards building intelligent, adaptable, and computationally efficient systems.