Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks (1809.10732v2)

Abstract: Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societal impact. Self-driving vehicles (SDVs) are expected to prevent road accidents and save millions of lives while improving the livelihood and life quality of many more. However, despite large interest and a number of industry players working in the autonomous domain, there still remains more to be done in order to develop a system capable of operating at a level comparable to best human drivers. One reason for this is high uncertainty of traffic behavior and large number of situations that an SDV may encounter on the roads, making it very difficult to create a fully generalizable system. To ensure safe and efficient operations, an autonomous vehicle is required to account for this uncertainty and to anticipate a multitude of possible behaviors of traffic actors in its surrounding. We address this critical problem and present a method to predict multiple possible trajectories of actors while also estimating their probabilities. The method encodes each actor's surrounding context into a raster image, used as input by deep convolutional networks to automatically derive relevant features for the task. Following extensive offline evaluation and comparison to state-of-the-art baselines, the method was successfully tested on SDVs in closed-course tests.

• The paper introduces a deep convolutional network framework that predicts multiple potential future trajectories with calibrated probabilities for vehicular behavior.
• It leverages novel loss functions, including a Multiple-Trajectory Prediction loss with an angle-based distance metric, to overcome mode collapse in trajectory predictions.
• Empirical results from 240 hours of driving data demonstrate significant improvements in both short-term and long-term prediction accuracy, especially at intersections.

Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks

The paper addresses a critical challenge in the development of self-driving vehicles (SDVs): predicting the multimodal trajectories of surrounding traffic actors. Given the inherent uncertainties in traffic behavior, this research proposes a method for anticipating multiple potential future paths of vehicles, thereby enhancing the safety and efficiency of autonomous systems.

Key Contributions

The authors present a sophisticated method employing deep convolutional networks to predict several possible trajectories of traffic actors, along with their associated probabilities. This approach extends beyond single trajectory prediction, accommodating the inherent multimodality of dynamic traffic environments which is crucial for long-term horizon predictions.

Methodology

The proposed methodology begins with encoding an actor's surrounding context into a bird's-eye view (BEV) raster image. This image, along with the actor's real-time state, serves as input for a convolutional neural network (CNN), specifically leveraging the MobileNet-v2 architecture. The network predicts multiple possible future trajectories and their probabilities.

Key components include:

1. Multimodal Optimization: The paper explores several loss functions, including Mixture-of-Experts (ME) and a novel Multiple-Trajectory Prediction (MTP) loss. MTP intelligently identifies the closest predicted mode to the actual trajectory, helping overcome issues like mode collapse seen in other prediction models.
2. Mode Selection and Handling: The paper compares trajectory distance functions for mode selection, finding that an angle-based distance function performs better in intersection scenarios.
3. Lane-Following Extensions: The approach allows for implicit multimodal predictions by utilizing lane information in the rasterization process, thus drawing from traditional lane-following strategies.

Results and Comparisons

Empirical evaluations reveal the efficacy of the proposed method over existing approaches. In experiments involving 240 hours of real-world driving data, the MTP model demonstrated substantial improvements in predicting both short-term and long-term trajectories, particularly at intersections. The results indicate that using three modes provides the best balance in prediction accuracy.

1. Prediction Accuracy: MTP outperformed traditional single-mode predictors like Unscented Kalman Filter and earlier single-trajectory predictors, as well as Mixture Density Networks which suffered from mode collapse.
2. Handling of Multiple Modes: The method proficiently handled multiple potential modes, such as varying velocities and turns, demonstrating well-calibrated probabilities associated with each predicted trajectory.

Implications and Future Directions

The capability to predict multiple potential trajectories with associated probabilities has significant practical implications for autonomous driving. This research not only enhances the safety and decision-making processes of SDVs but also paves the way for further developments in understanding complex traffic interactions. Future work could focus on refining trajectory predictions through enhancements in data representation and network architecture, along with real-world testing to validate these improvements under diverse driving conditions.

In summary, this paper contributes a robust framework for handling multimodal trajectory prediction in autonomous systems, a critical component in advancing the operational safety and effectiveness of self-driving technologies.