End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving (2102.04738v2)

Abstract: Inspired by the UNet architecture of semantic image segmentation, we propose a lightweight UNet using depthwise separable convolutions (DSUNet) for end-to-end learning of lane detection and path prediction (PP) in autonomous driving. We also design and integrate a PP algorithm with convolutional neural network (CNN) to form a simulation model (CNN-PP) that can be used to assess CNN's performance qualitatively, quantitatively, and dynamically in a host agent car driving along with other agents all in a real-time autonomous manner. DSUNet is 5.16x lighter in model size and 1.61x faster in inference than UNet. DSUNet-PP outperforms UNet-PP in mean average errors of predicted curvature and lateral offset for path planning in dynamic simulation. DSUNet-PP outperforms a modified UNet in lateral error, which is tested in a real car on real road. These results show that DSUNet is efficient and effective for lane detection and path prediction in autonomous driving.

• The paper improves computational efficiency by using depthwise separable convolutions, making DSUNet 5.16 times lighter and 1.61 times faster than UNet.
• The paper integrates a CNN-based path prediction model with DSUNet to achieve nearly 96% F1 score in lane detection and reduced geometric errors.
• The paper validates its approach through extensive simulations on synthetic and real-world datasets, demonstrating its potential for autonomous driving applications.

End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving

The paper presents a novel method for improving lane detection and path prediction in the context of autonomous driving by introducing a deep learning architecture, called DSUNet, which builds upon the foundational UNet architecture. By leveraging depthwise separable convolutions, DSUNet is positioned as a lightweight yet effective alternative to UNet in the domain of semantic road segmentation and lane detection. The research also integrates DSUNet into a comprehensive simulation environment to evaluate its efficacy in real-time autonomous driving.

Proposed Method and Architecture

The authors propose DSUNet as an enhancement over UNet, primarily targeting improvements in computational efficiency without significantly sacrificing accuracy. The primary modifications include the use of depthwise separable convolutions in place of the standard convolutional layers found in UNet. Specifically, DSUNet is reported to be 5.16 times lighter and 1.61 times faster in inference than UNet, which has substantial implications for applications in real-time systems like autonomous vehicles where computational resources can be a limiting factor.

In addition to lane detection, the paper introduces a path prediction algorithm embedded within a convolutional neural network (CNN), forming a combined simulation model (CNN-PP). This model is validated using both qualitative and quantitative measures by simulating a host agent navigating a dynamic environment with other agent vehicles. DSUNet-PP, the integrated model, showcases superior performance metrics including improved mean average errors in predicting curvature and lateral offset compared to a modified UNet, verified in both dynamic simulations and real-world road tests.

Results and Discussion

The paper illustrates the practical benefits of the DSUNet-PP via extensive simulations conducted in modified versions of The Open Racing Car Simulator (TORCS). The integrated system's performance is evaluated on various datasets, including both artificial and real-world images (e.g., LLAMAS, TuSimple datasets). DSUNet-PP achieves nearly 96% average F1 score in lane detection tasks across these datasets.

In terms of path prediction, DSUNet-PP exhibits reduced static and dynamic mean absolute errors (sMAE and dMAE) in curvature and lane center distance metrics on a complex racetrack compared to standard models. Particularly notable is the dynamic performance, which maintains high levels of accuracy even in challenging scenarios with multiple vehicles and environmental factors such as shadows or occlusions.

Implications and Future Directions

The results posit DSUNet-PP as an efficient method for real-time lane detection and path prediction, presenting potential for direct deployment in autonomous vehicle systems where model size and computational efficiency are critical. Moreover, it supports ongoing advancements in CNN architectures specifically tailored for real-time applications on constrained hardware, such as in-vehicle computers.

Future research could include expanding this integrated model's capabilities to account for more diverse and complex traffic scenarios, integrating additional sensory inputs, and optimizing the architecture further to better handle extremely resource-constrained environments. Additionally, applying the proposed techniques to other domains within autonomous systems could explore its adaptability and utility on varying datasets and tasks.

In conclusion, this paper contributes an insightful method by refining existing architectures with computationally efficient operations while maintaining—and in some cases improving—prediction accuracy, serving as a meaningful step toward fully autonomous real-time decision-making systems in automotive applications.