Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving (2403.01238v2)

Published 2 Mar 2024 in cs.CV

Abstract: End-to-end motion planning models equipped with deep neural networks have shown great potential for enabling full autonomous driving. However, the oversized neural networks render them impractical for deployment on resource-constrained systems, which unavoidably requires more computational time and resources during reference.To handle this, knowledge distillation offers a promising approach that compresses models by enabling a smaller student model to learn from a larger teacher model. Nevertheless, how to apply knowledge distillation to compress motion planners has not been explored so far. In this paper, we propose PlanKD, the first knowledge distillation framework tailored for compressing end-to-end motion planners. First, considering that driving scenes are inherently complex, often containing planning-irrelevant or even noisy information, transferring such information is not beneficial for the student planner. Thus, we design an information bottleneck based strategy to only distill planning-relevant information, rather than transfer all information indiscriminately. Second, different waypoints in an output planned trajectory may hold varying degrees of importance for motion planning, where a slight deviation in certain crucial waypoints might lead to a collision. Therefore, we devise a safety-aware waypoint-attentive distillation module that assigns adaptive weights to different waypoints based on the importance, to encourage the student to accurately mimic more crucial waypoints, thereby improving overall safety. Experiments demonstrate that our PlanKD can boost the performance of smaller planners by a large margin, and significantly reduce their reference time.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. CARLA autonomous driving leaderboard. https://leaderboard.carla.org/, 2023.
  2. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410, 2016.
  3. Mp3: A unified model to map, perceive, predict and plan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14403–14412, 2021.
  4. Learning from all vehicles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17222–17231, 2022.
  5. Learning by cheating. In Conference on Robot Learning, pages 66–75. PMLR, 2020a.
  6. Learning efficient object detection models with knowledge distillation. Advances in neural information processing systems, 30, 2017.
  7. Distilling knowledge via knowledge review. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5008–5017, 2021.
  8. On self-distilling graph neural network. arXiv preprint arXiv:2011.02255, 2020b.
  9. Bevdistill: Cross-modal bev distillation for multi-view 3d object detection. arXiv preprint arXiv:2211.09386, 2022.
  10. Neat: Neural attention fields for end-to-end autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15793–15803, 2021.
  11. End-to-end driving via conditional imitation learning. In 2018 IEEE international conference on robotics and automation (ICRA), pages 4693–4700. IEEE, 2018.
  12. Lookout: Diverse multi-future prediction and planning for self-driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16107–16116, 2021.
  13. Carla: An open urban driving simulator. In Conference on Robot Learning, pages 1–16. PMLR, 2017.
  14. Wael Farag. Complex trajectory tracking using pid control for autonomous driving. International Journal of Intelligent Transportation Systems Research, 18(2):356–366, 2020.
  15. Freekd: Free-direction knowledge distillation for graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 357–366, 2022.
  16. Lrc-bert: latent-representation contrastive knowledge distillation for natural language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 12830–12838, 2021.
  17. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  18. Knowledge adaptation for efficient semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 578–587, 2019.
  19. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  20. Cross-modality knowledge distillation network for monocular 3d object detection. In European Conference on Computer Vision, pages 87–104. Springer, 2022.
  21. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXVIII, pages 533–549. Springer, 2022.
  22. Planning-oriented autonomous driving, 2023.
  23. Glamd: Global and local attention mask distillation for object detectors. In European Conference on Computer Vision, pages 460–476. Springer, 2022.
  24. Instance-conditional knowledge distillation for object detection. Advances in Neural Information Processing Systems, 34:16468–16480, 2021.
  25. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  26. Cirl: Controllable imitative reinforcement learning for vision-based self-driving. In Proceedings of the European conference on computer vision (ECCV), pages 584–599, 2018.
  27. Multi-granularity structural knowledge distillation for language model compression. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 1001–1011, 2022.
  28. Structured knowledge distillation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2604–2613, 2019.
  29. Rectifier nonlinearities improve neural network acoustic models. In International Conference on Machine Learning, page 3. Atlanta, Georgia, USA, 2013.
  30. Learning situational driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11296–11305, 2020.
  31. Multi-modal fusion transformer for end-to-end autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7077–7087, 2021.
  32. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pages 618–626, 2017.
  33. Safety-enhanced autonomous driving using interpretable sensor fusion transformer. In Conference on Robot Learning, pages 726–737. PMLR, 2023.
  34. Resource-constrained onboard inference of 3d object detection and localisation in point clouds targeting self-driving applications. Sensors, 21(23):7933, 2021.
  35. Patient knowledge distillation for bert model compression. arXiv preprint arXiv:1908.09355, 2019.
  36. A survey of end-to-end driving: Architectures and training methods. IEEE Transactions on Neural Networks and Learning Systems, 33(4):1364–1384, 2020.
  37. The information bottleneck method. arXiv preprint physics/0004057, 2000.
  38. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  39. Learning interpretable end-to-end vision-based motion planning for autonomous driving with optical flow distillation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 13731–13737. IEEE, 2021.
  40. Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline. In Advances in Neural Information Processing Systems, 2022.
  41. Policy pre-training for autonomous driving via self-supervised geometric modeling. In The Eleventh International Conference on Learning Representations, 2023.
  42. End-to-end learning of driving models from large-scale video datasets. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2174–2182, 2017.
  43. Extract the knowledge of graph neural networks and go beyond it: An effective knowledge distillation framework. In Proceedings of the web conference 2021, pages 1227–1237, 2021.
  44. Focal and global knowledge distillation for detectors. In IEEE Conference on Computer Vision and Pattern Recognition, pages 4643–4652, 2022.
  45. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In International Conference on Learning Representations, 2017.
  46. Spatially-partitioned environmental representation and planning architecture for on-road autonomous driving. In 2017 IEEE Intelligent Vehicles Symposium (IV), pages 632–639. IEEE, 2017.
  47. Learning by watching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12711–12721, 2021.
  48. End-to-end urban driving by imitating a reinforcement learning coach. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15222–15232, 2021.
  49. Decoupled knowledge distillation. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 11953–11962, 2022.
  50. Localization distillation for dense object detection. In IEEE Conference on Computer Vision and Pattern Recognition, pages 9407–9416, 2022.
  51. Optimizing the gaussian kernel function with the formulated kernel target alignment criterion for two-class pattern classification. Pattern Recognition, 46(7):2045–2054, 2013.
  52. Better teacher better student: Dynamic prior knowledge for knowledge distillation. In The Eleventh International Conference on Learning Representations, 2023.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now