Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Efficient Perception, Planning, and Control Algorithm for Vision-Based Automated Vehicles (2209.07042v6)

Published 15 Sep 2022 in cs.RO, cs.AI, and cs.CV

Abstract: Autonomous vehicles have limited computational resources and thus require efficient control systems. The cost and size of sensors have limited the development of self-driving cars. To overcome these restrictions, this study proposes an efficient framework for the operation of vision-based automatic vehicles; the framework requires only a monocular camera and a few inexpensive radars. The proposed algorithm comprises a multi-task UNet (MTUNet) network for extracting image features and constrained iterative linear quadratic regulator (CILQR) and vision predictive control (VPC) modules for rapid motion planning and control. MTUNet is designed to simultaneously solve lane line segmentation, the ego vehicle's heading angle regression, road type classification, and traffic object detection tasks at approximately 40 FPS for 228 x 228 pixel RGB input images. The CILQR controllers then use the MTUNet outputs and radar data as inputs to produce driving commands for lateral and longitudinal vehicle guidance within only 1 ms. In particular, the VPC algorithm is included to reduce steering command latency to below actuator latency, preventing performance degradation during tight turns. The VPC algorithm uses road curvature data from MTUNet to estimate the appropriate correction for the current steering angle at a look-ahead point to adjust the turning amount. The inclusion of the VPC algorithm in a VPC-CILQR controller leads to higher performance on curvy roads than the use of CILQR alone. Our experiments demonstrate that the proposed autonomous driving system, which does not require high-definition maps, can be applied in current autonomous vehicles.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving,” J. Field Robot, vol. 37, pp. 362-386, 2020.
  2. E. Yurtsever, J. Lambert, A. Carballo and K. Takeda, “A Survey of Autonomous Driving: Common Practices and Emerging Technologies,” IEEE Access, vol. 8, pp. 58443-58469, 2020.
  3. A. Tampuu, T. Matiisen, M. Semikin, D. Fishman and N. Muhammad, “A Survey of End-to-End Driving: Architectures and Training Methods,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 4, pp. 1364-1384, April 2022.
  4. Y. Li and J. Ibanez-Guzman, “Lidar for Autonomous Driving: The Principles, Challenges, and Trends for Automotive Lidar and Perception Systems,” IEEE Signal Process. Mag., vol. 37, no. 4, pp. 50-61, July 2020.
  5. “Active Driving Assistance Systems: Test Results and Design Recommendations,” Consumer Reports, Nov 2020.
  6. M. Teichmann, M. Weber, M. Zöllner, R. Cipolla and R. Urtasun, “MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving,” in IEEE Intelligent Vehicles Symposium (IV), 2018, pp. 1013-1020.
  7. F. Pizzati and F. García, “Enhanced free space detection in multiple lanes based on single CNN with scene identification,” in IEEE Intelligent Vehicles Symposium (IV), 2019, pp. 2536-2541.
  8. Y. Qian, J. M. Dolan and M. Yang, “DLT-Net: Joint Detection of Drivable Areas, Lane Lines, and Traffic Objects,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 11, pp. 4670-4679, Nov. 2020.
  9. C.-Y. Lai, B.-X. Wu, V. M. Shivanna and J.-I. Guo, “MTSAN: Multi-Task Semantic Attention Network for ADAS Applications,” IEEE Access, vol. 9, pp. 50700-50714, 2021.
  10. D. Wu, M. Liao, W. Zhang, X. Wang, X. Bai, W. Cheng, W. Liu, “YOLOP: You Only Look Once for Panoptic Driving Perception,” Mach. Intell. Res., vol. 19, pp. 550–562, 2022.
  11. “Artificial Intelligence & Autopilot,” www.tesla.com/AI. https://www.tesla.com/AI (accessed May. 21, 2023).
  12. B. Paden, M. Čáp, S. Z. Yong, D. Yershov and E. Frazzoli, “A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles,” IEEE Trans. Intell. Veh., vol. 1, no. 1, pp. 33-55, March 2016.
  13. W. Lim, S. Lee, M. Sunwoo and K. Jo, “Hybrid Trajectory Planning for Autonomous Driving in On-Road Dynamic Scenarios,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 1, pp. 341-355, Jan. 2021.
  14. B. Gutjahr, L. Gröll and M. Werling, “Lateral Vehicle Trajectory Optimization Using Constrained Linear Time-Varying MPC,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 6, pp. 1586-1595, June 2017.
  15. V. Turri, A. Carvalho, H. E. Tseng, K. H. Johansson and F. Borrelli, “Linear model predictive control for lane keeping and obstacle avoidance on low curvature roads,” in 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), 2013, pp. 378-383.
  16. A. Katriniok, J. P. Maschuw, F. Christen, L. Eckstein and D. Abel, “Optimal vehicle dynamics control for combined longitudinal and lateral autonomous vehicle guidance,” in European Control Conference (ECC), 2013, pp. 974-979.
  17. S. E. Li, Z. Jia, K. Li and B. Cheng, “Fast Online Computation of a Model Predictive Controller and Its Application to Fuel Economy–Oriented Adaptive Cruise Control,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 3, pp. 1199-1209, June 2015.
  18. W. Lim, S. Lee, J. Yang, M. Sunwoo, Y. Na and K. Jo, “Automatic Weight Determination in Model Predictive Control for Personalized Car-Following Control,” IEEE Access, vol. 10, pp. 19812-19824, 2022.
  19. J. Mattingley and S. Boyd, “CVXGEN: A code generator for embedded convex optimization,” Optim. Eng., vol. 13, no. 1, pp. 1-27, 2012.
  20. J. Chen, W. Zhan and M. Tomizuka, “Constrained iterative LQR for on-road autonomous driving motion planning,” in IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017, pp. 1-7.
  21. J. Chen, W. Zhan and M. Tomizuka, “Autonomous Driving Motion Planning With Constrained Iterative LQR,” IEEE Trans. Intell. Veh., vol. 4, no. 2, pp. 244-254, June 2019.
  22. Y. Pan, Q. Lin, H. Shah and J. M. Dolan, “Safe Planning for Self-Driving Via Adaptive Constrained ILQR,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 2377-2383.
  23. O. Jahanmahin, Q. Lin, Y. Pan and J. M. Dolan, “Jerk-Minimized CILQR for Human-Like Driving on Two-Lane Roadway,” in IEEE Intelligent Vehicles Symposium (IV), 2021, pp. 1282-1289.
  24. J. Ma, Z. Cheng, X. Zhang, Z. Lin, F. L. Lewis and T. H. Lee, “Local Learning Enabled Iterative Linear Quadratic Regulator for Constrained Trajectory Planning,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 9, pp. 5354-5365, Sept. 2023.
  25. H. Fan, F. Zhu, C. Liu, L. Zhang, L. Zhuang, D. Li, W. Zhu, J. Hu, H. Li, Q. Kong, “Baidu Apollo EM Motion Planner,” 2018, arXiv:1807.08048. [Online]. Available: https://arxiv.org/abs/1807.08048
  26. K. Lee, S. Jeon, H. Kim and D. Kum, “Optimal Path Tracking Control of Autonomous Vehicle: Adaptive Full-State Linear Quadratic Gaussian (LQG) Control,” IEEE Access, vol. 7, pp. 109120-109133, 2019.
  27. D.-H. Lee, K.-L. Chen, K.-H. Liou, C.-L. Liu, and J.-L. Liu, “Deep learning and control algorithms of direct perception for autonomous driving,” Appl. Intell. vol. 51, pp. 237-247, 2021.
  28. D.-H. Lee, C.-L. Liu, “Multi-task UNet architecture for end-to-end autonomous driving”, 2021, arXiv:2112.08967. [Online]. Available: https://arxiv.org/abs/2112.08967
  29. D.-H. Lee, C.-L. Liu, “End-to-end deep learning of lane detection and path prediction for real-time autonomous driving,” Signal, Image and Video Processing, vol. 17, pp. 199–205, 2023.
  30. D. Li, D. Zhao, Q. Zhang, and Y. Chen, “Reinforcement learning and deep learning based lateral control for autonomous driving,” IEEE Comput. Intell. Mag. vol. 14, no. 2, pp. 83-98, May 2019.
  31. J. Liu, Z. Yang, Z. Huang, W. Li, S. Dang and H. Li, “Simulation Performance Evaluation of Pure Pursuit, Stanley, LQR, MPC Controller for Autonomous Vehicles,” in IEEE International Conference on Real-time Computing and Robotics (RCAR), 2021, pp. 1444-1449.
  32. P. Lu, C. Cui, S. Xu, H. Peng and F. Wang, “SUPER: A Novel Lane Detection System,” IEEE Trans. Intell. Veh., vol. 6, no. 3, pp. 583-593, Sept. 2021.
  33. A. Bochkovskiy, C.-Y. Wang, H.-Y. M. Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection,” 2020, arXiv:2004.10934. [Online]. Available: https://arxiv.org/abs/2004.10934
  34. Y. Su, K. K. Tan, and T. H. Lee, “Computation delay compensation for real time implementation of robust model predictive control,” J. of Process Control, vol. 23, pp. 1342-1349, 2013.
  35. N. Ibtehaz, M. S. Rahman, “MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation,” Neural Networks, vol. 121, pp. 74-87, 2020.
  36. K. Simonyan, A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in International Conference on Learning Representations (ICLR), 2015, pp. 1-14.
  37. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, 2015, pp. 234-241.
  38. T. Bruls, W. Maddern, A. A. Morye, and P. Newman, “Mark yourself: Road marking segmentation via weakly-supervised annotations from multimodal data,” in IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 1863-1870.
  39. S. Xie and Z. Tu, “Holistically-nested edge detection,” in IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1395-1403.
  40. Q. Zou, H. Jiang, Q. Dai, Y. Yue, L. Chen and Q. Wang, “Robust lane detection from continuous driving scenes using deep neural networks,” IEEE Trans. Veh. Technol, vol. 69, no. 1, pp. 41-54, Jan. 2020.
  41. H. Cui, V. Radosavljevic, F.-C. Chou, T.-H. Lin, T. Nguyen, T.-K. Huang, J. Schneider, and N. Djuric, “Multimodal trajectory predictions for autonomous driving using deep convolutional networks,” in International Conference on Robotics and Automation (ICRA), 2019, pp. 2090-2096.
  42. Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren. “Distance-IoU Loss: Faster and better learning for bounding box regression,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2020, vol. 34, no. 7, pp. 12993-13000.
  43. C. Guo, X.‑l. Lv, Y. Zhang, M.‑l. Zhang, “Improved YOLOv4-tiny network for real-time electronic component detection,” Sci Rep, vol. 11, pp. 22744, 2021.
  44. J. I. Choi and Q. Tian, “Adversarial Attack and Defense of YOLO Detectors in Autonomous Driving Scenarios,” in IEEE Intelligent Vehicles Symposium (IV), 2022, pp. 1011-1017.
  45. Y. Tassa, T. Erez and E. Todorov, “Synthesis and stabilization of complex behaviors through online trajectory optimization,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 4906-4913.
  46. Y. Tassa, N. Mansard and E. Todorov, “Control-limited differential dynamic programming,” in IEEE International Conference on Robotics and Automation (ICRA), 2014, pp. 1168-1175.
  47. B. Plancher, Z. Manchester and S. Kuindersma, “Constrained unscented dynamic programming,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 5674-5680.
  48. C. V. Samak, T. V. Samak, and S. Kandhasamy, “Control Strategies for Autonomous Vehicles,” 2021, arXiv:2011.08729. [Online]. Available: https://arxiv.org/abs/2011.08729
  49. W. Qiu, Q. Ting, Y. Shuyou, G. Hongyan, and C. Hong, “Autonomous vehicle longitudinal following control based on model predictive control,” in 34th Chinese Control Conference (CCC), IEEE, 2015, pp. 8126-8131.
  50. X. Zhang and X. Zhu, “Autonomous path tracking control of intelligent electric vehicles based on lane detection and optimal preview method,” Expert Syst. Appl., vol. 121, pp. 38-48, 2019.
  51. S. Xu, H. Peng, P. Lu, M. Zhu and Y. Tang, “Design and Experiments of Safeguard Protected Preview Lane Keeping Control for Autonomous Vehicles,” IEEE Access, vol. 8, pp. 29944-29953, 2020.
  52. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, and A. Berg, “SSD: Single Shot MultiBox Detector,” in European Conference on Computer Vision (ECCV), 2016, pp. 21-37.
  53. J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 6517-6525.
  54. X. Pan, J. Shi, P. Luo, X. Wang, and X. Tang. “Spatial as deep: spatial CNN for traffic scene understanding,” in The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), 2018, no. 891, pp. 7276–7283.
  55. K. Behrendt, R. Soussan, “Unsupervised Labeled Lane Markers Using Maps,” in IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019, pp. 832-839.
  56. K. Fitzpatrick, “Horizontal Curve Design: An Exercise in Comfort and Appearance,” Transp. Res. Rec., vol. 1445 pp. 47-53, 1994.
  57. C. Chen, A. Seff, A. Kornhauser, and J. Xiao, “DeepDriving: Learning affordance for direct perception in autonomous driving,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 2722-2730.
  58. T. Standley, A. R. Zamir, D. Chen, L. Guibas, J. Malik, and S. Savarese, “Which tasks should be learned together in multi-task learning?” in Proc. 37th Int. Conf. Mach. Learn. (ICML), vol. 119, Jul. 2020, pp. 9120-9132.
  59. I. Kokkinos, “UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 5454-5463.
  60. B. Houska, H. J. Ferreau, and M. Diehl, “ACADO toolkit-An open source framework for automatic control and dynamic optimization,” Optim. Control Appl. Methods. vol. 32, no. 3, pp. 298-312, 2011.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now