Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Monocular 3D lane detection for Autonomous Driving: Recent Achievements, Challenges, and Outlooks (2404.06860v3)

Published 10 Apr 2024 in cs.CV

Abstract: 3D lane detection is essential in autonomous driving as it extracts structural and traffic information from the road in three-dimensional space, aiding self-driving cars in logical, safe, and comfortable path planning and motion control. Given the cost of sensors and the advantages of visual data in color information, 3D lane detection based on monocular vision is an important research direction in the realm of autonomous driving, increasingly gaining attention in both industry and academia. Regrettably, recent advancements in visual perception seem inadequate for the development of fully reliable 3D lane detection algorithms, which also hampers the progress of vision-based fully autonomous vehicles. We believe that there is still considerable room for improvement in 3D lane detection algorithms for autonomous vehicles using visual sensors, and significant enhancements are needed. This review looks back and analyzes the current state of achievements in the field of 3D lane detection research. It covers all current monocular-based 3D lane detection processes, discusses the performance of these cutting-edge algorithms, analyzes the time complexity of various algorithms, and highlights the main achievements and limitations of ongoing research efforts. The survey also includes a comprehensive discussion of available 3D lane detection datasets and the challenges that researchers face but have not yet resolved. Finally, our work outlines future research directions and invites researchers and practitioners to join this exciting field.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (119)
  1. “A review of uncertainty quantification in deep learning: Techniques, applications and challenges” In Information fusion 76 Elsevier, 2021, pp. 243–297
  2. “AAQAL: A machine learning-based tool for performance optimization of parallel SPMV computations using block CSR” In Applied Sciences 12.14 MDPI, 2022, pp. 7073
  3. “WS-3D-Lane: Weakly Supervised 3D Lane Detection With 2D Lane Labels” In arXiv preprint arXiv:2209.11523, 2022
  4. “Flamingo: a visual language model for few-shot learning” In Advances in Neural Information Processing Systems 35, 2022, pp. 23716–23736
  5. “A comparison of uncertainty estimation approaches in deep learning components for autonomous vehicle applications” In arXiv preprint arXiv:2006.15172, 2020
  6. “Deep multi-sensor lane detection” In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 3102–3109 IEEE
  7. “CurveFormer: 3D Lane Detection by Curve Propagation with Curve Queries and Attention” In arXiv preprint arXiv:2209.07989, 2022
  8. “CurveFormer++: 3D Lane Detection by Curve Propagation with Temporal Curve Queries and Attention” In arXiv preprint arXiv:2402.06423, 2024
  9. “Unsupervised Labeled Lane Markers Using Maps” In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019, pp. 832–839 DOI: 10.1109/ICCVW.2019.00111
  10. “Autonomous driving in urban environments: approaches, lessons and challenges” In Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 368.1928 The Royal Society Publishing, 2010, pp. 4649–4672
  11. “Data dissemination for industry 4.0 applications in internet of vehicles based on short-term traffic prediction” In ACM Transactions on Internet Technology (TOIT) 22.1 ACM New York, NY, 2021, pp. 1–18
  12. “Persformer: 3d lane detection via perspective transformer and the openlane benchmark” In European Conference on Computer Vision, 2022, pp. 550–567 Springer
  13. Zhe Chen, Jing Zhang and Dacheng Tao “Progressive lidar adaptation for road detection” In IEEE/CAA Journal of Automatica Sinica 6.3 IEEE, 2019, pp. 693–702
  14. “Vision transformer adapter for dense predictions” In arXiv preprint arXiv:2205.08534, 2022
  15. “An Efficient Transformer for Simultaneous Learning of BEV and Lane Representations in 3D Lane Detection” In arXiv preprint arXiv:2306.04927, 2023
  16. “3d-lanenet+: Anchor free lane detection using a semi-local representation” In arXiv preprint arXiv:2011.01535, 2020
  17. “Semi-local 3d lane detection and uncertainty estimation” In arXiv preprint arXiv:2003.05257, 2020
  18. “Eva: Exploring the limits of masked visual representation learning at scale” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 19358–19369
  19. “Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges” In IEEE Transactions on Intelligent Transportation Systems 22.3 IEEE, 2020, pp. 1341–1360
  20. “Muad: Multiple uncertainties for autonomous driving, a benchmark for multiple uncertainty types and tasks” In arXiv preprint arXiv:2203.01437, 2022
  21. “Dropout as a bayesian approximation: Representing model uncertainty in deep learning” In international conference on machine learning, 2016, pp. 1050–1059 PMLR
  22. “Event-based vision: A survey” In IEEE transactions on pattern analysis and machine intelligence 44.1 IEEE, 2020, pp. 154–180
  23. “Vector: A versatile event-centric benchmark for multi-sensor slam” In IEEE Robotics and Automation Letters 7.3 IEEE, 2022, pp. 8217–8224
  24. “3d-lanenet: end-to-end 3d multiple lane detection” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 2921–2930
  25. “Gen-lanenet: A generalized and scalable approach for 3d lane detection” In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16, 2020, pp. 666–681 Springer
  26. “Decoupling the Curve Modeling and Pavement Regression for Lane Detection” In arXiv preprint arXiv:2309.10533, 2023
  27. “Vehicle Detection and Tracking in Adverse Weather Using a Deep Learning Framework” In IEEE Transactions on Intelligent Transportation Systems 22.7, 2021, pp. 4230–4242 DOI: 10.1109/TITS.2020.3014013
  28. Yinghua He, Hong Wang and Bo Zhang “Color-based road detection in urban traffic scenes” In IEEE Transactions on intelligent transportation systems 5.4 IEEE, 2004, pp. 309–318
  29. “Mobilenets: Efficient convolutional neural networks for mobile vision applications” In arXiv preprint arXiv:1704.04861, 2017
  30. Yuan-Ting Hu, Jia-Bin Huang and Alexander G Schwing “Videomatch: Matching based video object segmentation” In Proceedings of the European conference on computer vision (ECCV), 2018, pp. 54–70
  31. “Anchor3dlane: Learning to regress 3d anchors for monocular 3d lane detection” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17451–17460
  32. “Visual prompt tuning” In European Conference on Computer Vision, 2022, pp. 709–727 Springer
  33. Yanshu Jiang, Qingbo Dong and Liwei Deng “Robust 3D lane detection in complex traffic scenes using Att-Gen-LaneNet” In Scientific reports 12.1 Nature Publishing Group UK London, 2022, pp. 11077
  34. Dongkwon Jin, Dahyun Kim and Chang-Su Kim “Recursive Video Lane Detection” In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 8439–8448
  35. “Eigenlanes: Data-driven lane descriptors for structurally diverse lanes” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 17163–17171
  36. “Real-time road lane detection in urban areas using LiDAR data” In Electronics 7.11 MDPI, 2018, pp. 276
  37. Ori Kelner “Learning halfspaces with membership queries” In arXiv preprint arXiv:2012.10985, 2020
  38. “What uncertainties do we need in bayesian deep learning for computer vision?” In Advances in neural information processing systems 30, 2017
  39. “PMAL: A proxy model active learning approach for vision based industrial applications” In ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 18.2s ACM New York, NY, 2022, pp. 1–18
  40. “D-3DLD: Depth-Aware Voxel Space Mapping for Monocular 3D Lane Detection with Uncertainty” In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5 IEEE
  41. ZuWhan Kim “Robust lane detection and tracking in challenging scenarios” In IEEE Transactions on intelligent transportation systems 9.1 IEEE, 2008, pp. 16–26
  42. “Key points estimation and point instance segmentation approach for lane detection” In IEEE Transactions on Intelligent Transportation Systems 23.7 IEEE, 2021, pp. 8949–8958
  43. “Real-time UAV path planning for autonomous urban scene reconstruction” In 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020, pp. 1156–1162 IEEE
  44. Balaji Lakshminarayanan, Alexander Pritzel and Charles Blundell “Simple and scalable predictive uncertainty estimation using deep ensembles” In Advances in neural information processing systems 30, 2017
  45. “Reconstruct from top view: A 3d lane detection approach based on geometry structure prior” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4370–4379
  46. Mengyu Li, Phuong Minh Chu and Kyungeun Cho “Perspective Transformer and MobileNets-Based 3D Lane Detection from Single 2D Image” In Mathematics 10.19 MDPI, 2022, pp. 3697
  47. “Line-cnn: End-to-end traffic line detection with line proposal unit” In IEEE Transactions on Intelligent Transportation Systems 21.1 IEEE, 2019, pp. 248–258
  48. “Grouplane: End-to-end 3d lane detection with channel-wise grouping” In arXiv preprint arXiv:2307.09472, 2023
  49. “Multi-task multi-sensor fusion for 3d object detection” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7345–7353
  50. “Condlanenet: a top-to-down lane detection framework based on conditional convolution” In Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 3773–3782
  51. “End-to-end lane shape prediction with transformers” In Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 3694–3702
  52. “Learning to predict 3d lane shape and camera pose from a single image via geometry constraints” In Proceedings of the AAAI Conference on Artificial Intelligence 36.2, 2022, pp. 1765–1772
  53. “LATR: 3D Lane Detection from Monocular Images with Transformer” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 7941–7952
  54. “M^ 2-3DLaneNet: Multi-Modal 3D Lane Detection” In arXiv preprint arXiv:2209.05996, 2022
  55. Fulong Ma, Sheng Wang and Ming Liu “An automatic multi-lidar extrinsic calibration algorithm using corner planes” In 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2022, pp. 235–240 IEEE
  56. “Every Dataset Counts: Scaling up Monocular 3D Object Detection with Joint Datasets Training”, 2024 arXiv:2310.00920 [cs.CV]
  57. “Self-Supervised Drivable Area Segmentation Using LiDAR’s Depth Information for Autonomous Driving” In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 41–48 IEEE
  58. “Multiple lane detection algorithm based on optimised dense disparity map estimation” In 2018 IEEE International Conference on Imaging Systems and Techniques (IST), 2018, pp. 1–5 IEEE
  59. “3d object detection from images for autonomous driving: a survey” In IEEE Transactions on Pattern Analysis and Machine Intelligence IEEE, 2023
  60. “Video object segmentation without temporal information” In IEEE transactions on pattern analysis and machine intelligence 41.6 IEEE, 2018, pp. 1515–1530
  61. “One Million Scenes for Autonomous Driving: ONCE Dataset” In arXiv preprint arXiv:2106.11037, 2021
  62. “Concrete problems for autonomous vehicle safety: Advantages of bayesian deep learning”, 2017 International Joint Conferences on Artificial Intelligence, Inc.
  63. Rhiannon Michelmore, Marta Kwiatkowska and Yarin Gal “Evaluating uncertainty quantification in end-to-end autonomous driving control” In arXiv preprint arXiv:1811.06817, 2018
  64. “Towards end-to-end lane detection: an instance segmentation approach” In 2018 IEEE intelligent vehicles symposium (IV), 2018, pp. 286–291 IEEE
  65. “Spatial as deep: Spatial cnn for traffic scene understanding” In Proceedings of the AAAI Conference on Artificial Intelligence 32.1, 2018
  66. Zequn Qin, Huanyu Wang and Xi Li “Ultra fast structure-aware deep lane detection” In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIV 16, 2020, pp. 276–291 Springer
  67. “Focus on local: Detecting lane marker from bottom up via key point” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14122–14130
  68. “Improving language understanding by generative pre-training” OpenAI, 2018
  69. “A survey of deep active learning” In ACM computing surveys (CSUR) 54.9 ACM New York, NY, 2021, pp. 1–40
  70. Christos Sakaridis, Dengxin Dai and Luc Van Gool “ACDC: The adverse conditions dataset with correspondences for semantic driving scene understanding” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10765–10775
  71. Murat Sensoy, Lance Kaplan and Melih Kandemir “Evidential deep learning to quantify classification uncertainty” In Advances in neural information processing systems 31, 2018
  72. “Structure guided lane detection” In arXiv preprint arXiv:2105.05403, 2021
  73. “Towards all-in-one pre-training via maximizing multi-modal mutual information” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15888–15899
  74. “Scalability in Perception for Autonomous Driving: Waymo Open Dataset” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020
  75. “Scalability in perception for autonomous driving: Waymo open dataset” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2446–2454
  76. “Accurate lane detection with atrous convolution and spatial pyramid pooling for autonomous driving” In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2019, pp. 642–647 IEEE
  77. “Lane estimation via deep polynomial regression. arXiv 2020” In arXiv preprint arXiv:2004.10924
  78. “Keep your eyes on the lane: Real-time attention-guided lane detection” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 294–302
  79. Jigang Tang, Songbin Li and Peng Liu “A review of lane detection methods based on deep learning” In Pattern Recognition 111 Elsevier, 2021, pp. 107623
  80. “Siamese image modeling for self-supervised vision representation learning” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2132–2141
  81. “Fcos: Fully convolutional one-stage object detection” In Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9627–9636
  82. Muhammad Usman, Mian Ahmad Jan and Alireza Jolfaei “SPEED: A deep learning assisted privacy-preserved framework for intelligent transportation systems” In IEEE Transactions on Intelligent Transportation Systems 22.7 IEEE, 2020, pp. 4376–4384
  83. “End-to-end lane detection through differentiable least-squares fitting” In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019, pp. 0–0
  84. “Swiftnet: Real-time video object segmentation” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1296–1305
  85. “Git: A generative image-to-text transformer for vision and language” In arXiv preprint arXiv:2205.14100, 2022
  86. “A keypoint-based global association network for lane detection” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1392–1401
  87. “Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework” In International Conference on Machine Learning, 2022, pp. 23318–23340 PMLR
  88. “ECA-Net: Efficient channel attention for deep convolutional neural networks” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11534–11542
  89. “BEV Lane Det: Fast Lane Detection on BEV Ground” In arXiv preprint arXiv:2210.06006, 2022
  90. “BEV-LaneDet: a Simple and Effective 3D Lane Detection Baseline”, 2023 arXiv:2210.06006 [cs.CV]
  91. “Bev-lanedet: An efficient 3d lane detection based on virtual camera via key-points” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1002–1011
  92. “Internimage: Exploring large-scale vision foundation models with deformable convolutions” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14408–14419
  93. “Image as a foreign language: Beit pretraining for all vision and vision-language tasks” In arXiv preprint arXiv:2208.10442, 2022
  94. “Images speak in images: A generalist painter for in-context visual learning” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6830–6839
  95. “Spatio-Temporal Fusion-based Monocular 3D Lane Detection.” In BMVC, 2022, pp. 314
  96. Yue Wang, Eam Khwang Teoh and Dinggang Shen “Lane detection and tracking using B-Snake” In Image and Vision computing 22.4 Elsevier, 2004, pp. 269–280
  97. “Row anchor selection classification method for early-stage crop row-following” In Computers and Electronics in Agriculture 192 Elsevier, 2022, pp. 106577
  98. “Multi-objective optimization for resource allocation in vehicular cloud computing networks” In IEEE Transactions on Intelligent Transportation Systems 23.12 IEEE, 2021, pp. 25536–25545
  99. Wikipedia contributors “Event camera — Wikipedia, The Free Encyclopedia” [Online; accessed 18-March-2024], 2024 URL: https://en.wikipedia.org/w/index.php?title=Event_camera&oldid=1202151981
  100. Wikipedia contributors “Hungarian algorithm — Wikipedia, The Free Encyclopedia” [Online; accessed 30-March-2024], 2024 URL: https://en.wikipedia.org/w/index.php?title=Hungarian_algorithm&oldid=1212340485
  101. “Cbam: Convolutional block attention module” In Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19
  102. “Sequence level semantics aggregation for video object detection” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9217–9225
  103. “A brief overview of ChatGPT: The history, status quo and potential future development” In IEEE/CAA Journal of Automatica Sinica 10.5 IEEE, 2023, pp. 1122–1136
  104. “PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving”, 2021 arXiv:2112.12610 [cs.CV]
  105. “CenterLineDet: CenterLine Graph Detection for Road Lanes with Vehicle-mounted Sensors by Transformer for HD Map Generation” In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 3553–3559 IEEE
  106. “Rngdet: Road network graph detection by transformer in aerial images” In IEEE Transactions on Geoscience and Remote Sensing 60 IEEE, 2022, pp. 1–12
  107. “Once-3dlanes: Building monocular 3d lane detection” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 17143–17152
  108. “Cpt: Colorful prompt tuning for pre-trained vision-language models” In arXiv preprint arXiv:2109.11797, 2021
  109. “End-to-end lane marker detection via row-wise classification” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 1006–1007
  110. “Unified vision and language prompt learning” In arXiv preprint arXiv:2210.07225, 2022
  111. “End to end video segmentation for driving: Lane detection for autonomous car” In arXiv preprint arXiv:1812.05914, 2018
  112. “Vil-100: A new dataset and a baseline model for video instance lane detection” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15681–15690
  113. “Perception and sensing for autonomous vehicles under adverse weather conditions: A survey” In ISPRS Journal of Photogrammetry and Remote Sensing 196 Elsevier, 2023, pp. 146–177
  114. “CurbNet: Curb Detection Framework Based on LiDAR Point Cloud Segmentation”, 2024 arXiv:2403.16794 [cs.CV]
  115. “Clrnet: Cross layer refinement network for lane detection” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 898–907
  116. “Resa: Recurrent feature-shift aggregator for lane detection” In Proceedings of the AAAI Conference on Artificial Intelligence 35.4, 2021, pp. 3547–3554
  117. “A novel lane detection based on geometrical model and gabor filter” In 2010 IEEE Intelligent Vehicles Symposium, 2010, pp. 59–64 IEEE
  118. Xingyi Zhou, Dequan Wang and Philipp Krähenbühl “Objects as points” In arXiv preprint arXiv:1904.07850, 2019
  119. “A comparative analysis of LiDAR SLAM-based indoor navigation for autonomous vehicles” In IEEE Transactions on Intelligent Transportation Systems 23.7 IEEE, 2021, pp. 6907–6921
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Fulong Ma (14 papers)
  2. Weiqing Qi (7 papers)
  3. Guoyang Zhao (11 papers)
  4. Linwei Zheng (10 papers)
  5. Sheng Wang (239 papers)
  6. Ming Liu (421 papers)
  7. Yuxuan Liu (97 papers)
  8. Jun Ma (347 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets