Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CurbNet: Curb Detection Framework Based on LiDAR Point Cloud Segmentation (2403.16794v2)

Published 25 Mar 2024 in cs.CV and cs.RO

Abstract: Curb detection is a crucial function in intelligent driving, essential for determining drivable areas on the road. However, the complexity of road environments makes curb detection challenging. This paper introduces CurbNet, a novel framework for curb detection utilizing point cloud segmentation. To address the lack of comprehensive curb datasets with 3D annotations, we have developed the 3D-Curb dataset based on SemanticKITTI, currently the largest and most diverse collection of curb point clouds. Recognizing that the primary characteristic of curbs is height variation, our approach leverages spatially rich 3D point clouds for training. To tackle the challenges posed by the uneven distribution of curb features on the xy-plane and their dependence on high-frequency features along the z-axis, we introduce the Multi-Scale and Channel Attention (MSCA) module, a customized solution designed to optimize detection performance. Additionally, we propose an adaptive weighted loss function group specifically formulated to counteract the imbalance in the distribution of curb point clouds relative to other categories. Extensive experiments conducted on 2 major datasets demonstrate that our method surpasses existing benchmarks set by leading curb detection and point cloud segmentation models. Through the post-processing refinement of the detection results, we have significantly reduced noise in curb detection, thereby improving precision by 4.5 points. Similarly, our tolerance experiments also achieved state-of-the-art results. Furthermore, real-world experiments and dataset analyses mutually validate each other, reinforcing CurbNet's superior detection capability and robust generalizability. The project website is available at: https://github.com/guoyangzhao/CurbNet/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. T. Luettel, M. Himmelsbach, and H.-J. Wuensche, “Autonomous ground vehicles—concepts and a path to the future,” Proceedings of the IEEE, vol. 100, no. Special Centennial Issue, pp. 1831–1839, 2012.
  2. F. Ma, X. Yan, Y. Liu, and M. Liu, “Every dataset counts: Scaling up monocular 3d object detection with joint datasets training,” arXiv preprint arXiv:2310.00920, 2023.
  3. Z. Xu, Y. Sun, and M. Liu, “icurb: Imitation learning-based detection of road curbs using aerial images for autonomous driving,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1097–1104, 2021.
  4. J. K. Suhr, J. Jang, D. Min, and H. G. Jung, “Sensor fusion-based low-cost vehicle localization system for complex urban environments,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 5, pp. 1078–1086, 2016.
  5. A. Y. Hata, F. T. Ramos, and D. F. Wolf, “Monte carlo localization on gaussian process occupancy maps for urban environments,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 9, pp. 2893–2902, 2017.
  6. L. M. Romero, J. A. Guerrero, and G. Romero, “Road curb detection: A historical survey,” Sensors, vol. 21, no. 21, p. 6952, 2021.
  7. A. B. Hillel, R. Lerner, D. Levi, and G. Raz, “Recent progress in road and lane detection: a survey,” Machine vision and applications, vol. 25, no. 3, pp. 727–745, 2014.
  8. C. Wei, H. Li, J. Shi, G. Zhao, H. Feng, and L. Quan, “Row anchor selection classification method for early-stage crop row-following,” Computers and Electronics in Agriculture, vol. 192, p. 106577, 2022.
  9. P. Sun, X. Zhao, Z. Xu, R. Wang, and H. Min, “A 3d lidar data-based dedicated road boundary detection algorithm for autonomous vehicles,” IEEE Access, vol. 7, pp. 29 623–29 638, 2019.
  10. F. Ma, S. Wang, and M. Liu, “An automatic multi-lidar extrinsic calibration algorithm using corner planes,” in 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO).   IEEE, 2022, pp. 235–240.
  11. F. Ma, Y. Liu, S. Wang, J. Wu, W. Qi, and M. Liu, “Self-supervised drivable area segmentation using lidar’s depth information for autonomous driving,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2023, pp. 41–48.
  12. S. Ö. Demir, T. E. Ertop, A. B. Koku, and E. İ. Konukseven, “An adaptive approach for road boundary detection using 2d lidar sensor,” in 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).   IEEE, 2017, pp. 206–211.
  13. E. Horváth, C. Pozna, and M. Unger, “Real-time lidar-based urban road and sidewalk detection for autonomous vehicles,” Sensors, 2021.
  14. G. Wang, J. Wu, R. He, and B. Tian, “Speed and accuracy tradeoff for lidar data based road boundary detection,” IEEE/CAA Journal of Automatica Sinica, vol. 8, no. 6, pp. 1210–1220, 2020.
  15. T. Chen, B. Dai, D. Liu, J. Song, and Z. Liu, “Velodyne-based curb detection up to 50 meters away,” in 2015 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2015, pp. 241–248.
  16. A. Y. Hata, F. S. Osorio, and D. F. Wolf, “Robust curb detection and vehicle localization in urban environments,” in 2014 IEEE Intelligent Vehicles Symposium Proceedings.   IEEE, 2014, pp. 1257–1262.
  17. L. Zhou and G. Vosselman, “Mapping curbstones in airborne and mobile laser scanning data,” International Journal of Applied Earth Observation and Geoinformation, vol. 18, pp. 293–304, 2012.
  18. S. Xu, R. Wang, and H. Zheng, “Road curb extraction from mobile lidar point clouds,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 2, pp. 996–1009, 2016.
  19. Y. Zhang, J. Wang, X. Wang, and J. M. Dolan, “Road-segmentation-based curb detection method for self-driving via a 3d-lidar sensor,” IEEE transactions on intelligent transportation systems, vol. 19, no. 12, pp. 3981–3991, 2018.
  20. B. Qin, Z. Chong, T. Bandyopadhyay, M. H. Ang, E. Frazzoli, and D. Rus, “Curb-intersection feature based monte carlo localization on urban roads,” in 2012 IEEE International Conference on Robotics and Automation.   IEEE, 2012, pp. 2640–2646.
  21. G. Zhao, L. Quan, H. Li, H. Feng, S. Li, S. Zhang, and R. Liu, “Real-time recognition system of soybean seed full-surface defects based on deep learning,” Computers and Electronics in Agriculture, vol. 187, p. 106230, 2021.
  22. Z. Xu, Y. Sun, L. Wang, and M. Liu, “Cp-loss: Connectivity-preserving loss for road curb detection in autonomous driving with aerial images,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 1117–1123.
  23. Y. Jung, M. Jeon, C. Kim, S.-W. Seo, and S.-W. Kim, “Uncertainty-aware fast curb detection using convolutional networks in point clouds,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 12 882–12 888.
  24. T. Suleymanov, L. Kunze, and P. Newman, “Online inference and detection of curbs in partially occluded scenes with sparse lidar,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC).   IEEE, 2019, pp. 2693–2700.
  25. J. Gao, H. Jie, B. Xu, L. Liu, J. Hu, and W. Liu, “Lcdet: Lidar curb detection network with transformer,” in 2023 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2023, pp. 1–9.
  26. J. L. Apellániz, M. García, N. Aranjuelo, J. Barandiarán, and M. Nieto, “Lidar-based curb detection for ground truth annotation in automated driving validation,” arXiv preprint arXiv:2312.00534, 2023.
  27. H. Zhou, X. Zhu, X. Song, Y. Ma, Z. Wang, H. Li, and D. Lin, “Cylinder3d: An effective 3d framework for driving-scene lidar semantic segmentation,” arXiv preprint arXiv:2008.01550, 2020.
  28. D. Zai, J. Li, Y. Guo, M. Cheng, Y. Lin, H. Luo, and C. Wang, “3-d road boundary extraction from mobile laser scanning data via supervoxels and graph cuts,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 3, pp. 802–813, 2017.
  29. U. Ozertem and D. Erdogmus, “Locally defined principal curves and surfaces,” The Journal of Machine Learning Research, 2011.
  30. H. Jie, J. Gao, Q. Zhao, Z. Ning, J. Hu, L. Liu, and W. Liu, “An efficient curb detection and tracking method for intelligent vehicles via a high-resolution 3d-lidar,” in 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022), vol. 12257.   SPIE, 2022, pp. 310–317.
  31. W. Yao, Z. Deng, and L. Zhou, “Road curb detection using 3d lidar and integral laser points for intelligent vehicles,” in The 6th International Conference on Soft Computing and Intelligent Systems, and The 13th International Symposium on Advanced Intelligence Systems.   IEEE, 2012, pp. 100–105.
  32. A. Y. Hata and D. F. Wolf, “Feature detection for vehicle localization in urban environments using a multilayer lidar,” IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 2, pp. 420–429, 2015.
  33. B. Yang, L. Fang, and J. Li, “Semi-automated extraction and delineation of 3d roads of street scene from mobile laser scanning point clouds,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 79, pp. 80–93, 2013.
  34. D. Bai, T. Cao, J. Guo, and B. Liu, “How to build a curb dataset with lidar data for autonomous driving,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 2576–2582.
  35. J. Liang, N. Homayounfar, W.-C. Ma, S. Wang, and R. Urtasun, “Convolutional recurrent network for road boundary extraction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9512–9521.
  36. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18.   Springer, 2015, pp. 234–241.
  37. X. Pan, J. Shi, P. Luo, X. Wang, and X. Tang, “Spatial as deep: Spatial cnn for traffic scene understanding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018.
  38. D. Kukolj, I. Marinović, and S. Nemet, “Road edge detection based on combined deep learning and spatial statistics of lidar data,” Journal of Spatial Science, vol. 68, no. 2, pp. 245–259, 2023.
  39. H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11 621–11 631.
  40. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The kitti dataset,” The International Journal of Robotics Research, vol. 32, no. 11, pp. 1231–1237, 2013.
  41. J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall, “Semantickitti: A dataset for semantic scene understanding of lidar sequences,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9297–9307.
  42. H. Zhang, H. Zhang, C. Wang, and J. Xie, “Co-occurrent features in semantic segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 548–557.
  43. W. Chen, X. Zhu, R. Sun, J. He, R. Li, X. Shen, and B. Yu, “Tensor low-rank reconstruction for semantic segmentation,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII 16.   Springer, 2020, pp. 52–69.
  44. Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation,” IEEE transactions on medical imaging, vol. 39, no. 6, pp. 1856–1867, 2019.
  45. S. Jadon, “A survey of loss functions for semantic segmentation,” in 2020 IEEE conference on computational intelligence in bioinformatics and computational biology (CIBCB).   IEEE, 2020, pp. 1–7.
  46. E. Schubert, J. Sander, M. Ester, H. P. Kriegel, and X. Xu, “Dbscan revisited, revisited: why and how you should (still) use dbscan,” ACM Transactions on Database Systems (TODS), 2017.
  47. A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 12 697–12 705.
  48. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022.
  49. X. Dong, J. Bao, D. Chen, W. Zhang, N. Yu, L. Yuan, D. Chen, and B. Guo, “Cswin transformer: A general vision transformer backbone with cross-shaped windows,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 124–12 134.
  50. Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3d u-net: learning dense volumetric segmentation from sparse annotation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19.   Springer, 2016, pp. 424–432.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now