Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook (2401.01454v2)

Published 2 Jan 2024 in cs.CV

Abstract: Autonomous driving has rapidly developed and shown promising performance due to recent advances in hardware and deep learning techniques. High-quality datasets are fundamental for developing reliable autonomous driving algorithms. Previous dataset surveys either focused on a limited number or lacked detailed investigation of dataset characteristics. To this end, we present an exhaustive study of 265 autonomous driving datasets from multiple perspectives, including sensor modalities, data size, tasks, and contextual conditions. We introduce a novel metric to evaluate the impact of datasets, which can also be a guide for creating new datasets. Besides, we analyze the annotation processes, existing labeling tools, and the annotation quality of datasets, showing the importance of establishing a standard annotation pipeline. On the other hand, we thoroughly analyze the impact of geographical and adversarial environmental conditions on the performance of autonomous driving systems. Moreover, we exhibit the data distribution of several vital datasets and discuss their pros and cons accordingly. Finally, we discuss the current challenges and the development trend of the future autonomous driving datasets.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (317)
  1. J. Mao, S. Shi, X. Wang, and H. Li, “3d object detection for autonomous driving: A comprehensive survey,” International Journal of Computer Vision, pp. 1–55, 2023.
  2. G. Ros, L. Sellart, J. Materzynska, D. Vazquez, and A. M. Lopez, “The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3234–3243, 2016.
  3. A. Gaidon, Q. Wang, Y. Cabon, and E. Vig, “Virtual worlds as proxy for multi-object tracking analysis,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4340–4349, 2016.
  4. S. R. Richter, V. Vineet, S. Roth, and V. Koltun, “Playing for data: Ground truth from computer games,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pp. 102–118, Springer, 2016.
  5. S. R. Richter, Z. Hayder, and V. Koltun, “Playing for benchmarks,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 2213–2222, 2017.
  6. J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall, “Semantickitti: A dataset for semantic scene understanding of lidar sequences,” in Proceedings of the IEEE/CVF international conference on computer vision, pp. 9297–9307, 2019.
  7. D. Hendrycks, S. Basart, M. Mazeika, A. Zou, J. Kwon, M. Mostajabi, J. Steinhardt, and D. Song, “Scaling out-of-distribution detection for real-world settings,” arXiv preprint arXiv:1911.11132, 2019.
  8. B. Hurl, K. Czarnecki, and S. Waslander, “Precise synthetic image and lidar (presil) dataset for autonomous vehicle perception,” in 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 2522–2529, IEEE, 2019.
  9. Y. Cabon, N. Murray, and M. Humenberger, “Virtual kitti 2,” arXiv preprint arXiv:2001.10773, 2020.
  10. T. Sun, M. Segu, J. Postels, Y. Wang, L. Van Gool, B. Schiele, F. Tombari, and F. Yu, “Shift: a synthetic driving dataset for continuous multi-task domain adaptation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21371–21382, 2022.
  11. R. Xu, H. Xiang, Z. Tu, X. Xia, M.-H. Yang, and J. Ma, “V2x-vit: Vehicle-to-everything cooperative perception with vision transformer,” in European conference on computer vision, pp. 107–124, Springer, 2022.
  12. T. Wang, S. Kim, W. Ji, E. Xie, C. Ge, J. Chen, Z. Li, and P. Luo, “Deepaccident: A motion and accident prediction benchmark for v2x autonomous driving,” arXiv preprint arXiv:2304.01168, 2023.
  13. A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in 2012 IEEE conference on computer vision and pattern recognition, pp. 3354–3361, IEEE, 2012.
  14. M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213–3223, 2016.
  15. Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li, and S. Hu, “Traffic-sign detection and classification in the wild,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2110–2118, 2016.
  16. A. Rasouli, I. Kotseruba, and J. K. Tsotsos, “Are they going to cross? a benchmark dataset and baseline for pedestrian crosswalk behavior,” in Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 206–213, 2017.
  17. X. Huang, X. Cheng, Q. Geng, B. Cao, D. Zhou, P. Wang, Y. Lin, and R. Yang, “The apolloscape dataset for autonomous driving,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 954–960, 2018.
  18. R. Krajewski, J. Bock, L. Kloeker, and L. Eckstein, “The highd dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems,” in 2018 21st international conference on intelligent transportation systems (ITSC), pp. 2118–2125, IEEE, 2018.
  19. M.-F. Chang, J. Lambert, P. Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P. Carr, S. Lucey, D. Ramanan, et al., “Argoverse: 3d tracking and forecasting with rich maps,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8748–8757, 2019.
  20. W. Zhan, L. Sun, D. Wang, H. Shi, A. Clausse, M. Naumann, J. Kummerle, H. Konigshof, C. Stiller, A. de La Fortelle, et al., “Interaction dataset: An international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps,” arXiv preprint arXiv:1910.03088, 2019.
  21. A. Rasouli, I. Kotseruba, T. Kunic, and J. K. Tsotsos, “Pie: A large-scale dataset and models for pedestrian intention estimation and trajectory prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6262–6271, 2019.
  22. H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11621–11631, 2020.
  23. P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine, et al., “Scalability in perception for autonomous driving: Waymo open dataset,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2446–2454, 2020.
  24. F. Yu, H. Chen, X. Wang, W. Xian, Y. Chen, F. Liu, V. Madhavan, and T. Darrell, “Bdd100k: A diverse driving dataset for heterogeneous multitask learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2636–2645, 2020.
  25. J. Geyer, Y. Kassahun, M. Mahmudi, X. Ricou, R. Durgesh, A. S. Chung, L. Hauswald, V. H. Pham, M. Mühlegg, S. Dorn, et al., “A2d2: Audi autonomous driving dataset,” arXiv preprint arXiv:2004.06320, 2020.
  26. M. Sheeny, E. De Pellegrin, S. Mukherjee, A. Ahrabian, S. Wang, and A. Wallace, “Radiate: A radar dataset for automotive perception in bad weather,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–7, IEEE, 2021.
  27. H. Yu, Y. Luo, M. Shu, Y. Huo, Z. Yang, Y. Shi, Z. Guo, H. Li, X. Hu, J. Yuan, et al., “Dair-v2x: A large-scale dataset for vehicle-infrastructure cooperative 3d object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21361–21370, 2022.
  28. B. Wilson, W. Qi, T. Agarwal, J. Lambert, J. Singh, S. Khandelwal, B. Pan, R. Kumar, A. Hartnett, J. K. Pontes, et al., “Argoverse 2: Next generation datasets for self-driving perception and forecasting,” arXiv preprint arXiv:2301.00493, 2023.
  29. J. Mao, M. Niu, C. Jiang, H. Liang, J. Chen, X. Liang, Y. Li, C. Ye, W. Zhang, Z. Li, et al., “One million scenes for autonomous driving: Once dataset,” arXiv preprint arXiv:2106.11037, 2021.
  30. M. Alibeigi, W. Ljungbergh, A. Tonderski, G. Hess, A. Lilja, C. Lindström, D. Motorniuk, J. Fu, J. Widahl, and C. Petersson, “Zenseact open dataset: A large-scale and diverse multimodal dataset for autonomous driving,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20178–20188, 2023.
  31. H. Yin and C. Berger, “When to use what data set for your self-driving car algorithm: An overview of publicly available driving datasets,” in 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8, IEEE, 2017.
  32. Y. Kang, H. Yin, and C. Berger, “Test your self-driving algorithm: An overview of publicly available driving datasets and virtual testing environments,” IEEE Transactions on Intelligent Vehicles, vol. 4, no. 2, pp. 171–185, 2019.
  33. J. Guo, U. Kurup, and M. Shah, “Is it safe to drive? an overview of factors, metrics, and datasets for driveability assessment in autonomous driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 8, pp. 3135–3151, 2019.
  34. J. Janai, F. Güney, A. Behl, A. Geiger, et al., “Computer vision for autonomous vehicles: Problems, datasets and state of the art,” Foundations and Trends® in Computer Graphics and Vision, vol. 12, no. 1–3, pp. 1–308, 2020.
  35. W. Liu, Q. Dong, P. Wang, G. Yang, L. Meng, Y. Song, Y. Shi, and Y. Xue, “A survey on autonomous driving datasets,” in 2021 8th International Conference on Dependable Systems and Their Applications (DSA), pp. 399–407, IEEE, 2021.
  36. H. Li, Y. Li, H. Wang, J. Zeng, P. Cai, H. Xu, D. Lin, J. Yan, F. Xu, L. Xiong, et al., “Open-sourced data ecosystem in autonomous driving: the present and future,” arXiv preprint arXiv:2312.03408, 2023.
  37. D. Bogdoll, F. Schreyer, and J. M. Zöllner, “Ad-datasets: a meta-collection of data sets for autonomous driving,” arXiv preprint arXiv:2202.01909, 2022.
  38. D. Bogdoll, S. Uhlemeyer, K. Kowol, and J. M. Zöllner, “Perception datasets for anomaly detection in autonomous driving: A survey,” arXiv preprint arXiv:2302.02790, 2023.
  39. Z. Song, Z. He, X. Li, Q. Ma, R. Ming, Z. Mao, H. Pei, L. Peng, J. Hu, D. Yao, et al., “Synthetic datasets for autonomous driving: A survey,” arXiv preprint arXiv:2304.12205, 2023.
  40. B. Gao, Y. Pan, C. Li, S. Geng, and H. Zhao, “Are we hungry for 3d lidar data for semantic segmentation? a survey of datasets and methods,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6063–6081, 2021.
  41. Y. Wang, Z. Han, Y. Xing, S. Xu, and J. Wang, “A survey on datasets for decision-making of autonomous vehicle,” arXiv preprint arXiv:2306.16784, 2023.
  42. B. Kitchenham, “Procedures for performing systematic reviews,” Keele, UK, Keele University, vol. 33, no. 2004, pp. 1–26, 2004.
  43. S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving,” Journal of Field Robotics, vol. 37, no. 3, pp. 362–386, 2020.
  44. Y. Li and J. Ibanez-Guzman, “Lidar for autonomous driving: The principles, challenges, and trends for automotive lidar and perception systems,” IEEE Signal Processing Magazine, vol. 37, no. 4, pp. 50–61, 2020.
  45. T. Zhou, M. Yang, K. Jiang, H. Wong, and D. Yang, “Mmw radar-based technologies in autonomous driving: A review,” Sensors, vol. 20, no. 24, p. 7283, 2020.
  46. G. Chen, H. Cao, J. Conradt, H. Tang, F. Rohrbein, and A. Knoll, “Event-based neuromorphic vision for autonomous driving: A paradigm shift for bio-inspired visual sensing and perception,” IEEE Signal Processing Magazine, vol. 37, no. 4, pp. 34–49, 2020.
  47. R. Gade and T. B. Moeslund, “Thermal cameras and applications: a survey,” Machine vision and applications, vol. 25, pp. 245–262, 2014.
  48. E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda, “A survey of autonomous driving: Common practices and emerging technologies,” IEEE access, vol. 8, pp. 58443–58469, 2020.
  49. T. Huang, J. Liu, X. Zhou, D. C. Nguyen, M. R. Azghadi, Y. Xia, Q.-L. Han, and S. Sun, “V2x cooperative perception for autonomous driving: Recent advances and challenges,” arXiv preprint arXiv:2310.03525, 2023.
  50. H. Yu, W. Yang, H. Ruan, Z. Yang, Y. Tang, X. Gao, X. Hao, Y. Shi, Y. Pan, N. Sun, et al., “V2x-seq: A large-scale sequential dataset for vehicle-infrastructure cooperative perception and forecasting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5486–5495, 2023.
  51. D. Wu, W. Han, T. Wang, X. Dong, X. Zhang, and J. Shen, “Referring multi-object tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14633–14642, 2023.
  52. Y. Liao, J. Xie, and A. Geiger, “Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 3, pp. 3292–3310, 2022.
  53. A. Palazzi, D. Abati, F. Solera, R. Cucchiara, et al., “Predicting the driver’s focus of attention: the dr (eye) ve project,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 7, pp. 1720–1733, 2018.
  54. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740–755, Springer, 2014.
  55. A. Garcia-Garcia, S. Orts-Escolano, S. Oprea, V. Villena-Martinez, P. Martinez-Gonzalez, and J. Garcia-Rodriguez, “A survey on deep learning techniques for image and video semantic segmentation,” Applied Soft Computing, vol. 70, pp. 41–65, 2018.
  56. W. Luo, J. Xing, A. Milan, X. Zhang, W. Liu, and T.-K. Kim, “Multiple object tracking: A literature review,” Artificial intelligence, vol. 293, p. 103448, 2021.
  57. S. Guo, S. Wang, Z. Yang, L. Wang, H. Zhang, P. Guo, Y. Gao, and J. Guo, “A review of deep learning-based visual multi-object tracking algorithms for autonomous driving,” Applied Sciences, vol. 12, no. 21, p. 10741, 2022.
  58. Q. Li, Y. Wang, Y. Wang, and H. Zhao, “Hdmapnet: An online hd map construction and evaluation framework,” in 2022 International Conference on Robotics and Automation (ICRA), pp. 4628–4634, IEEE, 2022.
  59. J. Lambert and J. Hays, “Trust, but verify: Cross-modality fusion for hd map change detection,” arXiv preprint arXiv:2212.07312, 2022.
  60. J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 573–580, IEEE, 2012.
  61. Y. Huang, J. Du, Z. Yang, Z. Zhou, L. Zhang, and H. Chen, “A survey on trajectory-prediction methods for autonomous driving,” IEEE Transactions on Intelligent Vehicles, vol. 7, no. 3, pp. 652–674, 2022.
  62. S. Mozaffari, O. Y. Al-Jarrah, M. Dianati, P. Jennings, and A. Mouzakitis, “Deep learning-based vehicle behavior prediction for autonomous driving applications: A review,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 33–47, 2020.
  63. W. Ding, J. Chen, and S. Shen, “Predicting vehicle behaviors over an extended horizon using behavior interaction network,” in 2019 international conference on robotics and automation (ICRA), pp. 8634–8640, IEEE, 2019.
  64. N. Sharma, C. Dhiman, and S. Indu, “Pedestrian intention prediction for autonomous vehicles: A comprehensive survey,” Neurocomputing, 2022.
  65. B. Paden, M. Čáp, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Transactions on intelligent vehicles, vol. 1, no. 1, pp. 33–55, 2016.
  66. A. Tampuu, T. Matiisen, M. Semikin, D. Fishman, and N. Muhammad, “A survey of end-to-end driving: Architectures and training methods,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 4, pp. 1364–1384, 2020.
  67. L. Chen, P. Wu, K. Chitta, B. Jaeger, A. Geiger, and H. Li, “End-to-end autonomous driving: Challenges and frontiers,” arXiv preprint arXiv:2306.16927, 2023.
  68. X. Pan, J. Shi, P. Luo, X. Wang, and X. Tang, “Spatial as deep: Spatial cnn for traffic scene understanding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, 2018.
  69. S. Zhang, R. Benenson, and B. Schiele, “Citypersons: A diverse dataset for pedestrian detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213–3221, 2017.
  70. G. J. Brostow, J. Fauqueur, and R. Cipolla, “Semantic object classes in video: A high-definition ground truth database,” Pattern Recognition Letters, vol. 30, no. 2, pp. 88–97, 2009.
  71. G. Varma, A. Subramanian, A. Namboodiri, M. Chandraker, and C. Jawahar, “Idd: A dataset for exploring problems of autonomous navigation in unconstrained environments,” in 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1743–1751, IEEE, 2019.
  72. C. Sakaridis, D. Dai, and L. Van Gool, “Semantic foggy scene understanding with synthetic data,” International Journal of Computer Vision, vol. 126, pp. 973–992, 2018.
  73. S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, “Detection of traffic signs in real-world images: The german traffic sign detection benchmark,” in The 2013 international joint conference on neural networks (IJCNN), pp. 1–8, Ieee, 2013.
  74. P. Dollár, C. Wojek, B. Schiele, and P. Perona, “Pedestrian detection: A benchmark,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 304–311, IEEE, 2009.
  75. M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, and F. Heide, “Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11682–11692, 2020.
  76. C. Sakaridis, D. Dai, and L. Van Gool, “Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10765–10775, 2021.
  77. S. Lee, J. Kim, J. Shin Yoon, S. Shin, O. Bailo, N. Kim, T.-H. Lee, H. Seok Hong, S.-H. Han, and I. So Kweon, “Vpgnet: Vanishing point guided network for lane and road marking detection and recognition,” in Proceedings of the IEEE international conference on computer vision, pp. 1947–1955, 2017.
  78. T.-H. Wang, S. Manivasagam, M. Liang, B. Yang, W. Zeng, and R. Urtasun, “V2vnet: Vehicle-to-vehicle communication for joint perception and prediction,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pp. 605–621, Springer, 2020.
  79. Q. Chen, S. Tang, Q. Yang, and S. Fu, “Cooper: Cooperative perception for connected autonomous vehicles based on 3d point clouds,” in 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp. 514–524, IEEE, 2019.
  80. X. Ye, M. Shu, H. Li, Y. Shi, Y. Li, G. Wang, X. Tan, and E. Ding, “Rope3d: The roadside perception dataset for autonomous driving and monocular 3d object detection task,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21341–21350, 2022.
  81. Y. Li, D. Ma, Z. An, Z. Wang, Y. Zhong, S. Chen, and C. Feng, “V2x-sim: Multi-agent collaborative perception dataset and benchmark for autonomous driving,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10914–10921, 2022.
  82. R. Xu, X. Xia, J. Li, H. Li, S. Zhang, Z. Tu, Z. Meng, H. Xiang, X. Dong, R. Song, et al., “V2v4real: A real-world large-scale dataset for vehicle-to-vehicle cooperative perception,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13712–13722, 2023.
  83. E. Arnold, M. Dianati, R. de Temple, and S. Fallah, “Cooperative perception for 3d object detection in driving scenarios using infrastructure sensors,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 3, pp. 1852–1864, 2020.
  84. C. Creß, W. Zimmer, L. Strand, M. Fortkord, S. Dai, V. Lakshminarasimhan, and A. Knoll, “A9-dataset: Multi-sensor infrastructure-based dataset for mobility research,” in 2022 IEEE Intelligent Vehicles Symposium (IV), pp. 965–970, IEEE, 2022.
  85. S. Busch, C. Koetsier, J. Axmann, and C. Brenner, “Lumpi: The leibniz university multi-perspective intersection dataset,” in 2022 IEEE Intelligent Vehicles Symposium (IV), pp. 1127–1134, IEEE, 2022.
  86. R. Mao, J. Guo, Y. Jia, Y. Sun, S. Zhou, and Z. Niu, “Dolphins: Dataset for collaborative perception enabled harmonious and interconnected self-driving,” in Proceedings of the Asian Conference on Computer Vision, pp. 4361–4377, 2022.
  87. D. Du, Y. Qi, H. Yu, Y. Yang, K. Duan, G. Li, W. Zhang, Q. Huang, and Q. Tian, “The unmanned aerial vehicle benchmark: Object detection and tracking,” in Proceedings of the European conference on computer vision (ECCV), pp. 370–386, 2018.
  88. Y. Sun, B. Cao, P. Zhu, and Q. Hu, “Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 10, pp. 6700–6713, 2022.
  89. G. Neuhold, T. Ollmann, S. Rota Bulo, and P. Kontschieder, “The mapillary vistas dataset for semantic understanding of street scenes,” in Proceedings of the IEEE international conference on computer vision, pp. 4990–4999, 2017.
  90. Y. Xiang, R. Mottaghi, and S. Savarese, “Beyond pascal: A benchmark for 3d object detection in the wild,” in IEEE winter conference on applications of computer vision, pp. 75–82, IEEE, 2014.
  91. O. Zendel, K. Honauer, M. Murschitz, D. Steininger, and G. F. Dominguez, “Wilddash-creating hazard-aware benchmarks,” in Proceedings of the European Conference on Computer Vision (ECCV), pp. 402–416, 2018.
  92. S. Wang, M. Bai, G. Mattyus, H. Chu, W. Luo, B. Yang, J. Liang, J. Cheverie, S. Fidler, and R. Urtasun, “Torontocity: Seeing the world with a million eyes,” arXiv preprint arXiv:1612.00423, 2016.
  93. M. A. Kenk and M. Hassaballah, “Dawn: vehicle detection in adverse weather nature dataset,” arXiv preprint arXiv:2008.05402, 2020.
  94. K. Lis, K. Nakka, P. Fua, and M. Salzmann, “Detecting the unexpected via image resynthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2152–2161, 2019.
  95. P. Cong, X. Zhu, F. Qiao, Y. Ren, X. Peng, Y. Hou, L. Xu, R. Yang, D. Manocha, and Y. Ma, “Stcrowd: A multimodal dataset for pedestrian perception in crowded scenes. 2022 ieee,” in CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19576–19585, 2022.
  96. S. Manivasagam, S. Wang, K. Wong, W. Zeng, M. Sazanovich, S. Tan, B. Yang, W.-C. Ma, and R. Urtasun, “Lidarsim: Realistic lidar simulation by leveraging the real world. 2020 ieee,” in CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11164–11173, 2020.
  97. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision, vol. 88, pp. 303–338, 2010.
  98. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255, Ieee, 2009.
  99. A. Zyner, S. Worrall, and E. Nebot, “Naturalistic driver intention and path prediction using recurrent neural networks,” IEEE transactions on intelligent transportation systems, vol. 21, no. 4, pp. 1584–1594, 2019.
  100. J. Bock, R. Krajewski, T. Moers, S. Runde, L. Vater, and L. Eckstein, “The ind dataset: A drone dataset of naturalistic road user trajectories at german intersections,” in 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1929–1934, IEEE, 2020.
  101. A. Rasouli, T. Yau, P. Lakner, S. Malekmohammadi, M. Rohani, and J. Luo, “Pepscenes: A novel dataset and baseline for pedestrian action prediction in 3d,” arXiv preprint arXiv:2012.07773, 2020.
  102. A. Breuer, J.-A. Termöhlen, S. Homoceanu, and T. Fingscheidt, “opendd: A large-scale roundabout drone dataset,” in 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6, IEEE, 2020.
  103. H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari, “nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles,” arXiv preprint arXiv:2106.11810, 2021.
  104. T. Moers, L. Vater, R. Krajewski, J. Bock, A. Zlocki, and L. Eckstein, “The exid dataset: A real-world trajectory dataset of highly interactive highway scenarios in germany,” in 2022 IEEE Intelligent Vehicles Symposium (IV), pp. 958–964, IEEE, 2022.
  105. L. Gressenbuch, K. Esterle, T. Kessler, and M. Althoff, “Mona: The munich motion dataset of natural driving,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pp. 2093–2100, IEEE, 2022.
  106. J. Xue, J. Fang, T. Li, B. Zhang, P. Zhang, Z. Ye, and J. Dou, “Blvd: Building a large-scale 5d semantics benchmark for autonomous driving,” in 2019 International Conference on Robotics and Automation (ICRA), pp. 6685–6691, IEEE, 2019.
  107. R. Krajewski, T. Moers, J. Bock, L. Vater, and L. Eckstein, “The round dataset: A drone dataset of road user trajectories at roundabouts in germany,” in 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6, IEEE, 2020.
  108. J. Houston, G. Zuidhof, L. Bergamini, Y. Ye, L. Chen, A. Jain, S. Omari, V. Iglovikov, and P. Ondruska, “One thousand and one hours: Self-driving motion prediction dataset,” in Conference on Robot Learning, pp. 409–418, PMLR, 2021.
  109. H. Girase, H. Gang, S. Malla, J. Li, A. Kanehara, K. Mangalam, and C. Choi, “Loki: Long term and key intentions for trajectory prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9803–9812, 2021.
  110. A. Prabu, N. Ranjan, L. Li, R. Tian, S. Chien, Y. Chen, and R. Sherony, “Scendd: A scenario-based naturalistic driving dataset,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pp. 4363–4368, IEEE, 2022.
  111. V. Dewangan, T. Choudhary, S. Chandhok, S. Priyadarshan, A. Jain, A. K. Singh, S. Srivastava, K. M. Jatavallabhula, and K. M. Krishna, “Talk2bev: Language-enhanced bird’s-eye view maps for autonomous driving,” arXiv preprint arXiv:2310.02251, 2023.
  112. A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Conference on robot learning, pp. 1–16, PMLR, 2017.
  113. J. Binas, D. Neil, S.-C. Liu, and T. Delbruck, “Ddd17: End-to-end davis driving dataset,” arXiv preprint arXiv:1711.01458, 2017.
  114. S. Hwang, J. Park, N. Kim, Y. Choi, and I. So Kweon, “Multispectral pedestrian detection: Benchmark dataset and baseline,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1037–1045, 2015.
  115. R. Timofte, K. Zimmermann, and L. Van Gool, “Multi-view traffic sign detection, recognition, and 3d localisation,” Machine vision and applications, vol. 25, pp. 633–647, 2014.
  116. D. Dai and L. Van Gool, “Dark model adaptation: Semantic image segmentation from daytime to nighttime,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 3819–3824, IEEE, 2018.
  117. Z. Che, G. Li, T. Li, B. Jiang, X. Shi, X. Zhang, Y. Lu, G. Wu, Y. Liu, and J. Ye, “D22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT-city: a large-scale dashcam video dataset of diverse traffic scenarios,” arXiv preprint arXiv:1904.01975, 2019.
  118. M. Aly, “Real time detection of lane markers in urban streets,” in 2008 IEEE intelligent vehicles symposium, pp. 7–12, IEEE, 2008.
  119. Q.-H. Pham, P. Sevestre, R. S. Pahwa, H. Zhan, C. H. Pang, Y. Chen, A. Mustafa, V. Chandrasekhar, and J. Lin, “A 3d dataset: Towards autonomous driving in challenging environments,” in 2020 IEEE International conference on Robotics and Automation (ICRA), pp. 2267–2273, IEEE, 2020.
  120. A. Patil, S. Malla, H. Gang, and Y.-T. Chen, “The h3d dataset for full-surround 3d multi-object detection and tracking in crowded urban scenes,” in 2019 International Conference on Robotics and Automation (ICRA), pp. 9552–9557, IEEE, 2019.
  121. G. Singh, S. Akrigg, M. Di Maio, V. Fontana, R. J. Alitappeh, S. Khan, S. Saha, K. Jeddisaravi, F. Yousefi, J. Culley, et al., “Road: The road event awareness dataset for autonomous driving,” IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 1, pp. 1036–1054, 2022.
  122. A. Ess, B. Leibe, and L. Van Gool, “Depth and appearance for mobile scene analysis,” in 2007 IEEE 11th international conference on computer vision, pp. 1–8, IEEE, 2007.
  123. M. Pitropov, D. E. Garcia, J. Rebello, M. Smart, C. Wang, K. Czarnecki, and S. Waslander, “Canadian adverse driving conditions dataset,” The International Journal of Robotics Research, vol. 40, no. 4-5, pp. 681–690, 2021.
  124. O. Schumann, M. Hahn, N. Scheiner, F. Weishaupt, J. F. Tilly, J. Dickmann, and C. Wöhler, “Radarscenes: A real-world radar point cloud data set for automotive applications,” in 2021 IEEE 24th International Conference on Information Fusion (FUSION), pp. 1–8, IEEE, 2021.
  125. X. Weng, Y. Man, J. Park, Y. Yuan, M. O’Toole, and K. M. Kitani, “All-in-one drive: A comprehensive perception dataset with high-density long-range point clouds,” 2023.
  126. D. Temel, G. Kwon, M. Prabhushankar, and G. AlRegib, “Cure-tsr: Challenging unreal and real environments for traffic sign recognition,” arXiv preprint arXiv:1712.02463, 2017.
  127. X. Roynard, J.-E. Deschaud, and F. Goulette, “Paris-lille-3d: A large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification,” The International Journal of Robotics Research, vol. 37, no. 6, pp. 545–557, 2018.
  128. P. Xiao, Z. Shao, S. Hao, Z. Zhang, X. Chai, J. Jiao, Z. Li, J. Wu, K. Sun, K. Jiang, et al., “Pandaset: Advanced sensor suite dataset for autonomous driving,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 3095–3101, IEEE, 2021.
  129. L. Neumann, M. Karg, S. Zhang, C. Scharfenberger, E. Piegert, S. Mistr, O. Prokofyeva, R. Thiel, A. Vedaldi, A. Zisserman, et al., “Nightowls: A pedestrians at night dataset,” in Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part I 14, pp. 691–705, Springer, 2019.
  130. A. R. Sekkat, Y. Dupuis, V. R. Kumar, H. Rashed, S. Yogamani, P. Vasseur, and P. Honeine, “Synwoodscape: Synthetic surround-view fisheye camera dataset for autonomous driving,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8502–8509, 2022.
  131. K. Burnett, D. J. Yoon, Y. Wu, A. Z. Li, H. Zhang, S. Lu, J. Qian, W.-K. Tseng, A. Lambert, K. Y. Leung, et al., “Boreas: A multi-season autonomous driving dataset,” The International Journal of Robotics Research, vol. 42, no. 1-2, pp. 33–42, 2023.
  132. C. Wojek, S. Walk, and B. Schiele, “Multi-cue onboard pedestrian detection,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 794–801, IEEE, 2009.
  133. F. S. Saleh, M. S. Aliakbarian, M. Salzmann, L. Petersson, and J. M. Alvarez, “Effective use of synthetic data for urban scene semantic segmentation,” in Proceedings of the European Conference on Computer Vision (ECCV), pp. 84–100, 2018.
  134. J. Zhang, X. Zou, L.-D. Kuang, J. Wang, R. S. Sherratt, and X. Yu, “Cctsdb 2021: a more comprehensive traffic sign detection benchmark,” Human-centric Computing and Information Sciences, vol. 12, 2022.
  135. Y. Pan, B. Gao, J. Mei, S. Geng, C. Li, and H. Zhao, “Semanticposs: A point cloud dataset with large quantity of dynamic instances,” in 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 687–693, IEEE, 2020.
  136. E. Alberti, A. Tavera, C. Masone, and B. Caputo, “Idda: A large-scale multi-domain dataset for autonomous driving,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 5526–5533, 2020.
  137. L. Zheng, Z. Ma, X. Zhu, B. Tan, S. Li, K. Long, W. Sun, S. Chen, L. Zhang, M. Wan, et al., “Tj4dradset: A 4d radar dataset for autonomous driving,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pp. 493–498, IEEE, 2022.
  138. C. Caraffi, T. Vojíř, J. Trefnỳ, J. Šochman, and J. Matas, “A system for real-time detection and tracking of vehicles from a single car-mounted camera,” in 2012 15th international IEEE conference on intelligent transportation systems, pp. 975–982, IEEE, 2012.
  139. A. Teichman, J. Levinson, and S. Thrun, “Towards 3d object recognition via classification of arbitrary object tracks,” in 2011 IEEE International Conference on Robotics and Automation, pp. 4034–4041, IEEE, 2011.
  140. A. Ouaknine, A. Newson, J. Rebut, F. Tupin, and P. Pérez, “Carrada dataset: Camera and automotive radar with range-angle-doppler annotations,” in 2020 25th International Conference on Pattern Recognition (ICPR), pp. 5068–5075, IEEE, 2021.
  141. J. Han, X. Liang, H. Xu, K. Chen, L. Hong, J. Mao, C. Ye, W. Zhang, Z. Li, X. Liang, et al., “Soda10m: a large-scale 2d self/semi-supervised object detection dataset for autonomous driving,” arXiv preprint arXiv:2106.11118, 2021.
  142. L. Kong, Y. Liu, X. Li, R. Chen, W. Zhang, J. Ren, L. Pan, K. Chen, and Z. Liu, “Robo3d: Towards robust and reliable 3d perception against corruptions,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19994–20006, 2023.
  143. P. Pinggera, S. Ramos, S. Gehrig, U. Franke, C. Rother, and R. Mester, “Lost and found: detecting small road hazards for self-driving vehicles. in 2016 ieee,” in RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1099–1106.
  144. S. Malla, B. Dariush, and C. Choi, “Titan: Future forecast using action priors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11186–11196, 2020.
  145. K. Li, K. Chen, H. Wang, L. Hong, C. Ye, J. Han, Y. Chen, W. Zhang, C. Xu, D.-Y. Yeung, et al., “Coda: A real-world road corner case dataset for object detection in autonomous driving,” in European Conference on Computer Vision, pp. 406–423, Springer, 2022.
  146. J.-L. Déziel, P. Merriaux, F. Tremblay, D. Lessard, D. Plourde, J. Stanguennec, P. Goulet, and P. Olivier, “Pixset: An opportunity for 3d computer vision to go beyond point clouds with a full-waveform lidar dataset,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 2987–2993, IEEE, 2021.
  147. D.-H. Paek, S.-H. Kong, and K. T. Wijaya, “K-radar: 4d radar object detection for autonomous driving in various weather conditions,” Advances in Neural Information Processing Systems, vol. 35, pp. 3819–3829, 2022.
  148. R. Chan, K. Lis, S. Uhlemeyer, H. Blum, S. Honari, R. Siegwart, P. Fua, M. Salzmann, and M. Rottmann, “Segmentmeifyoucan: A benchmark for anomaly segmentation,” arXiv preprint arXiv:2104.14812, 2021.
  149. Y. Zhang, L. Zhu, W. Feng, H. Fu, M. Wang, Q. Li, C. Li, and S. Wang, “Vil-100: A new dataset and a baseline model for video instance lane detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15681–15690, 2021.
  150. M. Braun, S. Krebs, F. Flohr, and D. Gavrila, “The eurocity persons dataset: A novel benchmark for object detection. arxiv 2018,” arXiv preprint arXiv:1805.07193.
  151. “Tusimple: https://github.com/tusimple/tusimple-benchmark,”
  152. X. Zhang, Z. Li, Y. Gong, D. Jin, J. Li, L. Wang, Y. Zhu, and H. Liu, “Openmpd: An open multimodal perception dataset for autonomous driving,” IEEE Transactions on Vehicular Technology, vol. 71, no. 3, pp. 2437–2447, 2022.
  153. A. Kurup and J. Bos, “Dsor: A scalable statistical filter for removing falling snow from lidar point clouds in severe winter weather,” arXiv preprint arXiv:2109.07078, 2021.
  154. Z. Xie, S. Wang, K. Xu, Z. Zhang, X. Tan, Y. Xie, and L. Ma, “Boosting night-time scene parsing with learnable frequency,” IEEE Transactions on Image Processing, 2023.
  155. M. Hahner, C. Sakaridis, M. Bijelic, F. Heide, F. Yu, D. Dai, and L. Van Gool, “Lidar snowfall simulation for robust 3d object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16364–16374, 2022.
  156. K. Behrendt and R. Soussan, “Unsupervised labeled lane markers using maps,” in Proceedings of the IEEE/CVF international conference on computer vision workshops, pp. 0–0, 2019.
  157. H. Blum, P.-E. Sarlin, J. Nieto, R. Siegwart, and C. Cadena, “The fishyscapes benchmark: Measuring blind spots in semantic segmentation,” International Journal of Computer Vision, vol. 129, pp. 3119–3135, 2021.
  158. W. Tan, N. Qin, L. Ma, Y. Li, J. Du, G. Cai, K. Yang, and J. Li, “Toronto-3d: A large-scale mobile lidar dataset for semantic segmentation of urban roadways,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 202–203, 2020.
  159. L. Ding, M. Glazer, M. Wang, B. Mehler, B. Reimer, and L. Fridman, “Mit-avt clustered driving scene dataset: Evaluating perception systems in real-world naturalistic driving scenarios,” in 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 232–237, IEEE, 2020.
  160. A. Møgelmose, D. Liu, and M. M. Trivedi, “Traffic sign detection for us roads: Remaining challenges and a case for tracking,” in 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 1394–1399, IEEE, 2014.
  161. D. Griffiths and J. Boehm, “Synthcity: A large scale synthetic point cloud,” arXiv preprint arXiv:1907.04758, 2019.
  162. P. Jiang and S. Saripalli, “Lidarnet: A boundary-aware domain adaptation model for point cloud semantic segmentation,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 2457–2464, IEEE, 2021.
  163. G. Choe, S.-H. Kim, S. Im, J.-Y. Lee, S. G. Narasimhan, and I. S. Kweon, “Ranus: Rgb and nir urban scene dataset for deep scene parsing,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1808–1815, 2018.
  164. G. Franchi, X. Yu, A. Bursuc, A. Tena, R. Kazmierczak, S. Dubuisson, E. Aldea, and D. Filliat, “Muad: Multiple uncertainties for autonomous driving, a benchmark for multiple uncertainty types and tasks,” arXiv preprint arXiv:2203.01437, 2022.
  165. J. Cui, H. Qiu, D. Chen, P. Stone, and Y. Zhu, “Coopernaut: End-to-end driving with cooperative perception for networked vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17252–17262, 2022.
  166. Y. Dong, Y. Zhong, W. Yu, M. Zhu, P. Lu, Y. Fang, J. Hong, and H. Peng, “Mcity data collection for automated vehicles study,” arXiv preprint arXiv:1912.06258, 2019.
  167. H. Schafer, E. Santana, A. Haden, and R. Biasini, “A commute in data: The comma2k19 dataset,” arXiv preprint arXiv:1812.05752, 2018.
  168. W. Cheng, H. Luo, W. Yang, L. Yu, S. Chen, and W. Li, “Det: A high-resolution dvs dataset for lane extraction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0–0, 2019.
  169. X. Liu, Z. Deng, H. Lu, and L. Cao, “Benchmark for road marking detection: Dataset specification and performance baseline,” in 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6, IEEE, 2017.
  170. Z. Wang, S. Ding, Y. Li, J. Fenn, S. Roychowdhury, A. Wallin, L. Martin, S. Ryvola, G. Sapiro, and Q. Qiu, “Cirrus: A long-range bi-pattern lidar dataset,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 5744–5750, IEEE, 2021.
  171. E. Mohamed, M. Ewaisha, M. Siam, H. Rashed, S. Yogamani, W. Hamdy, M. El-Dakdouky, and A. El-Sallab, “Monocular instance motion segmentation for autonomous driving: Kitti instancemotseg dataset and multi-task baseline,” in 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 114–121, IEEE, 2021.
  172. Z. Bai, G. Wu, M. J. Barth, Y. Liu, E. A. Sisbot, and K. Oguchi, “Pillargrid: Deep learning-based cooperative perception for 3d object detection from onboard-roadside lidar,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pp. 1743–1749, IEEE, 2022.
  173. N. A. M. Mai, P. Duthon, L. Khoudour, A. Crouzil, and S. A. Velastin, “3d object detection with sls-fusion network in foggy weather conditions,” Sensors, vol. 21, no. 20, p. 6711, 2021.
  174. S. Nag, S. Adak, and S. Das, “What’s there in the dark,” in 2019 IEEE International Conference on Image Processing (ICIP), pp. 2996–3000, IEEE, 2019.
  175. O. Jayasinghe, S. Hemachandra, D. Anhettigama, S. Kariyawasam, R. Rodrigo, and P. Jayasekara, “Ceymo: see more on roads-a novel benchmark dataset for road marking detection,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3104–3113, 2022.
  176. J. Jin, A. Fatemi, W. M. P. Lira, F. Yu, B. Leng, R. Ma, A. Mahdavi-Amiri, and H. Zhang, “Raidar: A rich annotated image dataset of rainy street scenes,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2951–2961, 2021.
  177. D.-H. Paek, S.-H. Kong, and K. T. Wijaya, “K-lane: Lidar lane dataset and benchmark for urban roads and highways,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4450–4459, 2022.
  178. T. Matuszka, I. Barton, Á. Butykai, P. Hajas, D. Kiss, D. Kovács, S. Kunsági-Máté, P. Lengyel, G. Németh, L. Pető, et al., “aimotive dataset: A multimodal dataset for robust autonomous driving with long-range perception,” arXiv preprint arXiv:2211.09445, 2022.
  179. “Sap: https://cs.stanford.edu/ anenberg/ua_data/,”
  180. M. Meyer and G. Kuschk, “Automotive radar dataset for deep learning based 3d object detection,” in 2019 16th european radar conference (EuRAD), pp. 129–132, IEEE, 2019.
  181. Y. Yuan and M. Sester, “Comap: A synthetic dataset for collective multi-agent perception of autonomous driving,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 43, pp. 255–263, 2021.
  182. C. A. Diaz-Ruiz, Y. Xia, Y. You, J. Nino, J. Chen, J. Monica, X. Chen, K. Luo, Y. Wang, M. Emond, et al., “Ithaca365: Dataset and driving perception under repeated and challenging weather conditions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21383–21392, 2022.
  183. A. Singh, A. Kamireddypalli, V. Gandhi, and K. M. Krishna, “Lidar guided small obstacle segmentation,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8513–8520, IEEE, 2020.
  184. N. Gray, M. Moraes, J. Bian, A. Wang, A. Tian, K. Wilson, Y. Huang, H. Xiong, and Z. Guo, “Glare: A dataset for traffic sign detection in sun glare,” IEEE Transactions on Intelligent Transportation Systems, 2023.
  185. M. Howe, I. Reid, and J. Mackenzie, “Weakly supervised training of monocular 3d object detectors using wide baseline multi-view traffic camera data,” arXiv preprint arXiv:2110.10966, 2021.
  186. L. Ding, J. Terwilliger, R. Sherony, B. Reimer, and L. Fridman, “Value of temporal dynamics information in driving scene segmentation,” IEEE Transactions on Intelligent Vehicles, vol. 7, no. 1, pp. 113–122, 2021.
  187. J. Hou, Q. Chen, Y. Cheng, G. Chen, X. Xue, T. Zeng, and J. Pu, “Sups: A simulated underground parking scenario dataset for autonomous driving,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pp. 2265–2271, IEEE, 2022.
  188. J. Oh, G. Lee, J. Park, W. Oh, J. Heo, H. Chung, D. H. Kim, B. Park, C.-G. Lee, S. Choi, et al., “Towards defensive autonomous driving: Collecting and probing driving demonstrations of mixed qualities,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 12528–12533, IEEE, 2022.
  189. H. Sakashita, C. Flothow, N. Takemura, and Y. Sugano, “Driv100: In-the-wild multi-domain dataset and evaluation for real-world domain adaptation of semantic segmentation,” arXiv preprint arXiv:2102.00150, 2021.
  190. L. Wang, L. Lei, H. Song, and W. Wang, “The neolix open dataset for autonomous driving,” arXiv preprint arXiv:2011.13528, 2020.
  191. J. Gebele, B. Stuhr, and J. Haselberger, “Carlane: A lane detection benchmark for unsupervised domain adaptation from simulation to multiple real-world domains,” arXiv preprint arXiv:2206.08083, 2022.
  192. H. Wang, X. Zhang, J. Li, Z. Li, L. Yang, S. Pan, and Y. Deng, “Ips300+: a challenging multimodal dataset for intersection perception system,” arXiv preprint arXiv:2106.02781, 2021.
  193. W. Zimmer, C. Creß, H. T. Nguyen, and A. C. Knoll, “A9 intersection dataset: All you need for urban 3d camera-lidar roadside perception,” arXiv preprint arXiv:2306.09266, 2023.
  194. J. Breitenstein and T. Fingscheidt, “Amodal cityscapes: a new dataset, its generation, and an amodal semantic segmentation challenge baseline,” in 2022 IEEE Intelligent Vehicles Symposium (IV), pp. 1018–1025, IEEE, 2022.
  195. A. Marathe, D. Ramanan, R. Walambe, and K. Kotecha, “Wedge: A multi-weather autonomous driving dataset built from generative vision-language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3317–3326, 2023.
  196. Y. Yao, M. Xu, C. Choi, D. J. Crandall, E. M. Atkins, and B. Dariush, “Egocentric vision-based future vehicle localization for intelligent driving assistance systems,” in 2019 International Conference on Robotics and Automation (ICRA), pp. 9711–9717, IEEE, 2019.
  197. T. Sattler, W. Maddern, C. Toft, A. Torii, L. Hammarstrand, E. Stenborg, D. Safari, M. Okutomi, M. Pollefeys, J. Sivic, et al., “Benchmarking 6dof outdoor visual localization in changing conditions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8601–8610, 2018.
  198. J. Kim, T. Misu, Y.-T. Chen, A. Tawari, and J. Canny, “Grounding human-to-vehicle advice for self-driving vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10591–10599, 2019.
  199. F. Codevilla, E. Santana, A. M. López, and A. Gaidon, “Exploring the limitations of behavior cloning for autonomous driving,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9329–9338, 2019.
  200. W. Choi, K. Shahid, and S. Savarese, “What are they doing?: Collective activity classification using spatio-temporal relationship among people,” in 2009 IEEE 12th international conference on computer vision workshops, ICCV Workshops, pp. 1282–1289, IEEE, 2009.
  201. Y. Zhou, G. Wan, S. Hou, L. Yu, G. Wang, X. Rui, and S. Song, “Da4ad: End-to-end deep attention-based visual localization for autonomous driving,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVIII 16, pp. 271–289, Springer, 2020.
  202. K. Behrendt, L. Novak, and R. Botros, “A deep learning approach to traffic lights: Detection, tracking, and classification,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1370–1377, IEEE, 2017.
  203. M. Büchner, J. Zürn, I.-G. Todoran, A. Valada, and W. Burgard, “Learning and aggregating lane graphs for urban automated driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13415–13424, 2023.
  204. A. Lehner, S. Gasperini, A. Marcos-Ramiro, M. Schmidt, M.-A. N. Mahani, N. Navab, B. Busam, and F. Tombari, “3d-vfield: Adversarial augmentation of point clouds for domain generalization in 3d object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17295–17304, 2022.
  205. E. Arnold, S. Mozaffari, and M. Dianati, “Fast and robust registration of partially overlapping point clouds,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1502–1509, 2021.
  206. A. Kloukiniotis, A. Papandreou, C. Anagnostopoulos, A. Lalos, P. Kapsalas, D.-V. Nguyen, and K. Moustakas, “Carlascenes: A synthetic dataset for odometry in autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4520–4528, 2022.
  207. N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A. Dosovitskiy, and T. Brox, “A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4040–4048, 2016.
  208. D. M. Chen, G. Baatz, K. Köser, S. S. Tsai, R. Vedantham, T. Pylvänäinen, K. Roimela, X. Chen, J. Bach, M. Pollefeys, et al., “City-scale landmark identification on mobile devices,” in CVPR 2011, pp. 737–744, IEEE, 2011.
  209. V. Guizilini, R. Ambrus, S. Pillai, A. Raventos, and A. Gaidon, “3d packing for self-supervised monocular depth estimation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2485–2494, 2020.
  210. L. Wen, D. Du, Z. Cai, Z. Lei, M.-C. Chang, H. Qi, J. Lim, M.-H. Yang, and S. Lyu, “Ua-detrac: A new benchmark and protocol for multi-object detection and tracking,” Computer Vision and Image Understanding, vol. 193, p. 102907, 2020.
  211. N. Carlevaris-Bianco, A. K. Ushani, and R. M. Eustice, “University of michigan north campus long-term vision and lidar dataset,” The International Journal of Robotics Research, vol. 35, no. 9, pp. 1023–1035, 2016.
  212. A. Z. Zhu, D. Thakur, T. Özaslan, B. Pfrommer, V. Kumar, and K. Daniilidis, “The multivehicle stereo event camera dataset: An event camera dataset for 3d perception,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2032–2039, 2018.
  213. J.-L. Blanco-Claraco, F.-A. Moreno-Duenas, and J. González-Jiménez, “The málaga urban dataset: High-rate stereo and lidar in a realistic urban scenario,” The International Journal of Robotics Research, vol. 33, no. 2, pp. 207–214, 2014.
  214. D. Barnes, M. Gadd, P. Murcutt, P. Newman, and I. Posner, “The oxford radar robotcar dataset: A radar extension to the oxford robotcar dataset,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 6433–6438, IEEE, 2020.
  215. Y. Lou, Y. Bai, J. Liu, S. Wang, and L. Duan, “Veri-wild: A large dataset and a new method for vehicle re-identification in the wild,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3235–3243, 2019.
  216. R. Guerrero-Gómez-Olmedo, B. Torre-Jiménez, R. López-Sastre, S. Maldonado-Bascón, and D. Onoro-Rubio, “Extremely overlapping vehicle counting,” in Pattern Recognition and Image Analysis: 7th Iberian Conference, IbPRIA 2015, Santiago de Compostela, Spain, June 17-19, 2015, Proceedings 7, pp. 423–431, Springer, 2015.
  217. J. Jeong, Y. Cho, Y.-S. Shin, H. Roh, and A. Kim, “Complex urban dataset with multi-level sensors from highly diverse urban environments,” The International Journal of Robotics Research, vol. 38, no. 6, pp. 642–657, 2019.
  218. M. Wrenninge and J. Unger, “Synscapes: A photorealistic synthetic dataset for street scene parsing,” arXiv preprint arXiv:1810.08705, 2018.
  219. M. De Deuge, A. Quadros, C. Hung, and B. Douillard, “Unsupervised feature learning for classification of outdoor 3d scans,” in Australasian conference on robitics and automation, vol. 2, University of New South Wales Kensington, Australia, 2013.
  220. X. Song, P. Wang, D. Zhou, R. Zhu, C. Guan, Y. Dai, H. Su, H. Li, and R. Yang, “Apollocar3d: A large 3d car instance understanding benchmark for autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5452–5462, 2019.
  221. G. Kim, Y. S. Park, Y. Cho, J. Jeong, and A. Kim, “Mulran: Multimodal range dataset for urban place recognition,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 6246–6253, IEEE, 2020.
  222. A. Serna, B. Marcotegui, F. Goulette, and J.-E. Deschaud, “Paris-rue-madame database: a 3d mobile laser scanner dataset for benchmarking urban detection, segmentation and classification methods,” in 4th international conference on pattern recognition, applications and methods ICPRAM 2014, 2014.
  223. D. Pfeiffer, S. Gehrig, and N. Schneider, “Exploiting the power of stereo confidences,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 297–304, 2013.
  224. Y. Chen, J. Wang, J. Li, C. Lu, Z. Luo, H. Xue, and C. Wang, “Lidar-video driving dataset: Learning driving policies effectively,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5870–5878, 2018.
  225. M. Ramezani, Y. Wang, M. Camurri, D. Wisth, M. Mattamala, and M. Fallon, “The newer college dataset: Handheld lidar, inertial and vision with ground truth,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4353–4360, IEEE, 2020.
  226. A. P. Shah, J.-B. Lamare, T. Nguyen-Anh, and A. Hauptmann, “Cadp: A novel dataset for cctv traffic camera based accident analysis,” in 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–9, IEEE, 2018.
  227. A. Carballo, J. Lambert, A. Monrroy, D. Wong, P. Narksri, Y. Kitsukawa, E. Takeuchi, S. Kato, and K. Takeda, “Libre: The multiple 3d lidar dataset,” in 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1094–1101, IEEE, 2020.
  228. R. Xu, H. Xiang, X. Xia, X. Han, J. Li, and J. Ma, “Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communication,” in 2022 International Conference on Robotics and Automation (ICRA), pp. 2583–2589, IEEE, 2022.
  229. K. Matzen and N. Snavely, “Nyc3dcars: A dataset of 3d vehicles in geographic context,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 761–768, 2013.
  230. M. Wigness, S. Eum, J. G. Rogers, D. Han, and H. Kwon, “A rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5000–5007, IEEE, 2019.
  231. Z. Yan, L. Sun, T. Krajník, and Y. Ruichek, “Eu long-term dataset with multiple sensors for autonomous driving,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10697–10704, IEEE, 2020.
  232. M. Fabbri, G. Brasó, G. Maugeri, O. Cetintas, R. Gasparini, A. Ošep, S. Calderara, L. Leal-Taixé, and R. Cucchiara, “Motsynth: How can synthetic data help pedestrian detection and tracking?,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10849–10859, 2021.
  233. W. Kim, M. S. Ramanagopal, C. Barto, M.-Y. Yu, K. Rosaen, N. Goumas, R. Vasudevan, and M. Johnson-Roberson, “Pedx: Benchmark dataset for metric 3-d pose estimation of pedestrians in complex urban intersections,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1940–1947, 2019.
  234. W. Bao, Q. Yu, and Y. Kong, “Uncertainty-based traffic accident anticipation with spatio-temporal relational learning,” in Proceedings of the 28th ACM International Conference on Multimedia, pp. 2682–2690, 2020.
  235. F. R. Valverde, J. V. Hurtado, and A. Valada, “There is more than meets the eye: Self-supervised multi-object detection and tracking with sound by distilling multimodal knowledge,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11612–11621, 2021.
  236. T. Gruber, F. Julca-Aguilar, M. Bijelic, and F. Heide, “Gated2depth: Real-time dense lidar from gated images,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1506–1516, 2019.
  237. Y. Hu, J. Binas, D. Neil, S.-C. Liu, and T. Delbruck, “Ddd20 end-to-end event camera driving dataset: Fusing frames and events with deep learning for improved steering prediction,” in 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6, IEEE, 2020.
  238. J. Wiederer, A. Bouazizi, U. Kressel, and V. Belagiannis, “Traffic control gesture recognition for autonomous vehicles,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10676–10683, IEEE, 2020.
  239. R. Guzmán, J.-B. Hayet, and R. Klette, “Towards ubiquitous autonomous driving: The ccsad dataset,” in Computer Analysis of Images and Patterns: 16th International Conference, CAIP 2015, Valletta, Malta, September 2-4, 2015 Proceedings, Part I 16, pp. 582–593, Springer, 2015.
  240. T. Fleck, K. Daaboul, M. Weber, P. Schörner, M. Wehmer, J. Doll, S. Orf, N. Sußmann, C. Hubschneider, M. R. Zofka, et al., “Towards large scale urban traffic reference data: Smart infrastructure in the test area autonomous driving baden-württemberg,” in Intelligent Autonomous Systems 15: Proceedings of the 15th International Conference IAS-15, pp. 964–982, Springer, 2019.
  241. K. Behrendt, “Boxy vehicle detection in large images,” in Proceedings of the IEEE/CVF international conference on computer vision workshops, pp. 0–0, 2019.
  242. P. Koschorrek, T. Piccini, P. Oberg, M. Felsberg, L. Nielsen, and R. Mester, “A multi-sensor traffic scene dataset with omnidirectional video,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 727–734, 2013.
  243. A. Ligocki, A. Jelinek, and L. Zalud, “Brno urban dataset-the new data for self-driving agents and mapping tasks,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3284–3290, IEEE, 2020.
  244. P. Spannaus, P. Zechel, and K. Lenz, “Automatum data: Drone-based highway dataset for the development and validation of automated driving software for research and commercial applications,” in 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 1372–1377, IEEE, 2021.
  245. L. Li, K. N. Ismail, H. P. Shum, and T. P. Breckon, “Durlar: A high-fidelity 128-channel lidar dataset with panoramic ambient and reflectivity imagery for multi-modal autonomous driving applications,” in 2021 International Conference on 3D Vision (3DV), pp. 1227–1237, IEEE, 2021.
  246. B. Helou, A. Dusi, A. Collin, N. Mehdipour, Z. Chen, C. Lizarazo, C. Belta, T. Wongpiromsarn, R. D. Tebbens, and O. Beijbom, “The reasonable crowd: Towards evidence-based and interpretable models of driving behavior,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6708–6715, IEEE, 2021.
  247. N. Schneider and D. M. Gavrila, “Pedestrian path prediction with recursive bayesian filters: A comparative study,” in german conference on pattern recognition, pp. 174–183, Springer, 2013.
  248. S. Saralajew, L. Ohnemus, L. Ewecker, E. Asan, S. Isele, and S. Roos, “A dataset for provident vehicle detection at night,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 9750–9757, IEEE, 2021.
  249. K. Maag, R. Chan, S. Uhlemeyer, K. Kowol, and H. Gottschalk, “Two video data sets for tracking and retrieval of out of distribution objects,” in Proceedings of the Asian Conference on Computer Vision, pp. 3776–3794, 2022.
  250. K. Cordes, C. Reinders, P. Hindricks, J. Lammers, B. Rosenhahn, and H. Broszio, “Roadsaw: A large-scale dataset for camera-based road surface and wetness estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4440–4449, 2022.
  251. H. Quispe, J. Sumire, P. Condori, E. Alvarez, and H. Vera, “I see you: A vehicle-pedestrian interaction dataset from traffic surveillance cameras,” arXiv preprint arXiv:2211.09342, 2022.
  252. X. Wang, Z. Zhu, Y. Zhang, G. Huang, Y. Ye, W. Xu, Z. Chen, and X. Wang, “Are we ready for vision-centric driving streaming perception? the asap benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9600–9610, 2023.
  253. H. Wang, T. Li, Y. Li, L. Chen, C. Sima, Z. Liu, B. Wang, P. Jia, Y. Wang, S. Jiang, et al., “Openlane-v2: A topology reasoning benchmark for unified 3d hd mapping,” in Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
  254. X. Li, F. Flohr, Y. Yang, H. Xiong, M. Braun, S. Pan, K. Li, and D. M. Gavrila, “A new benchmark for vision-based cyclist detection,” in 2016 IEEE Intelligent Vehicles Symposium (IV), pp. 1028–1033, IEEE, 2016.
  255. A. Jain, H. S. Koppula, S. Soh, B. Raghavan, A. Singh, and A. Saxena, “Brain4cars: Car that knows before you do via sensory-fusion deep learning architecture,” arXiv preprint arXiv:1601.00740, 2016.
  256. I. Klein, “Nexet—the largest and most diverse road dataset in the world,” Medium, 2017.
  257. “Diml: https://dimlrgbd.github.io/,”
  258. E. Romera, L. M. Bergasa, and R. Arroyo, “Need data for driver behaviour analysis? presenting the public uah-driveset,” in 2016 IEEE 19th international conference on intelligent transportation systems (ITSC), pp. 387–392, IEEE, 2016.
  259. Y. Li, S. Li, X. Liu, M. Gong, K. Li, N. Chen, Z. Wang, Z. Li, T. Jiang, F. Yu, et al., “Sscbench: A large-scale 3d semantic scene completion benchmark for autonomous driving,” arXiv preprint arXiv:2306.09001, 2023.
  260. “Flir: https://www.flir.com/oem/adas/adas-dataset-form/,”
  261. K. Cordes and H. Broszio, “Vehicle lane merge visual benchmark,” in 2020 25th International Conference on Pattern Recognition (ICPR), pp. 715–722, IEEE, 2021.
  262. Z. Tang, M. Naphade, M.-Y. Liu, X. Yang, S. Birchfield, S. Wang, R. Kumar, D. Anastasiu, and J.-N. Hwang, “Cityflow: A city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8797–8806, 2019.
  263. R. Izquierdo, A. Quintanar, I. Parra, D. Fernández-Llorca, and M. Sotelo, “The prevention dataset: a novel benchmark for prediction of vehicles intentions,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3114–3121, IEEE, 2019.
  264. D. Barnes, W. Maddern, and I. Posner, “Find your own way: Weakly-supervised segmentation of path proposals for urban autonomy,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 203–210, IEEE, 2017.
  265. F. Saleh, M. S. Aliakbarian, M. Salzmann, L. Petersson, S. Gould, and J. M. Alvarez, “Built-in foreground/background prior for weakly-supervised semantic segmentation,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14, pp. 413–432, Springer, 2016.
  266. A. Petrovai, A. D. Costea, and S. Nedevschi, “Semi-automatic image annotation of street scenes,” in 2017 IEEE intelligent vehicles symposium (IV), pp. 448–455, IEEE, 2017.
  267. D. Acuna, H. Ling, A. Kar, and S. Fidler, “Efficient interactive annotation of segmentation datasets with polygon-rnn++,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 859–868, 2018.
  268. L. Castrejon, K. Kundu, R. Urtasun, and S. Fidler, “Annotating object instances with a polygon-rnn,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5230–5238, 2017.
  269. J. Xie, M. Kiefel, M.-T. Sun, and A. Geiger, “Semantic instance annotation of street scenes by 3d to 2d label transfer,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 3688–3697, 2016.
  270. Z. Chen, Q. Liao, Z. Wang, Y. Liu, and M. Liu, “Image detector based automatic 3d data labeling and training for vehicle detection on point cloud,” in 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 1408–1413, IEEE, 2019.
  271. H. Luo, C. Wang, C. Wen, Z. Chen, D. Zai, Y. Yu, and J. Li, “Semantic labeling of mobile lidar point clouds via active learning and higher order mrf,” IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 7, pp. 3631–3644, 2018.
  272. M. Liu, Y. Zhou, C. R. Qi, B. Gong, H. Su, and D. Anguelov, “Less: Label-efficient semantic segmentation for lidar point clouds,” in European Conference on Computer Vision, pp. 70–89, Springer, 2022.
  273. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “Labelme: a database and web-based tool for image annotation,” International journal of computer vision, vol. 77, pp. 157–173, 2008.
  274. B.-L. Wang, C.-T. King, and H.-K. Chu, “A semi-automatic video labeling tool for autonomous driving based on multi-object detector and tracker,” in 2018 sixth international symposium on computing and networking (CANDAR), pp. 201–206, IEEE, 2018.
  275. N. Manikandan and K. Ganesan, “Deep learning based automatic video annotation tool for self-driving car,” arXiv preprint arXiv:1904.12618, 2019.
  276. D. Schörkhuber, F. Groh, and M. Gelautz, “Bounding box propagation for semi-automatic video annotation of nighttime driving scenes,” in 2021 12th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 131–137, IEEE, 2021.
  277. Q. Meng, W. Wang, T. Zhou, J. Shen, Y. Jia, and L. Van Gool, “Towards a weakly supervised framework for 3d point cloud object detection and annotation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 8, pp. 4454–4468, 2021.
  278. D. Zhang, D. Liang, Z. Zou, J. Li, X. Ye, Z. Liu, X. Tan, and X. Bai, “A simple vision transformer for weakly semi-supervised 3d object detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8373–8383, 2023.
  279. W. Zimmer, A. Rangesh, and M. Trivedi, “3d bat: A semi-automatic, web-based 3d annotation toolbox for full-surround, multi-modal data streams,” in 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 1816–1821, IEEE, 2019.
  280. M. Wang, T. Ganjineh, and R. Rojas, “Action annotated trajectory generation for autonomous maneuvers on structured road networks,” in The 5th International conference on automation, robotics and applications, pp. 67–72, IEEE, 2011.
  281. S. Moosavi, B. Omidvar-Tehrani, R. B. Craig, and R. Ramnath, “Annotation of car trajectories based on driving patterns,” arXiv preprint arXiv:1705.05219, 2017.
  282. S. Jarl, L. Aronsson, S. Rahrovani, and M. H. Chehreghani, “Active learning of driving scenario trajectories,” Engineering Applications of Artificial Intelligence, vol. 113, p. 104972, 2022.
  283. O. Styles, A. Ross, and V. Sanchez, “Forecasting pedestrian trajectory with machine-annotated training data,” in 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 716–721, IEEE, 2019.
  284. P. Krähenbühl, “Free supervision from video games,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2955–2964, 2018.
  285. D. Krajzewicz, J. Erdmann, M. Behrisch, and L. Bieker, “Recent development and applications of sumo-simulation of urban mobility,” International journal on advances in systems and measurements, vol. 5, no. 3&4, 2012.
  286. A. Juliani, V.-P. Berges, E. Teng, A. Cohen, J. Harper, C. Elion, C. Goy, Y. Gao, H. Henry, M. Mattar, et al., “Unity: A general platform for intelligent agents,” arXiv preprint arXiv:1809.02627, 2018.
  287. H.-M. Heyn, K. M. Habibullah, E. Knauss, J. Horkoff, M. Borg, A. Knauss, and P. J. Li, “Automotive perception software development: An empirical investigation into data, annotation, and ecosystem challenges,” arXiv preprint arXiv:2303.05947, 2023.
  288. O. Inel and L. Aroyo, “Validation methodology for expert-annotated datasets: Event annotation case study,” in 2nd Conference on Language, Data and Knowledge (LDK 2019), Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.
  289. J. Kim, A. Rohrbach, T. Darrell, J. Canny, and Z. Akata, “Textual explanations for self-driving vehicles,” in Proceedings of the European conference on computer vision (ECCV), pp. 563–578, 2018.
  290. A. B. Vasudevan, D. Dai, and L. Van Gool, “Object referring in videos with language and human gaze,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4129–4138, 2018.
  291. H. Chen, A. Suhr, D. Misra, N. Snavely, and Y. Artzi, “Touchdown: Natural language navigation and spatial reasoning in visual street environments,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12538–12547, 2019.
  292. T. Deruyttere, S. Vandenhende, D. Grujicic, L. Van Gool, and M.-F. Moens, “Talk2car: Taking control of your self-driving car,” arXiv preprint arXiv:1909.10838, 2019.
  293. Y. Xu, X. Yang, L. Gong, H.-C. Lin, T.-Y. Wu, Y. Li, and N. Vasconcelos, “Explainable object-induced action decision for autonomous vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9523–9532, 2020.
  294. Q. Feng, V. Ablavsky, and S. Sclaroff, “Cityflow-nl: Tracking and retrieval of vehicles at city scale by natural language descriptions (2021),” arXiv preprint ArXiv:2101.04741, 2021.
  295. K. Jain, V. Chhangani, A. Tiwari, K. M. Krishna, and V. Gandhi, “Ground then navigate: Language-guided navigation in dynamic scenes,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 4113–4120, IEEE, 2023.
  296. D. Wu, W. Han, T. Wang, Y. Liu, X. Zhang, and J. Shen, “Language prompt for autonomous driving,” arXiv preprint arXiv:2309.04379, 2023.
  297. T. Qian, J. Chen, L. Zhuo, Y. Jiao, and Y.-G. Jiang, “Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario,” arXiv preprint arXiv:2305.14836, 2023.
  298. L. Chen, O. Sinavski, J. Hünermann, A. Karnsund, A. J. Willmott, D. Birch, D. Maund, and J. Shotton, “Driving with llms: Fusing object-level vector modality for explainable autonomous driving,” arXiv preprint arXiv:2310.01957, 2023.
  299. S. Malla, C. Choi, I. Dwivedi, J. H. Choi, and J. Li, “Drama: Joint risk localization and captioning in driving,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1043–1052, 2023.
  300. E. Sachdeva, N. Agarwal, S. Chundi, S. Roelofs, J. Li, B. Dariush, C. Choi, and M. Kochenderfer, “Rank2tell: A multimodal driving dataset for joint importance ranking and reasoning,” arXiv preprint arXiv:2309.06597, 2023.
  301. Y. Ma, C. Cui, X. Cao, W. Ye, P. Liu, J. Lu, A. Abdelraouf, R. Gupta, K. Han, A. Bera, et al., “Lampilot: An open benchmark dataset for autonomous driving with language model programs,” arXiv preprint arXiv:2312.04372, 2023.
  302. H. Shao, Y. Hu, L. Wang, S. L. Waslander, Y. Liu, and H. Li, “Lmdrive: Closed-loop end-to-end driving with large language models,” arXiv preprint arXiv:2312.07488, 2023.
  303. Y. Inoue, Y. Yada, K. Tanahashi, and Y. Yamaguchi, “Nuscenes-mqa: Integrated evaluation of captions and qa for autonomous driving datasets using markup annotations,” arXiv preprint arXiv:2312.06352, 2023.
  304. W. Wang, J. Xie, C. Hu, H. Zou, J. Fan, W. Tong, Y. Wen, S. Wu, H. Deng, Z. Li, et al., “Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving,” arXiv preprint arXiv:2312.09245, 2023.
  305. C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, P. Luo, A. Geiger, and H. Li, “Drivelm: Driving with graph visual question answering,” arXiv preprint arXiv:2312.14150, 2023.
  306. S. Yang, J. Liu, R. Zhang, M. Pan, Z. Guo, X. Li, Z. Chen, P. Gao, Y. Guo, and S. Zhang, “Lidar-llm: Exploring the potential of large language models for 3d lidar understanding,” arXiv preprint arXiv:2312.14074, 2023.
  307. Y. Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, et al., “Planning-oriented autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17853–17862, 2023.
  308. C. Cui, Y. Ma, X. Cao, W. Ye, Y. Zhou, K. Liang, J. Chen, J. Lu, Z. Yang, K.-D. Liao, et al., “A survey on multimodal large language models for autonomous driving,” arXiv preprint arXiv:2311.12320, 2023.
  309. X. Zhou, M. Liu, B. L. Zagar, E. Yurtsever, and A. C. Knoll, “Vision language models in autonomous driving and intelligent transportation systems,” arXiv preprint arXiv:2310.14414, 2023.
  310. S. W. Kim, J. Philion, A. Torralba, and S. Fidler, “Drivegan: Towards a controllable high-quality neural simulation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5820–5829, 2021.
  311. A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado, “Gaia-1: A generative world model for autonomous driving,” arXiv preprint arXiv:2309.17080, 2023.
  312. X. Wang, Z. Zhu, G. Huang, X. Chen, and J. Lu, “Drivedreamer: Towards real-world-driven world models for autonomous driving,” arXiv preprint arXiv:2309.09777, 2023.
  313. F. Jia, W. Mao, Y. Liu, Y. Zhao, Y. Wen, C. Zhang, X. Zhang, and T. Wang, “Adriver-i: A general world model for autonomous driving,” arXiv preprint arXiv:2311.13549, 2023.
  314. M. Schwonberg, J. Niemeijer, J.-A. Termöhlen, J. P. Schäfer, N. M. Schmidt, H. Gottschalk, and T. Fingscheidt, “Survey on unsupervised domain adaptation for semantic segmentation for visual perception in automated driving,” IEEE Access, 2023.
  315. Ö. Erkent and C. Laugier, “Semantic segmentation with unsupervised domain adaptation under varying weather conditions for autonomous vehicles,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3580–3587, 2020.
  316. Y. Wei, Z. Wei, Y. Rao, J. Li, J. Zhou, and J. Lu, “Lidar distillation: Bridging the beam-induced domain gap for 3d object detection,” in European Conference on Computer Vision, pp. 179–195, Springer, 2022.
  317. C. Hu, S. Hudson, M. Ethier, M. Al-Sharman, D. Rayside, and W. Melek, “Sim-to-real domain adaptation for lane detection and classification in autonomous driving,” in 2022 IEEE Intelligent Vehicles Symposium (IV), pp. 457–463, IEEE, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Mingyu Liu (30 papers)
  2. Ekim Yurtsever (17 papers)
  3. Xingcheng Zhou (16 papers)
  4. Jonathan Fossaert (1 paper)
  5. Yuning Cui (5 papers)
  6. Bare Luka Zagar (5 papers)
  7. Alois C. Knoll (22 papers)
  8. Walter Zimmer (24 papers)
Citations (18)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets