Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Collaborative Perception Datasets in Autonomous Driving: A Survey (2404.14022v1)

Published 22 Apr 2024 in cs.CV and cs.RO

Abstract: This survey offers a comprehensive examination of collaborative perception datasets in the context of Vehicle-to-Infrastructure (V2I), Vehicle-to-Vehicle (V2V), and Vehicle-to-Everything (V2X). It highlights the latest developments in large-scale benchmarks that accelerate advancements in perception tasks for autonomous vehicles. The paper systematically analyzes a variety of datasets, comparing them based on aspects such as diversity, sensor setup, quality, public availability, and their applicability to downstream tasks. It also highlights the key challenges such as domain shift, sensor setup limitations, and gaps in dataset diversity and availability. The importance of addressing privacy and security concerns in the development of datasets is emphasized, regarding data sharing and dataset creation. The conclusion underscores the necessity for comprehensive, globally accessible datasets and collaborative efforts from both technological and research communities to overcome these challenges and fully harness the potential of autonomous driving.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. T.-H. Wang, S. Manivasagam, M. Liang, B. Yang, W. Zeng, and R. Urtasun, “V2vnet: Vehicle-to-vehicle communication for joint perception and prediction,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16.   Springer, 2020, pp. 605–621.
  2. Z. Bai, G. Wu, M. J. Barth, Y. Liu, E. A. Sisbot, and K. Oguchi, “Pillargrid: Deep learning-based cooperative perception for 3d object detection from onboard-roadside lidar,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC).   IEEE, 2022, pp. 1743–1749.
  3. C. Zhang, J. Wei, S. Qu, C. Huang, J. Dai, P. Fu, Z. Wang, and X. Li, “Implementation of a V2P-Based VRU Warning System With C-V2X Technology,” IEEE Access, vol. 11, pp. 69 903–69 915, 2023. [Online]. Available: https://ieeexplore.ieee.org/document/10175872/
  4. Y. Li, D. Ma, Z. An, Z. Wang, Y. Zhong, S. Chen, and C. Feng, “V2x-sim: Multi-agent collaborative perception dataset and benchmark for autonomous driving,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 914–10 921, 2022.
  5. A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the KITTI vision benchmark suite,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition.   IEEE, pp. 3354–3361. [Online]. Available: http://ieeexplore.ieee.org/document/6248074/
  6. H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuScenes: A multimodal dataset for autonomous driving,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).   IEEE, pp. 11 618–11 628. [Online]. Available: https://ieeexplore.ieee.org/document/9156412/
  7. P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine et al., “Scalability in perception for autonomous driving: Waymo open dataset,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2446–2454.
  8. H. Wang, X. Zhang, Z. Li, J. Li, K. Wang, Z. Lei, and R. Haibing, “Ips300+: a challenging multi-modal data sets for intersection perception system,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 2539–2545.
  9. X. Ye, M. Shu, H. Li, Y. Shi, Y. Li, G. Wang, X. Tan, and E. Ding, “Rope3d: The roadside perception dataset for autonomous driving and monocular 3d object detection task,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).   IEEE, pp. 21 309–21 318. [Online]. Available: https://ieeexplore.ieee.org/document/9879696/
  10. R. Xu, H. Xiang, Z. Tu, X. Xia, M.-H. Yang, and J. Ma, “V2x-vit: Vehicle-to-everything cooperative perception with vision transformer,” in European conference on computer vision.   Springer, 2022, pp. 107–124.
  11. Y. Yuan and M. Sester, “Comap: A synthetic dataset for collective multi-agent perception of autonomous driving,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 43, pp. 255–263, 2021.
  12. S. Busch, C. Koetsier, J. Axmann, and C. Brenner, “LUMPI: The leibniz university multi-perspective intersection dataset,” in 2022 IEEE Intelligent Vehicles Symposium (IV).   IEEE, pp. 1127–1134. [Online]. Available: https://ieeexplore.ieee.org/document/9827157/
  13. D. Yongqiang, W. Dengjiang, C. Gang, M. Bing, G. Xijia, W. Yajun, L. Jianchao, F. Yanming, and L. Juanjuan, “Baai-vanjee roadside dataset: Towards the connected automated vehicle highway technologies in challenging environments of china,” arXiv preprint arXiv:2105.14370, 2021.
  14. W. Zimmer, C. Creß, H. T. Nguyen, and A. C. Knoll, “Tumtraf intersection dataset: All you need for urban 3d camera-lidar roadside perception,” in 2023 IEEE Intelligent Transportation Systems Conference (ITSC).   IEEE, 2023.
  15. R. Hao, S. Fan, Y. Dai, Z. Zhang, C. Li, Y. Wang, H. Yu, W. Yang, Y. Jirui, and Z. Nie, “Rcooper: A real-world large-scale dataset for roadside cooperative perception,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
  16. A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “PointPillars: Fast encoders for object detection from point clouds,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).   IEEE, pp. 12 689–12 697. [Online]. Available: https://ieeexplore.ieee.org/document/8954311/
  17. G. Brazil and X. Liu, “M3d-rpn: Monocular 3d region proposal network for object detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9287–9296.
  18. W. Zimmer, J. Birkner, M. Brucker, H. T. Nguyen, S. Petrovski, B. Wang, and A. C. Knoll, “Infradet3d: Multi-modal 3d object detection based on roadside infrastructure camera and lidar sensors,” in 2023 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2023.
  19. Q. Chen, X. Ma, S. Tang, J. Guo, Q. Yang, and S. Fu, “F-cooper: Feature based cooperative perception for autonomous vehicle edge computing system using 3d point clouds,” in Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, 2019, pp. 88–100.
  20. J. L. J. M. Runsheng Xu Hao Xiang, Xin Xia Xu Han, “Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communication,” in 2022 IEEE International Conference on Robotics and Automation (ICRA), 2022.
  21. R. Xu, Z. Tu, H. Xiang, W. Shao, B. Zhou, and J. Ma, “CoBEVT: Cooperative bird’s eye view semantic segmentation with sparse transformers.” [Online]. Available: http://arxiv.org/abs/2207.02202
  22. Y. Li, S. Ren, P. Wu, S. Chen, C. Feng, and W. Zhang, “Learning distilled collaboration graph for multi-agent perception,” Advances in Neural Information Processing Systems, vol. 34, pp. 29 541–29 552, 2021.
  23. R. Xu, Y. Guo, X. Han, X. Xia, H. Xiang, and J. Ma, “Opencda: an open cooperative driving automation framework integrated with co-simulation,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).   IEEE, 2021, pp. 1155–1162.
  24. H. Yu, Y. Luo, M. Shu, Y. Huo, Z. Yang, Y. Shi, Z. Guo, H. Li, X. Hu, J. Yuan, and Z. Nie, “DAIR-v2x: A large-scale dataset for vehicle-infrastructure cooperative 3d object detection,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).   IEEE, pp. 21 329–21 338. [Online]. Available: https://ieeexplore.ieee.org/document/9879243/
  25. R. Mao, J. Guo, Y. Jia, Y. Sun, S. Zhou, and Z. Niu, “Dolphins: Dataset for collaborative perception enabled harmonious and interconnected self-driving,” in Proceedings of the Asian Conference on Computer Vision, 2022, pp. 4361–4377.
  26. J. Axmann, R. Moftizadeh, J. Su, B. Tennstedt, Q. Zou, Y. Yuan, D. Ernst, H. Alkhatib, C. Brenner, and S. Schön, “LUCOOP: Leibniz university cooperative perception and urban navigation dataset,” in 2023 IEEE Intelligent Vehicles Symposium (IV).   IEEE, pp. 1–8. [Online]. Available: https://ieeexplore.ieee.org/document/10186693/
  27. R. Xu, X. Xia, J. Li, H. Li, S. Zhang, Z. Tu, Z. Meng, H. Xiang, X. Dong, R. Song et al., “V2v4real: A real-world large-scale dataset for vehicle-to-vehicle cooperative perception,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 712–13 722.
  28. H. Yu, W. Yang, H. Ruan, Z. Yang, Y. Tang, X. Gao, X. Hao, Y. Shi, Y. Pan, N. Sun et al., “V2x-seq: A large-scale sequential dataset for vehicle-infrastructure cooperative perception and forecasting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 5486–5495.
  29. T. Wang, S. Kim, W. Ji, E. Xie, C. Ge, J. Chen, Z. Li, and P. Luo, “DeepAccident: A motion and accident prediction benchmark for v2x autonomous driving.” [Online]. Available: http://arxiv.org/abs/2304.01168
  30. W. Zimmer, G. A. Wardana, S. Sritharan, X. Zhou, R. Song, and A. Knoll, “Tumtraf v2x cooperative perception dataset,” arXiv preprint arXiv:2403.01316, 2024.
  31. A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Conference on robot learning.   PMLR, 2017, pp. 1–16.
  32. D. Krajzewicz, J. Erdmann, M. Behrisch, and L. Bieker, “Recent development and applications of sumo-simulation of urban mobility,” International journal on advances in systems and measurements, vol. 5, no. 3&4, 2012.
  33. Y.-C. Liu, J. Tian, C.-Y. Ma, N. Glaser, C.-W. Kuo, and Z. Kira, “Who2com: Collaborative perception via learnable handshake communication,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 6876–6883.
  34. Y.-C. Liu, J. Tian, N. Glaser, and Z. Kira, “When2com: Multi-agent perception via communication graph grouping,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).   IEEE, pp. 4105–4114. [Online]. Available: https://ieeexplore.ieee.org/document/9156848/
  35. Y. Zhou and O. Tuzel, “VoxelNet: End-to-end learning for point cloud based 3d object detection,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.   IEEE, pp. 4490–4499. [Online]. Available: https://ieeexplore.ieee.org/document/8578570/
  36. D. Rukhovich, A. Vorontsova, and A. Konushin, “Imvoxelnet: Image to voxels projection for monocular and multi-view general-purpose 3d object detection,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 2397–2406.
  37. V. A. Sindagi, Y. Zhou, and O. Tuzel, “Mvx-net: Multimodal voxelnet for 3d object detection,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 7276–7282.
  38. Landesamt für Geoinformation und Landesvermessung Niedersachsen (LGLN), Landesvermessung und Geobasisinformation, “3D-Gebäudemodell (LoD2),” https://opengeodata.lgln.niedersachsen.de/#lod2, 2022, [Online; accessed 1-February-2024].
  39. M.-F. Chang, J. Lambert, P. Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P. Carr, S. Lucey, D. Ramanan et al., “Argoverse: 3d tracking and forecasting with rich maps. in 2019 ieee,” in CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8740–8749.
  40. “ASAM OpenLABEL,” https://www.asam.net/standards/detail/openlabel/, last accessed on 8th April 2024.
  41. H. Xiang, R. Xu, X. Xia, Z. Zheng, B. Zhou, and J. Ma, “V2xp-asg: Generating adversarial scenes for vehicle-to-everything perception,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 3584–3591.
  42. D. Bogdoll, S. Uhlemeyer, K. Kowol, and J. M. Zöllner, “Perception Datasets for Anomaly Detection in Autonomous Driving: A Survey,” in Intelligent Vehicles Symposium (IV), 2023.
  43. Y. Tian, J. Wang, Y. Wang, C. Zhao, F. Yao, and X. Wang, “Federated vehicular transformers and their federations: Privacy-preserving computing and cooperation for autonomous driving,” vol. 7, no. 3, pp. 456–465. [Online]. Available: https://ieeexplore.ieee.org/document/9857660/
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now