Papers
Topics
Authors
Recent
Search
2000 character limit reached

VRSO: Visual-Centric Reconstruction for Static Object Annotation

Published 22 Mar 2024 in cs.CV | (2403.15026v3)

Abstract: As a part of the perception results of intelligent driving systems, static object detection (SOD) in 3D space provides crucial cues for driving environment understanding. With the rapid deployment of deep neural networks for SOD tasks, the demand for high-quality training samples soars. The traditional, also reliable, way is manual labelling over the dense LiDAR point clouds and reference images. Though most public driving datasets adopt this strategy to provide SOD ground truth (GT), it is still expensive and time-consuming in practice. This paper introduces VRSO, a visual-centric approach for static object annotation. Experiments on the Waymo Open Dataset show that the mean reprojection error from VRSO annotation is only 2.6 pixels, around four times lower than the Waymo Open Dataset labels (10.6 pixels). VRSO is distinguished in low cost, high efficiency, and high quality: (1) It recovers static objects in 3D space with only camera images as input, and (2) manual annotation is barely involved since GT for SOD tasks is generated based on an automatic reconstruction and annotation pipeline.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. J. Mei, A. Z. Zhu, X. Yan, H. Yan, S. Qiao, L.-C. Chen, and H. Kretzschmar, “Waymo open dataset: Panoramic video panoptic segmentation,” in European Conference on Computer Vision.   Springer, 2022, pp. 53–72.
  2. Y. Hu, J. Yang et al., “Planning-oriented autonomous driving,” in IEEE Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 853–17 862.
  3. M. Alibeigi, W. Ljungbergh, A. Tonderski, G. Hess, A. Lilja, C. Lindstrom, D. Motorniuk, J. Fu, J. Widahl, and C. Petersson, “Zenseact open dataset: A large-scale and diverse multimodal dataset for autonomous driving,” in IEEE International Conference on Computer Vision, 2023.
  4. C. Yang, W. Wang, Y. Zhang, Z. Zhang, L. Shen, Y. Li, and J. See, “Mlife: A lite framework for machine learning lifecycle initialization,” Machine Learning, vol. 110, pp. 2993–3013, 2021.
  5. J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 4104–4113.
  6. M. Adorjan, “Opensfm: A collaborative structure-from-motion system,” Ph.D. dissertation, Wien, 2016.
  7. P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk, “From coarse to fine: Robust hierarchical localization at large scale,” in IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 12 716–12 725.
  8. R. Mei, W. Sui, J. Zhang, Q. Zhang, T. Peng, and C. Yang, “Rome: Towards large scale road surface reconstruction via mesh representation,” arXiv preprint arXiv:2306.11368, 2023.
  9. J. Zhang, S. Chen, H. Yin, R. Mei, X. Liu, C. Yang, Q. Zhang, and W. Sui, “A vision-centric approach for static map element annotation,” arXiv preprint arXiv:2309.11754, 2023.
  10. C. He, H. Zeng, J. Huang, X.-S. Hua, and L. Zhang, “Structure aware single-stage 3d object detection from point cloud,” in IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 873–11 882.
  11. C. Szegedy, A. Toshev, and D. Erhan, “Deep neural networks for object detection,” Advances in Neural Information Processing Systems, vol. 26, 2013.
  12. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Region-based convolutional networks for accurate object detection and segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 1, pp. 142–158, 2015.
  13. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2117–2125.
  14. R. Girshick, “Fast r-cnn,” in IEEE International Conference on Computer Vision, 2015, pp. 1440–1448.
  15. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 779–788.
  16. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in European Conference on Computer Vision.   Springer, 2016, pp. 21–37.
  17. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in IEEE International Conference on Computer Vision, 2017, pp. 2980–2988.
  18. H. Hu, J. Gu, Z. Zhang, J. Dai, and Y. Wei, “Relation networks for object detection,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3588–3597.
  19. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in European Conference on Computer Vision.   Springer, 2020, pp. 213–229.
  20. M. Wang, R. Liu, J. Yang, X. Lu, J. Yu, and H. Ren, “Traffic sign three-dimensional reconstruction based on point clouds and panoramic images,” The Photogrammetric Record, vol. 37, no. 177, pp. 87–110, 2022.
  21. H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 621–11 631.
  22. P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine et al., “Scalability in perception for autonomous driving: Waymo open dataset,” in IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 2446–2454.
  23. J. S. Grosman, P. H. Furtado, A. M. Rodrigues, G. G. Schardong, S. D. Barbosa, and H. C. Lopes, “Eras: Improving the quality control in the annotation process for natural language processing tasks,” Information Systems, vol. 93, p. 101553, 2020.
  24. D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self-supervised interest point detection and description,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 224–236.
  25. P. Voigtlaender, M. Krause, A. Osep, J. Luiten, B. B. G. Sekar, A. Geiger, and B. Leibe, “Mots: Multi-object tracking and segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 7942–7951.
  26. Z. Zong, G. Song, and Y. Liu, “Detrs with collaborative hybrid assignments training,” in IEEE International Conference on Computer Vision, 2023, pp. 6748–6758.
  27. A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo et al., “Segment anything,” in IEEE International Conference on Computer Vision, 2023, pp. 4015–4026.
  28. S. Agarwal, K. Mierle, and T. C. S. Team, “Ceres Solver,” 10 2023. [Online]. Available: https://github.com/ceres-solver/ceres-solver
  29. P. J. Huber, “Robust estimation of a location parameter,” in Breakthroughs in Statistics.   Springer, 1992, pp. 492–518.
  30. X. Lin, Z. Pei, T. Lin, L. Huang, and Z. Su, “Sparse4d v3: Advancing end-to-end 3d detection and tracking,” arXiv preprint arXiv:2311.11722, 2023.
  31. B. Wang, Z. Wang, C. Zhu, Z. Zhang, Z. Wang, P. Lin, J. Liu, and Q. Zhang, “Interpretable motion planner for urban driving via hierarchical imitation learning,” in International Conference on Intelligent Robots and Systems, 2023, pp. 1691–1696.
  32. C. Pan, Y. He, J. Peng, Q. Zhang, W. Sui, and Z. Zhang, “Baeformer: Bi-directional and early interaction transformers for bird’s eye view semantic segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2023, pp. 9590–9599.
  33. S. Zhu, R. Zhang, L. Zhou, T. Shen, T. Fang, P. Tan, and L. Quan, “Very large-scale global sfm by distributed motion averaging,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4568–4577.
  34. Z. Wang, Y. Li, X. Chen, S.-N. Lim, A. Torralba, H. Zhao, and S. Wang, “Detecting everything in the open world: Towards universal object detection,” in IEEE Conference on Computer Vision and Pattern Recognition, 2023, pp. 11 433–11 443.
  35. J. Lee, S. Walsh, A. Harakeh, and S. L. Waslander, “Leveraging pre-trained 3d object detection models for fast ground truth generation,” in International Conference on Intelligent Transportation Systems, 2018, pp. 2504–2510.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free