Papers
Topics
Authors
Recent
Search
2000 character limit reached

Deep Learning-Based Point Cloud Registration: A Comprehensive Survey and Taxonomy

Published 22 Apr 2024 in cs.CV | (2404.13830v3)

Abstract: Point cloud registration involves determining a rigid transformation to align a source point cloud with a target point cloud. This alignment is fundamental in applications such as autonomous driving, robotics, and medical imaging, where precise spatial correspondence is essential. Deep learning has greatly advanced point cloud registration by providing robust and efficient methods that address the limitations of traditional approaches, including sensitivity to noise, outliers, and initialization. However, a well-constructed taxonomy for these methods is still lacking, making it difficult to systematically classify and compare the various approaches. In this paper, we present a comprehensive survey and taxonomy on deep learning-based point cloud registration (DL-PCR). We begin with a formal description of the point cloud registration problem, followed by an overview of the datasets, evaluation metrics, and loss functions commonly used in DL-PCR. Next, we categorize existing DL-PCR methods into supervised and unsupervised approaches, as they focus on significantly different key aspects. For supervised DL-PCR methods, we organize the discussion based on key aspects, including the registration procedure, optimization strategy, learning paradigm, network enhancement, and integration with traditional methods; For unsupervised DL-PCR methods, we classify them into correspondence-based and correspondence-free approaches, depending on whether they require explicit identification of point-to-point correspondences. To facilitate a more comprehensive and fair comparison, we conduct quantitative evaluations of all recent state-of-the-art approaches, using a unified training setting and consistent data partitioning strategy. Lastly, we highlight the open challenges and discuss potential directions for future study. A comprehensive collection is available at https://github.com/yxzhang15/PCR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Spinnet: Learning a general surface descriptor for 3d point cloud registration. In CVPR, 2021.
  2. Buffer: Balancing accuracy, efficiency, and generalizability in point cloud registration. In CVPR, 2023.
  3. Pointnetlk: Robust & efficient point cloud registration using pointnet. In CVPR, 2019.
  4. Least-squares fitting of two 3-d point sets. IEEE TPAMI, (5), 1987.
  5. Pointdsc: Robust point cloud registration using deep spatial consistency. In CVPR, 2021.
  6. Pcam: Product of cross-attention matrices for rigid registration of point clouds. In ICCV, 2021.
  7. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015.
  8. Imlovenet: Misaligned image-supported registration network for low-overlap point cloud pairs. In SIGGRAPH, pages 1–9, 2022.
  9. Sc22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT-pcr: A second order spatial compatibility for efficient and robust point cloud registration. In CVPR, 2022.
  10. Detarnet: Decoupling translation and rotation by siamese network for point cloud registration. In AAAI, 2022.
  11. Point cloud registration based on learning gaussian mixture models with global-weighted local representations. IEEE Geosci. Remote Sens. Lett., 20:1–5, 2023.
  12. Sira-pcr: Sim-to-real adaptation for 3d point cloud registration. In ICCV, 2023.
  13. Diffusionpcr: Diffusion models for robust multi-step point cloud registration. arXiv preprint arXiv:2312.03053, 2023.
  14. Sc22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT-pcr++: Rethinking the generation and selection for efficient and robust boint cloud registration. IEEE TPAMI, 2023.
  15. Robust reconstruction of indoor scenes. In CVPR, 2015.
  16. B. Curless and M. Levoy. A volumetric method for building complex models from range images. In SIGGRAPH, 1996.
  17. Ppf-foldnet: Unsupervised learning of rotation invariant 3d local descriptors. In ECCV, 2018.
  18. 3d local features for direct pairwise registration. In CVPR, 2019.
  19. Registration of large-scale terrestrial laser scanner point clouds: A review and benchmark. ISPRS J. Photogramm. Remote Sens., 163:327–342, 2020.
  20. Hgmr: Hierarchical gaussian mixtures for adaptive 3d registration. In ECCV, 2018.
  21. Stickypillars: Robust and efficient feature matching on point clouds using graph neural networks. In CVPR, 2021.
  22. M. Fischler and R. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.
  23. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012.
  24. The perfect match: 3d point cloud matching with smoothed densities. In CVPR, 2019.
  25. Learning multiview 3d point cloud registration. In CVPR, 2020.
  26. A review of research on point cloud registration methods. In IOP Conf. Ser.: Mater. Sci. Eng., volume 782, page 022070. IOP Publishing, 2020.
  27. Predator: Registration of 3d point clouds with low overlap. In CVPR, 2021.
  28. A comprehensive survey on point cloud registration. arXiv preprint arXiv:2103.02690, 2021.
  29. Unsupervised point cloud registration by learning unified gaussian mixture models. IEEE Robotics Autom. Lett., 7(3):7028–7035, 2022.
  30. Gmf: General multimodal fusion framework for correspondence outlier rejection. IEEE Robotics Autom. Lett., 7(4):12585–12592, 2022.
  31. Imfnet: Interpretable multimodal fusion for point cloud registration. IEEE Robotics Autom. Lett., 7(4):12323–12330, 2022.
  32. Sampling network guided cross-entropy method for unsupervised point cloud registration. In ICCV, 2021.
  33. Robust outlier rejection for 3d registration with variational bayes. In CVPR, 2023.
  34. Deep hough voting for robust global registration. In ICCV, 2021.
  35. Iterative distance-aware similarity matrix convolution with mutual-supervised point elimination for efficient point cloud registration. In ECCV, 2020.
  36. End-to-end learning local multi-view descriptors for 3d point clouds. In CVPR, 2020.
  37. Pointnetlk revisited. In CVPR, 2021.
  38. Deepvcp: An end-to-end deep neural network for point cloud registration. In CVPR, 2019.
  39. L3-net: Towards learning based lidar localization for autonomous driving. In CVPR, 2019.
  40. Hregnet: A hierarchical network for large-scale outdoor lidar point cloud registration. In ICCV, 2021.
  41. B. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In IJCAI, 1981.
  42. Overlap-guided gaussian mixture models for point cloud registration. In WACV, 2023.
  43. Unsupervised deep probabilistic approach for partial point cloud registration. In CVPR, 2023.
  44. 3dregnet: A deep neural network for 3d point registration. In CVPR, 2020.
  45. F. Poiesi and D. Boscaini. Learning general and distinctive 3d local deep descriptors for point cloud registration. IEEE TPAMI, 45(3), 2022.
  46. Challenging data sets for point cloud registration algorithms. The International Journal of Robotics Research, 31(14):1705–1711, 2012.
  47. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017.
  48. Geotransformer: Fast and robust point cloud registration with geometric transformer. IEEE TPAMI, 45(8):9806–9821, 2023.
  49. Pcrnet: Point cloud registration network using pointnet encoding. arXiv preprint arXiv:1908.07906, 2019.
  50. Reliable inlier evaluation for unsupervised point cloud registration. In AAAI, 2022.
  51. Research and application on cross-source point cloud registration method based on unsupervised learning. In CYBER, 2023.
  52. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In ICCV, 2019.
  53. Attention is all you need. In NeurIPS, 2017.
  54. Y. Wang and J. Solomon. Deep closest point: Learning representations for point cloud registration. In ICCV, 2019.
  55. Y. Wang and J. Solomon. Prnet: Self-supervised learning for partial-to-partial registration. In NeurIPS, 2019.
  56. You only hypothesize once: Point cloud registration with rotation-equivariant descriptors. In ACM MM, 2022.
  57. Storm: Structure-based overlap matching for partial point cloud registration. IEEE TPAMI, 45(1):1135–1149, 2022.
  58. Robust multiview point cloud registration with reliable pose graph initialization and history reweighting. In CVPR, 2023.
  59. Roreg: Pairwise point cloud registration with oriented descriptors and local rotations. IEEE TPAMI, 45(8), 2023.
  60. 3d shapenets: A deep representation for volumetric shapes. In CVPR, 2015.
  61. Feature interactive representation for point cloud registration. In ICCV, 2021.
  62. Omnet: Learning overlapping mask for partial-to-partial point cloud registration. In ICCV, 2021.
  63. Finet: Dual branches feature interaction for partial-to-partial point cloud registration. In AAAI, 2022.
  64. One-inlier is first: Towards efficient position encoding for point cloud registration. In NeurIPS, 2022.
  65. Z. Yew and G. Lee. Rpm-net: Robust point matching using learned features. In CVPR, 2020.
  66. Z. Yew and G. Lee. Regtr: End-to-end point cloud correspondences with transformers. In CVPR, 2022.
  67. Rotation-invariant transformer for point cloud matching. In CVPR, 2023.
  68. Peal: Prior-embedded explicit attention learning for low-overlap point cloud registration. In CVPR, 2023.
  69. Deepgmr: Learning latent gaussian mixture models for registration. In ECCV, 2020.
  70. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In CVPR, 2017.
  71. Deep learning based point cloud registration: an overview. Virtual Real. Intell. Hardw., 2(3):222–246, 2020.
  72. A representation separation perspective to correspondence-free unsupervised 3-d point cloud registration. IEEE Geosci. Remote Sens. Lett., 19:1–5, 2021.
  73. Partial point cloud registration with deep local feature. IEEE TAI, 4(5):1317–1327, 2022.
  74. Point cloud registration using multiattention mechanism and deep hybrid features. IEEE Intell. Syst., 38(1):58–68, 2022.
  75. Pcr-cg: Point cloud registration via deep explicit color and geometry. In ECCV, 2022.
  76. 3d registration with maximal cliques. In CVPR, 2023.
  77. Mm-pcqa: Multi-modal learning for no-reference point cloud quality assessment. In IJCAI, 2023.
  78. Learning and matching multi-view descriptors for registration of point clouds. In ECCV, 2018.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up