Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PWR-Align: Leveraging Part-Whole Relationships for Part-wise Rigid Point Cloud Registration in Mixed Reality Applications (2306.06717v1)

Published 11 Jun 2023 in cs.CV and cs.GR

Abstract: We present an efficient and robust point cloud registration (PCR) workflow for part-wise rigid point cloud alignment using the Microsoft HoloLens 2. Point Cloud Registration (PCR) is an important problem in Augmented and Mixed Reality use cases, and we present a study for a special class of non-rigid transformations. Many commonly encountered objects are composed of rigid parts that move relative to one another about joints resulting in non-rigid deformation of the whole object such as robots with manipulators, and machines with hinges. The workflow presented allows us to register the point cloud with various configurations of the point cloud.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. What is mixed reality? In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–15, 2019.
  2. A review of point cloud registration algorithms for mobile robotics. Foundations and Trends® in Robotics, 4(1):1–104, 2015.
  3. Mixed reality-based user interface for quality control inspection of car body surfaces. Journal of Manufacturing Systems, 53:75–92, 2019.
  4. Mixed reality environment for mission critical systems servicing and repair. In International Conference on Virtual, Augmented and Mixed Reality, pages 201–210. Springer, 2013.
  5. Empirical evidence, evaluation criteria and challenges for the effectiveness of virtual and mixed reality tools for training operators of car service maintenance. Computers in Industry, 67:17–26, 2015.
  6. Marvel: A mixed-reality learning environment for vocational training in mechatronics. In Proceedings of the Technology Enhanced Learning International Conference (TEL 03), 2003.
  7. Creating next generation blended learning environments using mixed reality, video games and simulations. TechTrends, 49(3):42–53, 2005.
  8. Mixed reality technology applications in construction equipment operator training. In Proceedings of the 21st International Symposium on Automation and Robotics in Construction (ISARC 2004), pages 21–25, 2004.
  9. Deep projective 3d semantic segmentation. In International Conference on Computer Analysis of Images and Patterns, pages 95–107. Springer, 2017.
  10. Voxnet: A 3d convolutional neural network for real-time object recognition. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 922–928. IEEE, 2015.
  11. Classification of point cloud scenes with multiscale voxel deep network. arXiv preprint arXiv:1804.03583, 2018.
  12. 3dmfv: Three-dimensional point cloud classification in real-time using convolutional neural networks. IEEE Robotics and Automation Letters, 3(4):3145–3152, 2018.
  13. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6411–6420, 2019.
  14. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3075–3084, 2019.
  15. Pointdsc: Robust point cloud registration using deep spatial consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15859–15869, 2021.
  16. 3dregnet: A deep neural network for 3d point registration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7193–7203, 2020.
  17. Registration with the point cloud library: A modular framework for aligning in 3-d. IEEE Robotics & Automation Magazine, 22(4):110–124, 2015.
  18. Efficient ransac for point-cloud shape detection. In Computer graphics forum, volume 26, pages 214–226. Wiley Online Library, 2007.
  19. Omnet: Learning overlapping mask for partial-to-partial point cloud registration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3132–3141, 2021.
  20. Dpfm: Deep partial functional maps. In 2021 International Conference on 3D Vision (3DV), pages 175–185. IEEE, 2021.
  21. Partial functional correspondence in computer graphics forum, 2017.
  22. Geometric features and their relevance for 3d point cloud classification. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences, 4, 2017.
  23. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017.
  24. Least-squares fitting of two 3-d point sets. IEEE Transactions on pattern analysis and machine intelligence, (5):698–700, 1987.
  25. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, pages 586–606. Spie, 1992.
  26. Zhengyou Zhang. Iterative point matching for registration of free-form curves and surfaces. International journal of computer vision, 13(2):119–152, 1994.
  27. Global correspondence optimization for non-rigid registration of depth scans. In Computer graphics forum, volume 27, pages 1421–1430. Wiley Online Library, 2008.
  28. 4dcomplete: Non-rigid motion estimation beyond the observable surface. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12706–12716, 2021.
  29. Open source 3d model of nasa viking lander. https://nasa3d.arc.nasa.gov/detail/viking-lander. Accessed: 2023-03-03.
  30. Open source 3d model of cubebot in dancing posture. https://sketchfab.com/3d-models/dancing-cubebot-fe41ed9dce7e41b5a46fed8705a6821e. Accessed: 2023-03-03.
  31. Lepard: Learning partial point cloud matching in rigid and deformable scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5554–5564, 2022.
  32. A survey of iterative closest point algorithm. In 2017 Chinese Automation Congress (CAC), pages 4395–4399. IEEE, 2017.
  33. Hololens 2 research mode as a tool for computer vision research. arXiv preprint arXiv:2008.11239, 2020.
  34. Manorama Jha. Point cloud registration of non-rigid objects in sparse 3d scans with applications in mixed reality. arXiv preprint arXiv:2212.03856, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now