Unsupervised Template-assisted Point Cloud Shape Correspondence Network (2403.16412v1)
Abstract: Unsupervised point cloud shape correspondence aims to establish point-wise correspondences between source and target point clouds. Existing methods obtain correspondences directly by computing point-wise feature similarity between point clouds. However, non-rigid objects possess strong deformability and unusual shapes, making it a longstanding challenge to directly establish correspondences between point clouds with unconventional shapes. To address this challenge, we propose an unsupervised Template-Assisted point cloud shape correspondence Network, termed TANet, including a template generation module and a template assistance module. The proposed TANet enjoys several merits. Firstly, the template generation module establishes a set of learnable templates with explicit structures. Secondly, we introduce a template assistance module that extensively leverages the generated templates to establish more accurate shape correspondences from multiple perspectives. Extensive experiments on four human and animal datasets demonstrate that TANet achieves favorable performance against state-of-the-art methods.
- An overview of augmented reality. Computers, 11(2):28, 2022.
- Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching. Proceedings of the National Academy of Sciences, 103(5):1168–1172, 2006.
- Numerical geometry of non-rigid shapes. Springer Science & Business Media, 2008.
- Self-supervised learning for multimodal non-rigid 3d shape matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17735–17744, 2023.
- Supervised descriptor learning for non-rigid shape matching. In European conference on computer vision, pages 283–298. Springer, 2014.
- Shrec’16: Partial matching of deformable shapes. Proc. 3DOR, 2(9):12, 2016.
- Se-ornet: Self-ensembling orientation-aware network for unsupervised point cloud shape correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5364–5373, 2023.
- Learning elementary structures for 3d shape generation and matching. Advances in Neural Information Processing Systems, 32, 2019.
- Deep geometric functional maps: Robust feature learning for shape correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8592–8601, 2020.
- Learning 3d object shape and layout without 3d supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1695–1704, 2022.
- 3d-coded: 3d correspondences by deep deformation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 230–246, 2018.
- Pct: Point cloud transformer. Computational Visual Media, 7(2):187–199, 2021.
- Hierarchical shape-consistent transformer for unsupervised point cloud shape correspondence. IEEE Transactions on Image Processing, 2023.
- Non-rigid registration under isometric deformations. In Computer Graphics Forum, pages 1449–1457. Wiley Online Library, 2008.
- Utilizing a game engine for interactive 3d topographic data visualization. ISPRS International Journal of Geo-Information, 8(8):361, 2019.
- Dpc: Unsupervised deep point correspondence via cross and self construction. In 2021 International Conference on 3D Vision (3DV), pages 1442–1451. IEEE, 2021.
- Category-level 6d object pose and size estimation using self-supervised deep prior deformation networks. In European Conference on Computer Vision, pages 19–34. Springer, 2022.
- Deep functional maps: Structured prediction for dense shape correspondence. In Proceedings of the IEEE international conference on computer vision, pages 5659–5667, 2017.
- Learning spectral descriptors for deformable shape correspondence. IEEE transactions on pattern analysis and machine intelligence, 36(1):171–180, 2013.
- Towards high-fidelity single-view holistic reconstruction of indoor scenes. In European Conference on Computer Vision, pages 429–446. Springer, 2022.
- Ming Liu. Robotic online path planning on point cloud. IEEE transactions on cybernetics, 46(5):1217–1228, 2015.
- Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
- Query refinement transformer for 3d instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 18516–18526, 2023.
- Correspondence learning via linearly-invariant embedding. Advances in Neural Information Processing Systems, 33:1608–1620, 2020.
- Shrec 2019: Matching humans with different connectivity. In Eurographics Workshop on 3D Object Retrieval, page 3, 2019.
- Learning 3d scene priors with 2d supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 792–802, 2023.
- Functional maps: a flexible representation of maps between shapes. ACM Transactions on Graphics (ToG), 31(4):1–11, 2012.
- Automatic differentiation in pytorch. 2017.
- Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017a.
- Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017b.
- Unsupervised deep learning for structured shape matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1617–1627, 2019.
- Towards 3d point cloud based object maps for household environments. Robotics and Autonomous Systems, 56(11):927–941, 2008.
- Mask3d: Mask transformer for 3d semantic instance segmentation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 8216–8223. IEEE, 2023.
- Implicit field supervision for robust non-rigid shape matching. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part III, pages 344–362. Springer, 2022.
- Intrinsic shape matching by planned landmark sampling. In Computer graphics forum, pages 543–552. Wiley Online Library, 2011.
- Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6411–6420, 2019.
- Shape prior deformation for categorical 6d object pose and size estimation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16, pages 530–546. Springer, 2020.
- Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog), 38(5):1–12, 2019.
- Semaffinet: Semantic-affine transformation for point cloud segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11819–11829, 2022.
- Owlii dynamic human mesh sequence dataset. In ISO/IEC JTC1/SC29/WG11 m41658, 120th MPEG Meeting, 2017.
- Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 206–215, 2018.
- Corrnet3d: Unsupervised end-to-end learning of dense correspondence for 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6052–6061, 2021.
- Holistic 3d scene understanding from a single image with implicit representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8833–8842, 2021.
- Point transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16259–16268, 2021.
- 3d menagerie: Modeling the 3d shape and pose of animals. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6365–6373, 2017.