Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing (2402.17464v3)

Published 27 Feb 2024 in cs.CV

Abstract: Generative 3D part assembly involves understanding part relationships and predicting their 6-DoF poses for assembling a realistic 3D shape. Prior work often focus on the geometry of individual parts, neglecting part-whole hierarchies of objects. Leveraging two key observations: 1) super-part poses provide strong hints about part poses, and 2) predicting super-part poses is easier due to fewer superparts, we propose a part-whole-hierarchy message passing network for efficient 3D part assembly. We first introduce super-parts by grouping geometrically similar parts without any semantic labels. Then we employ a part-whole hierarchical encoder, wherein a super-part encoder predicts latent super-part poses based on input parts. Subsequently, we transform the point cloud using the latent poses, feeding it to the part encoder for aggregating super-part information and reasoning about part relationships to predict all part poses. In training, only ground-truth part poses are required. During inference, the predicted latent poses of super-parts enhance interpretability. Experimental results on the PartNet dataset show that our method achieves state-of-the-art performance in part and connectivity accuracy and enables an interpretable hierarchical part assembly. Code is available at https://github.com/pkudba/3DHPA.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Learning representations and generative models for 3d point clouds. In International conference on machine learning, pages 40–49. PMLR, 2018.
  2. Probabilistic reasoning for assembly-based 3d modeling. In ACM SIGGRAPH 2011 papers, pages 1–10. 2011.
  3. Neural shape mating: Self-supervised object assembly with adversarial shape priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12724–12733, 2022.
  4. Score-pa: Score-based 3d part assembly. arXiv preprint arXiv:2309.04220, 2023.
  5. Auto-encoding variational bayes. In Proceedings of the International Conference on Learning Representations (ICLR), 2014.
  6. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 605–613, 2017.
  7. Modeling by example. ACM transactions on graphics (TOG), 23(3):652–663, 2004.
  8. Sdm-net: Deep generative network for structured deformable mesh. ACM Transactions on Graphics (TOG), 38(6):1–15, 2019.
  9. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  10. Assembly-based conceptual 3d modeling with unlabeled components using probabilistic factor graph. Computer-Aided Design, 74:45–54, 2016.
  11. Shapeassembly: Learning to generate programs for 3d shape structure synthesis. ACM Transactions on Graphics (TOG), 39(6):1–20, 2020.
  12. A probabilistic model for component-based shape synthesis. Acm Transactions on Graphics (TOG), 31(4):1–11, 2012.
  13. Grass: Generative recursive autoencoders for shape structures. ACM Transactions on Graphics (TOG), 36(4):1–14, 2017.
  14. Learning 3d part assembly from a single image. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16, pages 664–682. Springer, 2020.
  15. Learning pose estimation for high-precision robotic assembly using simulated depth images. In 2019 International Conference on Robotics and Automation (ICRA), pages 3521–3527. IEEE, 2019.
  16. Reinforcement learning on variable impedance controller for high-precision robotic assembly. In 2019 International Conference on Robotics and Automation (ICRA), pages 3080–3087. IEEE, 2019.
  17. Structurenet: Hierarchical graph networks for 3d shape generation. arXiv preprint arXiv:1908.00575, 2019a.
  18. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 909–918, 2019b.
  19. Pt2pc: Learning to generate 3d point cloud shapes from part tree conditions. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16, pages 683–701. Springer, 2020.
  20. Fast marching farthest point sampling. Technical report, University of Cambridge, Computer Laboratory, 2003.
  21. Rgl-net: A recurrent graph learning framework for progressive part assembly. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 78–87, 2022.
  22. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017.
  23. Olinde Rodrigues. Des lois géométriques qui régissent les déplacements d’un système solide dans l’espace, et de la variation des coordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire. Journal de mathématiques pures et appliquées, 5:380–440, 1840.
  24. Componet: Learning to generate the unseen by part synthesis and composition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8759–8768, 2019.
  25. Learning to scaffold the development of robotic manipulation skills. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 5671–5677. IEEE, 2020.
  26. 3d point cloud generative adversarial network based on tree structured graph convolutions. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3859–3868, 2019.
  27. Complementme: Weakly-supervised component suggestions for 3d modeling. ACM Transactions on Graphics (TOG), 36(6):1–12, 2017.
  28. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  29. Pq-net: A generative part seq2seq network for 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 829–838, 2020.
  30. Learning compact representations for lidar completion and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1074–1083, 2023.
  31. Unsupervised 3d shape reconstruction by part retrieval and assembly. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8559–8567, 2023.
  32. Form2fit: Learning shape priors for generalizable assembly from disassembly. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 9404–9410. IEEE, 2020.
  33. Generative 3d part assembly via dynamic graph learning. Advances in Neural Information Processing Systems, 33:6315–6326, 2020.
  34. 3d part assembly generation with instance encoded transformer. IEEE Robotics and Automation Letters, 7(4):9051–9058, 2022.
  35. Learning to generate realistic lidar point clouds. In European Conference on Computer Vision, pages 17–35. Springer, 2022.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now