Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Survey on Modeling of Human-made Articulated Objects (2403.14937v2)

Published 22 Mar 2024 in cs.CV

Abstract: 3D modeling of articulated objects is a research problem within computer vision, graphics, and robotics. Its objective is to understand the shape and motion of the articulated components, represent the geometry and mobility of object parts, and create realistic models that reflect articulated objects in the real world. This survey provides a comprehensive overview of the current state-of-the-art in 3D modeling of articulated objects, with a specific focus on the task of articulated part perception and articulated object creation (reconstruction and generation). We systematically review and discuss the relevant literature from two perspectives: geometry modeling (i.e., structure and shape of articulated parts) and articulation modeling (i.e., dynamics and motion of parts). Through this survey, we highlight the substantial progress made in these areas, outline the ongoing challenges, and identify gaps for future research. Our survey aims to serve as a foundational reference for researchers and practitioners in computer vision and graphics, offering insights into the complexities of articulated object modeling.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (83)
  1. “Voronoi-based variational reconstruction of unoriented point sets” In Proceedings of the Eurographics Symposium on Geometry Processing 7, 2007, pp. 39–48
  2. “Learning to infer kinematic hierarchies for novel object instances” In Proceedings of the International Conference on Robotics and Automation (ICRA), 2022, pp. 8461–8467
  3. Mehmet Aygün and Oisin Mac Aodha “SAOR: Single-View Articulated Object Reconstruction” In arXiv preprint arXiv:2303.13514, 2023
  4. Ben Abbatematteo, Stefanie Tellex and George Konidaris “Learning to generalize kinematic models to novel objects” In Proceedings of the Conference on Robot Learning (CoRL), 2019, pp. 1289–1299
  5. “Generative neural articulated radiance fields” In Advances in neural information processing systems (NeurIPS) 35, 2022, pp. 19900–19916
  6. Paul J Besl and Neil D McKay “Method for registration of 3-D shapes” In Sensor fusion IV: control paradigms and data structures 1611, 1992, pp. 586–606 Spie
  7. “ShapeNet: An information-rich 3D model repository” In arXiv preprint arXiv:1512.03012, 2015
  8. “Fast-SNARF: A fast deformer for articulated neural fields” In IEEE Transactions on Pattern Analysis and Machine Intelligence IEEE, 2023
  9. “Banana: Banach fixed-Point network for pointcloud segmentation with inter-part equivariance” In arXiv preprint arXiv:2305.16314, 2023
  10. Samir Yitzhak Gadre, Kiana Ehsani and Shuran Song “Act the part: Learning interaction strategies for articulated object part discovery” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021, pp. 15752–15761
  11. Benjamin Graham, Martin Engelcke and Laurens Maaten “3D Semantic Segmentation with Submanifold Sparse Convolutional Networks” In CVPR, 2018, pp. 9224–9232
  12. Natasha Gelfand and Leonidas J Guibas “Shape segmentation using local slippage analysis” In Proceedings of the Eurographics Symposium on Geometry Processing, 2004, pp. 214–223
  13. “Vrkitchen: an interactive 3d virtual environment for task-oriented learning” In arXiv preprint arXiv:1903.05757, 2019
  14. “GAPartNet: Cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 7081–7091
  15. “CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 21201–21210
  16. Jonathan Ho, Ajay Jain and Pieter Abbeel “Denoising diffusion probabilistic models” In Advances in neural information processing systems (NeurIPS) 33, 2020, pp. 6840–6851
  17. “Learning to predict part mobility from a single static snapshot” In ACM Transactions on Graphics (TOG) 36.6, 2017, pp. 1–13
  18. “Active articulation model estimation through interactive perception” In Proceedings of the International Conference on Robotics and Automation (ICRA), 2015, pp. 3305–3312
  19. Ruizhen Hu, Manolis Savva and Oliver Kaick “Functionality representations and applications for shape analysis” In Computer Graphics Forum 37.2, 2018, pp. 603–624
  20. “Deep residual learning for image recognition” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778
  21. “Interaction context (ICON) towards a geometric functionality descriptor” In ACM Transactions on Graphics (TOG) 34.4, 2015, pp. 1–12
  22. Trimble Inc. “3D Warehouse” Accessed: 2024-01-22, 2017 URL: https://3dwarehouse.sketchup.com/
  23. Trimble Inc. “SketchUp” Accessed: 2024-01-22, 2017 URL: https://www.sketchup.com/
  24. Zhenyu Jiang, Cheng-Chun Hsu and Yuke Zhu “Ditto: Building digital twins of articulated objects from interaction” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 5616–5626
  25. “ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory” In Proceedings of the International Conference on Robotics and Automation (ICRA), 2021, pp. 13670–13677
  26. “OPD: Single-view 3D openable part detection” In Proceedings of the European Conference on Computer Vision (ECCV), 2022, pp. 410–426
  27. “A primal-dual framework for real-time dense RGB-D scene flow” In Proceedings of the International Conference on Robotics and Automation (ICRA), 2015, pp. 98–104
  28. Michael Kazhdan, Matthew Bolitho and Hugues Hoppe “Poisson surface reconstruction” In Proceedings of the Eurographics Symposium on Geometry Processing 7, 2006
  29. “AI2-THOR: An interactive 3d environment for visual AI” In arXiv preprint arXiv:1712.05474, 2017
  30. “NAP: Neural 3D articulation prior” In arXiv preprint arXiv:2305.16315, 2023
  31. “Learning the 3D Fauna of the Web” In arXiv preprint arXiv:2401.02400, 2024
  32. Jiayi Liu, Ali Mahdavi-Amiri and Manolis Savva “PARIS: Part-level reconstruction and motion analysis for articulated objects” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 352–363
  33. David G Lowe “Distinctive image features from scale-invariant keypoints” In International journal of computer vision 60, 2004, pp. 91–110
  34. “Semi-weakly supervised object kinematic motion prediction” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 21726–21735
  35. “CAGE: Controllable Articulation GEneration” In arXiv preprint arXiv:2312.09570, 2023
  36. “Mobility fitting using 4D RANSAC” In Computer Graphics Forum 35.5, 2016, pp. 79–88
  37. “GART: Gaussian articulated template models” In arXiv preprint arXiv:2311.16099, 2023
  38. “Category-level articulated object pose estimation” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 3706–3715
  39. “AKB-48: A real-world articulated object knowledge base” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 14809–14818
  40. “Toward real-world category-level articulation pose estimation” In IEEE Transactions on Image Processing 31, 2022, pp. 1072–1083
  41. “Self-supervised category-Level articulated object pose estimation with part-Level SE(3) Equivariance” In Proceedings of the International Conference on Learning Representations (ICLR), 2023
  42. “Animatable gaussians: Learning pose-dependent gaussian maps for high-fidelity human avatar modeling” In arXiv preprint arXiv:2311.16096, 2023
  43. Roberto Martín-Martín, Clemens Eppner and Oliver Brock “The RBO dataset of articulated objects and interactions” In The International Journal of Robotics Research 38.9, 2019, pp. 1013–1019
  44. “Where2act: From pixels to actions for articulated 3D objects” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021, pp. 6813–6823
  45. “A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021, pp. 12981–12991
  46. “NeRF: Representing scenes as neural radiance fields for view synthesis” In Communications of the ACM 65.1, 2021, pp. 99–106
  47. Andreas Mueller “Modern robotics: Mechanics, planning, and control [bookshelf]” In IEEE Control Systems Magazine 39.6, 2019, pp. 100–102
  48. “Illustrating how mechanical assemblies work” In ACM Transactions on Graphics (TOG) 29.4, 2010, pp. 58
  49. “PartNet: A large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 909–918
  50. “MultiScan: Scalable RGBD scanning for 3D environments with articulated objects” In Advances in neural information processing systems (NeurIPS) 35, 2022, pp. 9058–9071
  51. “Deformable 3D shape registration based on local similarity transforms” In Computer Graphics Forum 30.5, 2011, pp. 1493–1502
  52. “Articulated object reconstruction and markerless motion capture from depth video” In Computer Graphics Forum 27.2, 2008, pp. 399–408
  53. “Virtualhome: Simulating household activities via programs” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8494–8502
  54. “Articulated Objects: From Detection to Manipulation—Survey” In International Conference on Intelligent Autonomous Systems, 2022, pp. 495–508
  55. “Habitat 3.0: A co-habitat for humans, avatars and robots” In arXiv preprint arXiv:2310.13724, 2023
  56. Shengyi Qian and David F Fouhey “Understanding 3d object interaction from a single image” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2023, pp. 21753–21763
  57. “Understanding 3D object articulation in internet videos” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 1599–1609
  58. “PointNet: deep learning on point sets for 3D classification and segmentation. 2017” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 77–85
  59. “3dgs-avatar: Animatable avatars via deformable 3d gaussian splatting” In arXiv preprint arXiv:2312.09228, 2023
  60. “Pointnet++: Deep hierarchical feature learning on point sets in a metric space” In Advances in neural information processing systems (NeurIPS) 30, 2017
  61. “As-rigid-as-possible surface modeling” In Proceedings of the Eurographics Symposium on Geometry Processing 4, 2007, pp. 109–116
  62. Shih-Yang Su, Timur Bagautdinov and Helge Rhodin “Danbo: Disentangled articulated neural body representations via graph neural networks” In European Conference on Computer Vision, 2022, pp. 107–124 Springer
  63. Yahao Shi, Xinyu Cao and Bin Zhou “Self-Supervised Learning of Part Mobility from Point Cloud Sequence” In Computer Graphics Forum 40.6, 2021, pp. 104–116
  64. “Mobility-trees for indoor scenes manipulation” In Computer Graphics Forum 33.1, 2014, pp. 2–14
  65. “OPDMulti: Openable Part Detection for Multiple Objects” In arXiv preprint arXiv:2303.14087, 2023
  66. “Habitat: A platform for embodied ai research” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019, pp. 9339–9347
  67. “A-NeRF: Articulated neural radiance fields for learning human shape, appearance, and pose” In Advances in neural information processing systems (NeurIPS) 34, 2021, pp. 12278–12291
  68. “CLA-NeRF: Category-level articulated neural radiance field” In Proceedings of the International Conference on Robotics and Automation (ICRA), 2022, pp. 8454–8460
  69. Shinji Umeyama “Least-squares estimation of transformation parameters between two point patterns” In IEEE Transactions on Pattern Analysis & Machine Intelligence 13.04, 1991, pp. 376–380
  70. Chung-Yi Weng, Brian Curless and Ira Kemelmacher-Shlizerman “Photo wake-up: 3D character animation from a single photo” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5908–5917
  71. “Self-supervised neural articulated shape and appearance models” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15816–15826
  72. “Magicpony: Learning articulated 3D animals in the wild” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 8792–8802
  73. “Arah: Animatable volume rendering of articulated human sdfs” In European conference on computer vision, 2022, pp. 1–19 Springer
  74. “Shape2Motion: Joint analysis of motion parts and attributes from 3D shapes” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 8876–8884
  75. “SAPIEN: A SimulAted Part-based Interactive ENvironment” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11097–11107
  76. “Joint-aware manipulation of deformable models” In ACM Transactions on Graphics (TOG) 28.3, 2009, pp. 1–9
  77. “Deep part induction from articulated object pairs” In ACM Transactions on Graphics (TOG) 37.6, 2018, pp. 1–15
  78. “RPM-Net: recurrent prediction of motion and parts from point cloud” In ACM Transactions on Graphics (TOG) 38.6, 2019, pp. 1–15
  79. “Space-time co-segmentation of articulated point cloud sequences” In Computer Graphics Forum 35.2, 2016, pp. 419–429
  80. “ARTIC3D: Learning robust articulated 3d shapes from noisy web image collections” In Advances in neural information processing systems (NeurIPS) 36, 2024
  81. “Banmo: Building animatable 3d neural models from many casual videos” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 2863–2873
  82. “Object Wake-up: 3D Object Rigging from a Single Image” In Proceedings of the European Conference on Computer Vision (ECCV), 2022, pp. 311–327 Springer
  83. “Visual identification of articulated object parts” In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021, pp. 2443–2450
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets