Anymate: A Dataset and Baselines for Learning 3D Object Rigging (2505.06227v1)

Abstract: Rigging and skinning are essential steps to create realistic 3D animations, often requiring significant expertise and manual effort. Traditional attempts at automating these processes rely heavily on geometric heuristics and often struggle with objects of complex geometry. Recent data-driven approaches show potential for better generality, but are often constrained by limited training data. We present the Anymate Dataset, a large-scale dataset of 230K 3D assets paired with expert-crafted rigging and skinning information -- 70 times larger than existing datasets. Using this dataset, we propose a learning-based auto-rigging framework with three sequential modules for joint, connectivity, and skinning weight prediction. We systematically design and experiment with various architectures as baselines for each module and conduct comprehensive evaluations on our dataset to compare their performance. Our models significantly outperform existing methods, providing a foundation for comparing future methods in automated rigging and skinning. Code and dataset can be found at https://anymate3d.github.io/.

Anymate: A Dataset and Baselines for Learning 3D Object Rigging

The paper discusses a significant contribution to automated 3D object rigging and skinning through the introduction of the Anymate Dataset. With an emphasis on enhancing the pipeline for 3D animation, this paper focuses on solving cumbersome manual tasks by leveraging data-driven methods. The Anymate Dataset is substantial, encompassing 230,716 3D assets complete with expert-crafted rigging and skinning information, thereby providing a vast resource 70 times larger than previously available datasets.

Dataset Composition and Utility

The primary utility of the Anymate Dataset lies in its scale and diversity, which spans comprehensive categories ranging from humanoid and animal characters to everyday objects like furniture. This variety and volume enable more robust training of machine learning models designed for 3D object auto-rigging—a task traditionally characterized by labor-intensive manual rigging processes. The dataset curates assets from the Objaverse-XL dataset, simplifying the synchronization of 3D meshes, bone skeletons, and skinning weights into a standardized format ready for machine learning applications.

Rigging and Skinning Framework

Leveraging the dataset, the paper proposes a structured, learning-based framework to automate rigging and skinning processes. The model operates through three sequential tasks: joint prediction, connectivity prediction, and skinning weight prediction, establishing the foundational elements necessary to animate 3D models.

1. Joint Prediction: This module employs both regression and diffusion-based architectures to predict the spatial positions of skeletal joints, handling the inherent variability in the number of target joints across different assets. The regression approach shows a particular advantage in scalability, while diffusion models offer flexibility in specifying joint quantities.
2. Connectivity Prediction: Building upon joint predictions, the connectivity module constructs a kinematic skeleton by predicting connections between joint pairs. Two architectural variants—token-conditioned and two-branch—are assessed for this task. The connectivity module is pivotal in ensuring the resultant skeleton can be correctly animated by the system.
3. Skinning Weight Prediction: This task involves estimating the skinning weights for each vertex relative to the derived bone structures. The architecture here emphasizes utilizing cross-attention mechanisms to inform predictions, supported by a cosine similarity loss function during training.

Experimental Evaluation

The framework's performance advances significantly over traditional and previous learning-based methods, such as RigNet and Pinocchio, reflecting improvements in accuracy and computation efficiency. The experimental evaluations confirm the framework's capability to generalize across diverse object types, supported by robust quantitative results across key metrics including Chamfer Distance, Earth Mover’s Distance, and precision-recall analytics.

Implications and Future Directions

The implications of this research are multifaceted. Practically, it facilitates the creation of realistic animations with reduced manual intervention. Theoretically, it showcases the potential impact of large-scale training data and advanced machine learning architectures on automated rigging and skinning. Moreover, the framework sets a benchmark for future research, encouraging refinements and adaptations aimed at further improving animation model generality and efficiency.

The future trajectory in AI-driven 3D content creation could take advantage of the insights provided by this paper, moving towards more intricate deformation models, enhancing real-time applications in AR/VR, and exploring alternative representations for complex materials and nuanced animations. The availability of the Anymate Dataset and open-source implementations supports continued innovation, offering a resource for both academic research and industrial application.