3D Menagerie: Modeling the 3D shape and pose of animals (1611.07700v2)

Abstract: There has been significant work on learning realistic, articulated, 3D models of the human body. In contrast, there are few such models of animals, despite many applications. The main challenge is that animals are much less cooperative than humans. The best human body models are learned from thousands of 3D scans of people in specific poses, which is infeasible with live animals. Consequently, we learn our model from a small set of 3D scans of toy figurines in arbitrary poses. We employ a novel part-based shape model to compute an initial registration to the scans. We then normalize their pose, learn a statistical shape model, and refine the registrations and the model together. In this way, we accurately align animal scans from different quadruped families with very different shapes and poses. With the registration to a common template we learn a shape space representing animals including lions, cats, dogs, horses, cows and hippos. Animal shapes can be sampled from the model, posed, animated, and fit to data. We demonstrate generalization by fitting it to images of real animals including species not seen in training.

• The paper introduces a novel approach that leverages limited 3D scans of toy animal figurines to build a statistical shape model for four-legged mammals.
• It employs pose normalization and a part-based refinement using As-Rigid-As-Possible constraints to effectively align varying animal shapes.
• The resulting SMAL model generalizes well to real animal images and unseen species, offering promising applications in biology, neuroscience, and entertainment.

Overview

The paper presents an innovative approach to creating 3D models of four-legged mammals. Unlike existing human body models that utilize large datasets of 3D scans, animal modeling faces unique challenges due to their diversity and the impracticality of acquiring numerous 3D scans. To overcome this, researchers used a small dataset of 3D scans of toy animal figurines and applied techniques from human shape and pose modeling.

Model Development

The first step involved scanning a limited selection of animal figurines and using these scans to compute initial registrations. The researchers normalized the poses of these registrations and then learned a statistical shape model capturing the animals' shape variations. This model successfully aligned scans from different poses and produced a low-dimensional Euclidean shape space representing various animals including cats, dogs, horses, and hippos.

Model Refinement

Further refinements improved initial registrations through a novel part-based shape model and a model-free refinement using As-Rigid-As-Possible constraints. They then applied a pose normalization technique to these refined registrations and derived a shape space via principal component analysis. The result was a Skinned Multi-Animal Linear (SMAL) model that allowed for generation of new shapes and reposing.

Generalization and Applications

The SMAL model showed impressive generalization capabilities. Despite being trained on toy figurine scans, it was able to produce realistic fitting results to images of real animals and even to animals not seen during training. This has potential applications in biology, neuroscience, and entertainment, providing a rich basis for future research and development in animal shape and motion capture.

Conclusion

The creation of realistic 3D animal models from minimal data represents a significant advancement, showing that the strategies learned from human pose and shape modeling can be extended to animals. The paper's approach demonstrates a method to capture the variability in animal shapes across different species, offering promising directions for integrating 3D scans and image data to develop more comprehensive models.