Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape (2308.11737v2)

Published 22 Aug 2023 in cs.CV and cs.LG

Abstract: Accurately estimating the 3D pose and shape is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. However, research in this area is held back by the lack of a comprehensive and diverse dataset with high-quality 3D pose and shape annotations. In this paper, we propose Animal3D, the first comprehensive dataset for mammal animal 3D pose and shape estimation. Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model. All annotations were labeled and checked manually in a multi-stage process to ensure highest quality results. Based on the Animal3D dataset, we benchmark representative shape and pose estimation models at: (1) supervised learning from only the Animal3D data, (2) synthetic to real transfer from synthetically generated images, and (3) fine-tuning human pose and shape estimation models. Our experimental results demonstrate that predicting the 3D shape and pose of animals across species remains a very challenging task, despite significant advances in human pose estimation. Our results further demonstrate that synthetic pre-training is a viable strategy to boost the model performance. Overall, Animal3D opens new directions for facilitating future research in animal 3D pose and shape estimation, and is publicly available.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Scape: Shape completion and animation of people. ACM Trans. Graph., 24(3):408–416, jul 2005.
  2. Who left the dogs out: 3D animal reconstruction with expectation maximization in the loop. In ECCV, 2020.
  3. 3d multi-bodies: Fitting sets of plausible 3d human models to ambiguous image data. In Advances in neural information processing systems, 2020.
  4. Creatures great and smal: Recovering the shape and motion of animals from video. In Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part V 14, pages 3–19. Springer, 2019.
  5. Keep it smpl: Automatic estimation of 3d human pose and shape from a single image. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, pages 561–578. Springer, 2016.
  6. Cross-domain adaptation for animal pose estimation. In The IEEE International Conference on Computer Vision (ICCV), October 2019.
  7. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  8. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR, 2015.
  9. Stuart Geman. Statistical methods for tomographic image reconstruction. Bulletin of International Statistical Institute, 4:5–21, 1987.
  10. HoloPose: Holistic 3D human reconstruction in-the-wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10884–10894, 2019.
  11. Partimagenet: A large, high-quality dataset of parts. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VIII, pages 128–145. Springer, 2022.
  12. Towards accurate marker-less human shape and pose estimation over time. In 2017 international conference on 3D vision (3DV), 2017.
  13. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence, 36(7):1325–1339, 2013.
  14. Total capture: A 3D deformation model for tracking faces, hands, and bodies. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8320–8329, 2018.
  15. Acinoset: A 3d pose estimation dataset and baseline models for cheetahs in the wild, 2021.
  16. End-to-end recovery of human shape and pose. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7122–7131, 2018.
  17. Pare: Part attention regressor for 3d human body estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11127–11137, 2021.
  18. Learning to reconstruct 3d human pose and shape via model-fitting in the loop. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2252–2261, 2019.
  19. Learning to reconstruct 3D human pose and shape via model-fitting in the loop. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2252–2261, 2019.
  20. Probabilistic modeling for human mesh recovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11605–11614, 2021.
  21. Articulation-aware canonical surface mapping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 452–461, 2020.
  22. Unite the People: Closing the loop between 3D and 2D human representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4704–4713, 2017.
  23. From synthetic to real: Unsupervised domain adaptation for animal pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1482–1491, June 2021.
  24. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  25. SMPL: A skinned multi-person linear model. In ACM Trans. Graphics (Proc. SIGGRAPH Asia), 2015.
  26. Pretraining boosts out-of-domain robustness for pose estimation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1859–1868, 2021.
  27. Monocular 3d human pose estimation in the wild using improved cnn supervision. In 2017 international conference on 3D vision (3DV), pages 506–516. IEEE, 2017.
  28. Monocular 3d human pose estimation in the wild using improved cnn supervision. In 2017 International Conference on 3D Vision (3DV), pages 506–516, 2017.
  29. Learning from synthetic animals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12386–12395, 2020.
  30. Learning from synthetic animals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  31. Neural body fitting: Unifying deep learning and model-based human pose and shape estimation. In 2018 international conference on 3D vision (3DV), 2018.
  32. Expressive body capture: 3D hands, face, and body from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10975–10985, 2019.
  33. Coarse-to-fine volumetric prediction for single-image 3D human pose. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2017.
  34. Learning to estimate 3D human pose and shape from a single color image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 459–468, 2018.
  35. Lcr-net: Localization-classification-regression for human pose. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3433–3441, 2017.
  36. Barc: Learning to regress 3d dog shape from images by exploiting breed information. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.
  37. Integral human pose regression. In Proceedings of the European conference on computer vision (ECCV), pages 529–545, 2018.
  38. Indirect deep structured learning for 3d human body shape and pose prediction. 2017.
  39. Self-supervised learning of motion capture. In Advances in neural information processing systems, pages 5236–5246, 2017.
  40. Recovering accurate 3d human pose in the wild using imus and a moving camera. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
  41. Monocular total capture: Posing face, body, and hands in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10965–10974, 2019.
  42. Beyond pascal: A benchmark for 3d object detection in the wild. In IEEE Winter Conference on Applications of Computer Vision, pages 75–82, 2014.
  43. Denserac: Joint 3d pose and shape estimation by dense render-and-compare. In Proceedings of the IEEE/CVF international conference on computer vision, October 2019.
  44. Lassie: Learning articulated shapes from sparse image ensemble via 3d part discovery. arXiv preprint arXiv:2207.03434, 2022.
  45. Ap-10k: A benchmark for animal pose estimation in the wild. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
  46. Towards 3d human pose estimation in the wild: a weakly-supervised approach. In Proceedings of the IEEE/CVF international conference on computer vision, pages 398–407, 2017.
  47. Three-d safari: Learning to estimate zebra pose, shape, and texture from images ”in the wild”. In International Conference on Computer Vision, Oct. 2019.
  48. Lions and tigers and bears: Capturing non-rigid, 3d, articulated shape from images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  49. 3d menagerie: Modeling the 3d shape and pose of animals. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6365–6373, 2017.
  50. 3d menagerie: Modeling the 3d shape and pose of animals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
Citations (15)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. Animal3D