Papers
Topics
Authors
Recent
2000 character limit reached

Motion Capture Dataset for Practical Use of AI-based Motion Editing and Stylization

Published 15 Jun 2023 in cs.CV and cs.AI | (2306.08861v2)

Abstract: In this work, we proposed a new style-diverse dataset for the domain of motion style transfer. The motion dataset uses an industrial-standard human bone structure and thus is industry-ready to be plugged into 3D characters for many projects. We claim the challenges in motion style transfer and encourage future work in this domain by releasing the proposed motion dataset both to the public and the market. We conduct a comprehensive study on motion style transfer in the experiment using the state-of-the-art method, and the results show the proposed dataset's validity for the motion style transfer task.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Unpaired motion style transfer from video to animation. ACM Transactions on Graphics (TOG) 39, 4 (2020), 64–1.
  2. Ijaz Akhter and Michael J. Black. 2015. Pose-conditioned joint angle limits for 3D human pose reconstruction. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1446–1455. https://doi.org/10.1109/CVPR.2015.7298751
  3. Emotion from Motion. In Proceedings of the Graphics Interface 1996 Conference, May 22-24, 1996, Toronto, Ontario, Canada. Canadian Human-Computer Communications Society, 222–229. http://graphicsinterface.org/wp-content/uploads/gi1996-26.pdf
  4. Inc. BandaiNamcoResearch. 2023. Mirai Komachi Official Page. https://www.miraikomachi.com/
  5. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 1 (2021), 172–186. https://doi.org/10.1109/TPAMI.2019.2929257
  6. A Virtual Reality Dance Training System Using Motion Capture Technology. IEEE Transactions on Learning Technologies 4, 2 (2011), 187–195. https://doi.org/10.1109/TLT.2010.27
  7. Graphics Lab CMU. 2000. CMU Graphics Lab Motion Capture Database. http://mocap.cs.cmu.edu/
  8. Gutemberg Guerra-Filho. 2005. Optical Motion Capture: Theory and Implementation. RITA 12 (2005), 61–90.
  9. Robust Motion In-Betweening. 39, 4 (2020).
  10. A Deep Learning Framework for Character Motion Synthesis and Editing. ACM Trans. Graph. 35, 4, Article 138 (jul 2016), 11 pages. https://doi.org/10.1145/2897824.2925975
  11. Learning Motion Manifolds with Convolutional Autoencoders. In SIGGRAPH Asia 2015 Technical Briefs (Kobe, Japan) (SA ’15). Association for Computing Machinery, New York, NY, USA, Article 18, 4 pages. https://doi.org/10.1145/2820903.2820918
  12. Sleight of Hand: Perception of Finger Motion from Reduced Marker Sets. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (Costa Mesa, California) (I3D ’12). Association for Computing Machinery, New York, NY, USA, 79–86. https://doi.org/10.1145/2159616.2159630
  13. Xun Huang and Serge Belongie. 2017. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision. 1501–1510.
  14. Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 7 (jul 2014), 1325–1339.
  15. Motion Puzzle: Arbitrary Motion Style Transfer by Body Part. ACM Transactions on Graphics (TOG) (2022).
  16. AI Golf: Golf Swing Analysis Tool for Self-Training. IEEE Access 10 (2022), 106286–106295. https://doi.org/10.1109/ACCESS.2022.3210261
  17. AI Coach: A Motor Skill Training System Using Motion Discrepancy Detection. In Proceedings of the Augmented Humans International Conference 2023 (Glasgow, United Kingdom) (AHs ’23). Association for Computing Machinery, New York, NY, USA, 179–189. https://doi.org/10.1145/3582700.3582710
  18. Modeling Style and Variation in Human Motion. In Eurographics/ ACM SIGGRAPH Symposium on Computer Animation, MZoran Popovic and Miguel Otaduy (Eds.). The Eurographics Association. https://doi.org/10.2312/SCA/SCA10/021-030
  19. The KIT whole-body human motion database. In 2015 International Conference on Advanced Robotics (ICAR). 329–336. https://doi.org/10.1109/ICAR.2015.7251476
  20. Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases. arXiv preprint arXiv:2201.04439 (2022).
  21. Real-Time Sign Language Detection Using Human Pose Estimation. In Computer Vision – ECCV 2020 Workshops, Adrien Bartoli and Andrea Fusiello (Eds.). Springer International Publishing, Cham, 237–248.
  22. Movella. n.d. Xsens MVN MoCap Systems. https://www.vicon.com/
  23. Efficient and Robust Annotation of Motion Capture Data. In Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (New Orleans, Louisiana) (SCA ’09). Association for Computing Machinery, New York, NY, USA, 17–26. https://doi.org/10.1145/1599470.1599473
  24. Diverse Motion Stylization for Multiple Style Domains via Spatial-Temporal Graph-Based Generative Model. Proceedings of the ACM on Computer Graphics and Interactive Techniques 4, 3 (2021), 1–17.
  25. Design of a video game for rehabilitation using motion capture, EMG analysis and virtual reality. In 2016 International Conference on Electronics, Communications and Computers (CONIELECOMP). 198–204. https://doi.org/10.1109/CONIELECOMP.2016.7438575
  26. Xsens MVN: Full 6DOF human motion tracking using miniature inertial sensors. Xsens Motion Technol. BV Tech. Rep. 3 (01 2009).
  27. InterPoser: Visualizing Interpolated Movements for Bouldering Training. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312779
  28. Nikolaus F. Troje. 2002. Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. Journal of vision 2 5 (2002), 371–87.
  29. Fourier Principles for Emotion-Based Human Figure Animation. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’95). Association for Computing Machinery, New York, NY, USA, 91–96. https://doi.org/10.1145/218380.218419
  30. Vicon. n.d. Vicon Motion Systems. https://www.vicon.com/
  31. A Cyclic Consistency Motion Style Transfer Method Combined with Kinematic Constraints. Journal of Sensors 2021 (06 2021), 1–17. https://doi.org/10.1155/2021/5548614
  32. Deep High-Resolution Representation Learning for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 10 (2021), 3349–3364. https://doi.org/10.1109/TPAMI.2020.2983686
  33. Autoregressive Stylized Motion Synthesis with Generative Flow. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13607–13607. https://doi.org/10.1109/CVPR46437.2021.01340
  34. Realtime Style Transfer for Unlabeled Heterogeneous Human Motion. ACM Trans. Graph. 34, 4 (jul 2015). https://doi.org/10.1145/2766999
  35. M Ersin Yumer and Niloy J Mitra. 2016. Spectral style transfer for human motion between independent actions. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–8.
  36. MuscleRehab: Improving Unsupervised Physical Rehabilitation by Monitoring and Visualizing Muscle Engagement. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 33, 14 pages. https://doi.org/10.1145/3526113.3545705
Citations (7)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up