Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Two-Person Interaction Augmentation with Skeleton Priors (2404.05490v2)

Published 8 Apr 2024 in cs.CV

Abstract: Close and continuous interaction with rich contacts is a crucial aspect of human activities (e.g. hugging, dancing) and of interest in many domains like activity recognition, motion prediction, character animation, etc. However, acquiring such skeletal motion is challenging. While direct motion capture is expensive and slow, motion editing/generation is also non-trivial, as complex contact patterns with topological and geometric constraints have to be retained. To this end, we propose a new deep learning method for two-body skeletal interaction motion augmentation, which can generate variations of contact-rich interactions with varying body sizes and proportions while retaining the key geometric/topological relations between two bodies. Our system can learn effectively from a relatively small amount of data and generalize to drastically different skeleton sizes. Through exhaustive evaluation and comparison, we show it can generate high-quality motions, has strong generalizability and outperforms traditional optimization-based methods and alternative deep learning solutions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (72)
  1. Skeleton-aware networks for deep motion retargeting. ACM Trans. Graph., 39(4), 2020.
  2. Robust physics-based motion retargeting with realistic body shapes. Computer Graphics Forum, 37(8):81–92, 2018.
  3. Carnegie Mellon Graphics Lab. CMU graphics lab motion capture database. http://mocap.cs.cmu.edu, accessed 2022.
  4. Performance animation from low-dimensional control signals. ACM Trans. Graph., 24(3):686–696, 2005.
  5. Dynamic Future Net: Diversified Human Motion Generation, page 2131–2139. Association for Computing Machinery, New York, NY, USA, 2020.
  6. Interaction transformer for human reaction generation. IEEE Transactions on Multimedia, pages 1–13, 2023.
  7. Automatic detection of human interactions from rgb-d data for social activity classification. In IEEE Int. Symposium on Robot and Human Interactive Communication (RO-MAN), pages 871–876, 2017.
  8. Basar:black-box attack on skeletal action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  9. Remos: 3d motion-conditioned reaction synthesis for two-person interactions, 2024.
  10. Michael Gleicher. Retargetting motion to new characters. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, page 33–42, New York, NY, USA, 1998. Association for Computing Machinery.
  11. Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding. Computer Graphics Forum, 2022.
  12. Action2motion: Conditioned generation of 3d human motions. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2021–2029, 2020.
  13. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5152–5161, 2022a.
  14. Multi-person extreme motion prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13053–13064, 2022b.
  15. Real-time motion retargeting to highly varied user-created morphologies. ACM Trans. Graph., 27(3):1–11, 2008.
  16. Indexing and retrieving motions of characters in close contact. IEEE Transactions on Visualization and Computer Graphics, 15(3):481–492, 2009.
  17. Edmond S. L. Ho and Hubert P. H. Shum. Motion adaptation for humanoid robots in constrained environments. In 2013 IEEE International Conference on Robotics and Automation, pages 3813–3818, 2013.
  18. Spatial relationship preserving character motion adaptation. ACM Trans. Graph., 29(4), 2010.
  19. A multi-resolution approach for adapting close character interaction. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, page 97–106, New York, NY, USA, 2014. Association for Computing Machinery.
  20. A deep learning framework for character motion synthesis and editing. ACM Trans. Graph., 35(4), 2016.
  21. Spatial temporal graph attention network for skeleton-based action recognition. arXiv preprint arXiv:2208.08599, 2022.
  22. Capturing and inferring dense full-body human-scene contact. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 13274–13285, 2022.
  23. Full-body articulated human-object interaction. In ICCV, 2023.
  24. Interactive multi-character motion retargeting. Computer Animation and Virtual Worlds, 32(3-4):e2015, 2021.
  25. Hierarchically decomposed graph convolutional networks for skeleton-based action recognition. arXiv preprint arXiv:2208.10741, 2022.
  26. Interactive character animation by learning multi-objective control. ACM Trans. Graph., 37(6), 2018.
  27. Same: Skeleton-agnostic motion embedding for character animation. In SIGGRAPH Asia 2023 Conference Papers, pages 1–11, 2023.
  28. Dynamic multiscale graph neural networks for 3d skeleton based human motion prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 214–223, 2020.
  29. Intergen: Diffusion-based multi-human motion generation under complex interactions. International Journal of Computer Vision, 2024.
  30. Character controllers using motion vaes. ACM Trans. Graph., 39(4), 2020.
  31. A quadruple diffusion convolutional recurrent network for human motion prediction. IEEE Transactions on Circuits and Systems for Video Technology, 31(9):3417–3432, 2021a.
  32. Gan-based reactive motion synthesis with class-aware discriminators for human-human interaction. Computers and Graphics, 2021b.
  33. A survey on deep learning for skeleton-based human animation. Computer Graphics Forum, 41(1):122–157, 2022.
  34. Motion templates for automatic classification and retrieval of motion capture data. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, page 137–146, Goslar, DEU, 2006. Eurographics Association.
  35. Documentation Mocap Database HDM05. Technical Report CG-2007-2, Universität Bonn, 2007. ISSN: 1610-8892.
  36. Multi-character motion retargeting for large-scale transformations. In Advances in Computer Graphics, pages 94–106, Cham, 2019. Springer International Publishing.
  37. Trajectory-aware body interaction transformer for multi-person pose forecasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17121–17130, 2023.
  38. Robust marker trajectory repair for mocap using kinematic reference. In Motion, Interaction and Games, New York, NY, USA, 2019. Association for Computing Machinery.
  39. Action-conditioned 3D human motion synthesis with transformer VAE. In Proc. International Conference on Computer Vision (ICCV), pages 10985–10995, 2021.
  40. Hierarchical generation of human-object interactions with diffusion probabilistic models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15061–15073, 2023.
  41. UT-Interaction Dataset, ICPR contest on Semantic Description of Human Activities (SDHA). http://cvrc.ece.utexas.edu/SDHA2010/Human_Interaction.html, 2010.
  42. NTU RGB+D: A large scale dataset for 3D human activity analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1010–1019, 2016.
  43. Interaction-based human activity comparison. IEEE Transactions on Visualization and Computer Graphics, 26(8):115673–115684, 2020.
  44. Real-time posture reconstruction for microsoft kinect. IEEE Transactions on Cybernetics, 43(5):1357–1369, 2013.
  45. Neural animation layering for synthesizing martial arts movements. ACM Trans. Graph., 40(4), 2021.
  46. Human action recognition from various data modalities: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–20, 2022.
  47. A physically-based motion retargeting filter. ACM Trans. Graph., 24(1):98–117, 2005.
  48. Role-aware interaction generation from textual description. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15999–16009, 2023.
  49. Rsmt: Real-time stylized motion transition for characters. In ACM SIGGRAPH, pages 1–10, 2023.
  50. Social diffusion: Long-term multiple human motion anticipation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9601–9611, 2023.
  51. Human motion diffusion model. arXiv preprint arXiv:2209.14916, 2022.
  52. A variational u-net for motion retargeting. Computer Animation and Virtual Worlds, 31(4-5):e1947, 2020.
  53. Contact-aware retargeting of skinned motion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9720–9729, 2021.
  54. An energy-driven motion planning method for two distant postures. IEEE transactions on visualization and computer graphics, 21(1):18–30, 2015.
  55. Understanding the robustness of skeleton-based action recognition under adversarial attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021a.
  56. Spatio-temporal manifold learning for human motions via long-horizon modeling. IEEE Transactions on Visualization and Computer Graphics, 27(1):216–227, 2021b.
  57. Defending black-box skeleton-based human activity classifiers, 2022a.
  58. Towards diverse and natural scene-aware 3d human motion synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20460–20469, 2022b.
  59. Skeleton-based action recognition via temporal-channel aggregation. arXiv preprint arXiv:2205.15936, 2022c.
  60. History repeats itself: Human motion prediction via motion attention. In ECCV, 2020.
  61. Autoregressive stylized motion synthesis with generative flow. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13607–13607, 2021.
  62. Control strategies for physically simulated characters performing two-player competitive sports. ACM Transactions on Graphics (TOG), 40(4):1–11, 2021.
  63. Regennet: Towards human action-reaction synthesis, 2024.
  64. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. In ICCV, 2023a.
  65. Stochastic multi-person 3d motion forecasting. In ICLR, 2023b.
  66. Transmomo: Invariance-driven unsupervised video motion retargeting. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  67. Human-aware object placement for visual environment reconstruction. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3959–3970, 2022.
  68. Two-person interaction detection using body-pose features and multiple instance learning. In 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pages 28–35, 2012.
  69. Kinematic motion retargeting via neural latent optimization for learning sign language. IEEE Robotics and Automation Letters, 7(2):4582–4589, 2022a.
  70. T2m-gpt: Generating human motion from textual descriptions with discrete representations, 2023a.
  71. Motiondiffuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022b.
  72. Simulation and retargeting of complex multi-character interactions. In ACM SIGGRAPH 2023 Conference Proceedings, New York, NY, USA, 2023b. Association for Computing Machinery.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube