Universal Humanoid Motion Representations for Physics-Based Control (2310.04582v2)
Abstract: We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. Due to the high dimensionality of humanoids and the inherent difficulties in reinforcement learning, prior methods have focused on learning skill embeddings for a narrow range of movement styles (e.g. locomotion, game characters) from specialized motion datasets. This limited scope hampers their applicability in complex tasks. We close this gap by significantly increasing the coverage of our motion representation space. To achieve this, we first learn a motion imitator that can imitate all of human motion from a large, unstructured motion dataset. We then create our motion representation by distilling skills directly from the imitator. This is achieved by using an encoder-decoder structure with a variational information bottleneck. Additionally, we jointly learn a prior conditioned on proprioception (humanoid's own pose and velocities) to improve model expressiveness and sampling efficiency for downstream tasks. By sampling from the prior, we can generate long, stable, and diverse human motions. Using this latent space for hierarchical RL, we show that our policies solve tasks using human-like behavior. We demonstrate the effectiveness of our motion representation by solving generative tasks (e.g. strike, terrain traversal) and motion tracking using VR controllers.
- Imitate and repurpose: Learning reusable robot movement skills from human and animal behaviors. March 2022.
- Physics-based motion capture imitation with deep reinforcement learning. Proceedings - MIG 2018: ACM SIGGRAPH Conference on Motion, Interaction, and Games, 2018.
- CMU. CMU graphics lab motion capture database. http://mocap.cs.cmu.edu/, 2002.
- Deep whole-body control: Learning a unified policy for manipulation and locomotion. October 2022.
- K Fukushima. Cognitron: a self-organizing multilayered neural network. Biol. Cybern., 20(3-4):121–136, November 1975.
- SuperTrack: motion tracking for physically simulated characters using supervised learning. ACM Trans. Graph., 40(6):1–13, December 2021.
- TM2T: Stochastic and tokenized modeling for the reciprocal generation of 3D human motions and texts. July 2022.
- Latent space policies for hierarchical reinforcement learning. April 2018.
- CoMic: Complementary task learning & mimicry for reusable skills. http://proceedings.mlr.press/v119/hasenclever20a/hasenclever20a.pdf. Accessed: 2023-2-13.
- Gaussian error linear units (GELUs). June 2016.
- MotionGPT: Human motion as a foreign language. June 2023.
- Task-Generic hierarchical human motion prior using VAEs. June 2021.
- Character controllers using motion VAEs. ACM Trans. Graph., 39(4):12, 2020.
- Discrete-Valued neural communication. July 2021.
- MoSh: Motion and shape capture from sparse markers. ACM Trans. Graph., 33(6), 2014.
- SMPL: A skinned multi-person linear model. ACM Trans. Graph., 34(6), 2015.
- PoseGPT: Quantization-based 3D human motion generation and forecasting. October 2022.
- CARL: Controllable agent with reinforcement learning for quadruped locomotion. May 2020a.
- 3D human motion estimation via motion compression and refinement. Technical report, 2020b.
- Dynamics-regulated kinematic policy for egocentric pose estimation. NeurIPS, 34:25019–25032, 2021.
- Embodied scene-aware human pose estimation. NeurIPS, June 2022.
- Perpetual humanoid control for real-time simulated avatars. May 2023.
- AMASS: Archive of motion capture as surface shapes. Proceedings of the IEEE International Conference on Computer Vision, 2019-Octob:5441–5450, 2019.
- Isaac gym: High performance GPU-based physics simulation for robot learning. August 2021.
- Neural probabilistic motor primitives for humanoid control. Technical report, 2018.
- Catch and carry: Reusable neural controllers for Vision-Guided Whole-Body tasks. ACM Trans. Graph., 39(4), 2020.
- Rectified linear units improve restricted boltzmann machines.
- DeepMimic. ACM Trans. Graph., 37(4):1–14, 2018.
- MCP: Learning composable hierarchical control with multiplicative compositional policies. May 2019.
- AMP: Adversarial motion priors for stylized Physics-Based character control. ACM Trans. Graph., (4):1–20, April 2021.
- ASE: Large-Scale reusable adversarial skill embeddings for physically simulated characters. May 2022.
- Action-Conditioned 3D human motion synthesis with transformer VAE. April 2021.
- HuMoR: 3D human motion model for robust pose estimation. May 2021.
- Trace and pace: Controllable pedestrian animation via guided trajectory diffusion. April 2023.
- DiffMimic: Efficient motion mimicking with differentiable physics. April 2023.
- A reduction of imitation learning and structured prediction to no-regret online learning. November 2010.
- Learning to walk in minutes using massively parallel deep reinforcement learning. September 2021.
- Policy distillation. November 2015.
- Kickstarting deep reinforcement learning. March 2018.
- Proximal policy optimization algorithms. Technical report, 2017.
- CALM: Conditional adversarial latent models for directable virtual characters.
- Neural discrete representation learning. Adv. Neural Inf. Process. Syst., 2017-Decem(Nips):6307–6316, 2017.
- Estimating egocentric 3D human pose in global space. April 2021.
- UniCon: Universal neural controller for physics-based character motion. arXiv, 2020.
- QuestSim: Human motion tracking from sparse sensors with simulated avatars. September 2022.
- A scalable approach to control diverse behaviors for physically simulated characters. ACM Trans. Graph., 39(4), 2020.
- Physics-based character controllers using conditional VAEs. ACM Trans. Graph., 41(4):1–12, July 2022.
- ControlVAE: Model-based learning of generative controllers for physics-based characters. October 2022.
- Ye Yuan and Kris Kitani. Residual force control for agile human behavior imitation and extended motion synthesis. (NeurIPS), June 2020a.
- Ye Yuan and Kris Kitani. DLow: Diversifying latent flows for diverse human motion prediction. Lect. Notes Comput. Sci., 12354 LNCS:346–364, 2020b.
- SimPoE: Simulated character control for 3D human pose estimation. CVPR, April 2021.
- PhysDiff: Physics-guided human motion diffusion model. arXiv [cs.CV], December 2022.
- Learning physically simulated tennis skills from broadcast videos. ACM Trans. Graph., 42(4):1–14, August 2023a.
- MotionGPT: Finetuned LLMs are General-Purpose motion generators. June 2023b.
- On the continuity of rotation representations in neural networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June:5738–5746, 2019.
- Neural categorical priors for Physics-Based character control.