Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild (2304.10888v3)

Published 21 Apr 2023 in cs.RO and cs.AI

Abstract: Recently, reinforcement learning has become a promising and polular solution for robot legged locomotion. Compared to model-based control, reinforcement learning based controllers can achieve better robustness against uncertainties of environments through sim-to-real learning. However, the corresponding learned gaits are in general overly conservative and unatural. In this paper, we propose a new framework for learning robust, agile and natural legged locomotion skills over challenging terrain. We incorporate an adversarial training branch based on real animal locomotion data upon a teacher-student training pipeline for robust sim-to-real transfer. Empirical results on both simulation and real world of a quadruped robot demonstrate that our proposed algorithm enables robustly traversing challenging terrains such as stairs, rocky ground and slippery floor with only proprioceptive perception. Meanwhile, the gaits are more agile, natural, and energy efficient compared to the baselines. Both qualitative and quantitative results are presented in this paper.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Dynamic locomotion in the mit cheetah 3 through convex model-predictive control. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 1–9. IEEE, 2018.
  2. Gait and trajectory optimization for legged systems through phase-based end-effector parameterization. IEEE Robotics and Automation Letters, 3(3):1560–1567, 2018.
  3. Dynamically diverse legged locomotion for rough terrain. In 2009 IEEE International Conference on Robotics and Automation, pages 1607–1608. IEEE, 2009.
  4. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, pages 91–100. PMLR, 2022.
  5. Learning quadrupedal locomotion over challenging terrain. Science robotics, 5(47):eabc5986, 2020.
  6. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021.
  7. Robust recovery controller for a quadrupedal robot using deep reinforcement learning. arXiv preprint arXiv:1901.07517, 2019.
  8. Learning vision-guided quadrupedal locomotion end-to-end with cross-modal transformers. arXiv preprint arXiv:2107.03996, 2021.
  9. Amp: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (TOG), 40(4):1–20, 2021.
  10. Adversarial motion priors make good substitutes for complex reward functions. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 25–32. IEEE, 2022.
  11. Learning agile robotic locomotion skills by imitating animals. arXiv preprint arXiv:2004.00784, 2020.
  12. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions On Graphics (TOG), 37(4):1–14, 2018.
  13. Adapting rapid motor adaptation for bipedal robots. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1161–1168. IEEE, 2022.
  14. Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion. IEEE Robotics and Automation Letters, 7(2):4630–4637, 2022.
  15. Emergent real-world robotic skills via unsupervised off-policy reinforcement learning. arXiv preprint arXiv:2004.12974, 2020.
  16. Dynamics randomization revisited: A case study for quadrupedal locomotion. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 4955–4961. IEEE, 2021.
  17. Imitate and repurpose: Learning reusable robot movement skills from human and animal behaviors. arXiv preprint arXiv:2203.17138, 2022.
  18. Learning to jump from pixels. arXiv preprint arXiv:2110.15344, 2021.
  19. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7(62):eabk2822, 2022.
  20. Rapid locomotion via reinforcement learning. arXiv preprint arXiv:2205.02824, 2022.
  21. Daydreamer: World models for physical robot learning. arXiv preprint arXiv:2206.14176, 2022.
  22. A walk in the park: Learning to walk in 20 minutes with model-free reinforcement learning. arXiv preprint arXiv:2208.07860, 2022.
  23. Iterative reinforcement learning based design of dynamic locomotion skills for cassie. arXiv preprint arXiv:1903.09537, 2019.
  24. Blind bipedal stair traversal via sim-to-real reinforcement learning. arXiv preprint arXiv:2105.08328, 2021.
  25. Sim-to-real learning of all common bipedal gaits via periodic reward composition. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 7309–7315. IEEE, 2021.
  26. Adapting human motion for the control of a humanoid robot. In Proceedings 2002 IEEE international conference on robotics and automation (Cat. No. 02CH37292), volume 2, pages 1390–1397. IEEE, 2002.
  27. Dynamic imitation in a humanoid robot through nonparametric probabilistic inference. In Robotics: science and systems, pages 199–206. Cambridge, MA, 2006.
  28. On human motion imitation by humanoid robot. In 2008 IEEE International conference on robotics and automation, pages 2697–2704. IEEE, 2008.
  29. Controlling humanoid robots with human motion data: Experimental validation. In 2010 10th IEEE-RAS International Conference on Humanoid Robots, pages 504–510. IEEE, 2010.
  30. Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Transactions On Graphics (TOG), 41(4):1–17, 2022.
  31. Martin L Puterman. Markov decision processes. Handbooks in operations research and management science, 2:331–434, 1990.
  32. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99–134, 1998.
  33. Mode-adaptive neural networks for quadruped motion control. ACM Transactions on Graphics (TOG), 37(4):1–11, 2018.
  34. Isaac gym: High performance gpu-based physics simulation for robot learning, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Yikai Wang (78 papers)
  2. Zheyuan Jiang (4 papers)
  3. Jianyu Chen (69 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now