Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Quadruped Locomotion Policies using Logical Rules (2107.10969v3)

Published 23 Jul 2021 in cs.RO and cs.AI

Abstract: Quadruped animals are capable of exhibiting a diverse range of locomotion gaits. While progress has been made in demonstrating such gaits on robots, current methods rely on motion priors, dynamics models, or other forms of extensive manual efforts. People can use natural language to describe dance moves. Could one use a formal language to specify quadruped gaits? To this end, we aim to enable easy gait specification and efficient policy learning. Leveraging Reward Machines~(RMs) for high-level gait specification over foot contacts, our approach is called RM-based Locomotion Learning~(RMLL), and supports adjusting gait frequency at execution time. Gait specification is enabled through the use of a few logical rules per gait (e.g., alternate between moving front feet and back feet) and does not require labor-intensive motion priors. Experimental results in simulation highlight the diversity of learned gaits (including two novel gaits), their energy consumption and stability across different terrains, and the superior sample-efficiency when compared to baselines. We also demonstrate these learned policies with a real quadruped robot. Video and supplementary materials: https://sites.google.com/view/rm-locomotion-learning/home

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Reward machines for vision-based robotic manipulation. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE.
  2. Diverse exploration for fast and safe policy improvement. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
  3. Learning a contact-adaptive controller for robust, efficient legged locomotion. In Conference on Robot Learning. PMLR.
  4. Multi-Agent Intention Progression with Reward Machines. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI, 215–222.
  5. Seeing-Eye Quadruped Navigation with Force Responsive Locomotion Control. In Conference on Robot Learning (CoRL). PMLR.
  6. Dynamic locomotion in the mit cheetah 3 through convex model-predictive control. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS).
  7. Inferring Probabilistic Reward Machines from Non-Markovian Reward Signals for Reinforcement Learning. In Proceedings of the International Conference on Automated Planning and Scheduling.
  8. Minimizing energy consumption leads to the emergence of gaits in legged robots. arXiv preprint arXiv:2111.01674.
  9. Gait and the energetics of locomotion in horses. Nature, 292(5820): 239–240.
  10. Using reward machines for high-level task specification and decomposition in reinforcement learning. In International Conference on Machine Learning, 2107–2116.
  11. Reward machines: Exploiting reward function structure in reinforcement learning. Journal of Artificial Intelligence Research, 73: 173–208.
  12. Policies modulating trajectory generators. In Conference on Robot Learning.
  13. Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion. IEEE Robotics and Automation Letters, 7(2): 4630–4637.
  14. Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control. arXiv preprint arXiv:1909.06586.
  15. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034.
  16. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470.
  17. Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning. PMLR.
  18. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics.
  19. Reward machines for cooperative multi-agent reinforcement learning. arXiv preprint arXiv:2007.01962.
  20. Learning agile robotic locomotion skills by imitating animals. arXiv preprint arXiv:2004.00784.
  21. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning.
  22. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
  23. Learning free gait transition for quadruped robots via phase-guided controller. IEEE Robotics and Automation Letters, 7(2).
  24. Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition. arXiv preprint arXiv:2011.01387.
  25. Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World. arXiv preprint arXiv:2110.05457.
  26. Sim-to-real: Learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804.10332.
  27. SayTap: Language to Quadrupedal Locomotion. arXiv preprint arXiv:2306.07580.
  28. Learning reward machines for partially observable reinforcement learning. Advances in Neural Information Processing Systems, 32: 15523–15534.
  29. Joint inference of reward machines and policies for reinforcement learning. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 30, 590–598.
  30. Fast and efficient locomotion via learned gait transitions. In Conference on Robot Learning.
  31. Language to Rewards for Robotic Skill Synthesis. arXiv preprint arXiv:2306.08647.
  32. Robot Parkour Learning. arXiv preprint arXiv:2309.05665.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. David DeFazio (5 papers)
  2. Yohei Hayamizu (5 papers)
  3. Shiqi Zhang (88 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets