Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DeepTransition: Viability Leads to the Emergence of Gait Transitions in Learning Anticipatory Quadrupedal Locomotion Skills (2306.07419v2)

Published 12 Jun 2023 in cs.RO, cs.AI, and cs.LG

Abstract: Quadruped animals seamlessly transition between gaits as they change locomotion speeds. While the most widely accepted explanation for gait transitions is energy efficiency, there is no clear consensus on the determining factor, nor on the potential effects from terrain properties. In this article, we propose that viability, i.e. the avoidance of falls, represents an important criterion for gait transitions. We investigate the emergence of gait transitions through the interaction between supraspinal drive (brain), the central pattern generator in the spinal cord, the body, and exteroceptive sensing by leveraging deep reinforcement learning and robotics tools. Consistent with quadruped animal data, we show that the walk-trot gait transition for quadruped robots on flat terrain improves both viability and energy efficiency. Furthermore, we investigate the effects of discrete terrain (i.e. crossing successive gaps) on imposing gait transitions, and find the emergence of trot-pronk transitions to avoid non-viable states. Compared with other potential criteria such as peak forces and energy efficiency, viability is the only improved factor after gait transitions on both flat and discrete gap terrains, suggesting that viability could be a primary and universal objective of gait transitions, while other criteria are secondary objectives and/or a consequence of viability. Moreover, we deploy our learned controller in sim-to-real hardware experiments and demonstrate state-of-the-art quadruped agility in challenging scenarios, where the Unitree A1 quadruped autonomously transitions gaits between trot and pronk to cross consecutive gaps of up to 30 cm (83.3 % of the body-length) at over 1.3 m/s.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. Current principles of motor control, with special reference to vertebrate locomotion. \JournalTitlePhysiological reviews 100, 271–320 (2020).
  2. Gait and the energetics of locomotion in horses. \JournalTitleNature 292, 239–240 (1981).
  3. Hreljac, A. Preferred and energetically optimal gait transition speeds in human locomotion. \JournalTitleMedicine and Science in Sports and Exercise 25, 1158–1162 (1993).
  4. Energy cost and stride duration variability at preferred transition gait speed between walking and running. \JournalTitleCanadian Journal of Applied Physiology 21, 471–480 (1996).
  5. Comparison between preferred and energetically optimal transition speeds in adolescents. \JournalTitleEuropean journal of applied physiology 88, 117–121 (2002).
  6. A mechanical trigger for the trot-gallop transition in horses. \JournalTitleScience 253, 306–308 (1991).
  7. Granatosky, M. C. et al. Inter-stride variability triggers gait transitions in mammals and birds. \JournalTitleProceedings of the Royal Society B 285, 20181766 (2018).
  8. On the central generation of locomotion in the low spinal cat. \JournalTitleExperimental brain research 34, 241–261 (1979).
  9. Brown, T. G. The intrinsic factors in the act of progression in the mammal. \JournalTitleProceedings of the Royal Society of London. Series B, containing papers of a biological character 84, 308–319 (1911).
  10. Control of walking and running by means of electrical stimulation of the mesencephalon. \JournalTitleElectroencephalography and clinical neurophysiology 26, 549–549 (1969).
  11. Whelan, P. J. Control of locomotion in the decerebrate cat. \JournalTitleProgress in neurobiology 49, 481–515 (1996).
  12. On the initiation of the swing phase of locomotion in chronic spinal cats. \JournalTitleBrain research 146, 269–277 (1978).
  13. Ijspeert, A. J. Biorobotics: Using robots to emulate and investigate agile locomotion. \JournalTitlescience 346, 196–203 (2014).
  14. A terradynamics of legged locomotion on granular media. \JournalTitlescience 339, 1408–1412 (2013).
  15. Nyakatura, J. A. et al. Reverse-engineering the locomotion of a stem amniote. \JournalTitleNature 565, 351–355 (2019).
  16. Ijspeert, A. J. Central pattern generators for locomotion control in animals and robots: A review. \JournalTitleNeural Networks 21, 642–653, DOI: https://doi.org/10.1016/j.neunet.2008.03.014 (2008). Robotics and Neuroscience.
  17. A survey on cpg-inspired control models and system implementation. \JournalTitleIEEE transactions on neural networks and learning systems 25, 441–456 (2013).
  18. An energy landscape approach to locomotor transitions in complex 3d terrain. \JournalTitleProceedings of the National Academy of Sciences 117, 14987–14995 (2020).
  19. Spröwitz, A. et al. Towards dynamic trot gait locomotion: Design, control, and experiments with cheetah-cub, a compliant quadruped robot. \JournalTitleThe International Journal of Robotics Research 32, 932–950, DOI: 10.1177/0278364913489205 (2013).
  20. Central pattern generators augmented with virtual model control for quadruped rough terrain locomotion. In 2013 IEEE International Conference on Robotics and Automation, 3321–3328, DOI: 10.1109/ICRA.2013.6631040 (2013).
  21. Adaptive control strategies for interlimb coordination in legged robots: a review. \JournalTitleFrontiers in neurorobotics 11, 39 (2017).
  22. Adaptive dynamic walking of a quadruped robot on natural ground based on biological concepts. \JournalTitleThe International Journal of Robotics Research 26, 475–490 (2007).
  23. Versatile modular neural locomotion control with fast learning. \JournalTitleNature Machine Intelligence 4, 169–179 (2022).
  24. Learning cpg-based biped locomotion with a policy gradient method: Application to a humanoid robot. \JournalTitleThe International Journal of Robotics Research 27, 213–228 (2008).
  25. From swimming to walking with a salamander robot driven by a spinal cord model. \JournalTitlescience 315, 1416–1420 (2007).
  26. Thandiackal, R. et al. Emergence of robust self-organized undulatory swimming based on local hydrodynamic force sensing. \JournalTitleScience Robotics 6 (2021).
  27. Dutta, S. et al. Programmable coupled oscillators for synchronized locomotion. \JournalTitleNature communications 10, 1–10 (2019).
  28. A quadruped robot exhibiting spontaneous gait transitions from walking to trotting to galloping. \JournalTitleScientific reports 7, 1–10 (2017).
  29. A simple rule for quadrupedal gait generation determined by leg loading feedback: a modeling study. \JournalTitleScientific reports 5, 1–11 (2015).
  30. Autonomous gait transition and galloping over unperceived obstacles of a quadruped robot with cpg modulated by vestibular feedback. \JournalTitleRobotics and Autonomous Systems 111, 1–19 (2019).
  31. Hysteresis in the gait transition of a quadruped investigated using simple body mechanical and oscillator network models. \JournalTitlePhysical Review E 83, 061909 (2011).
  32. Aoi, S. et al. A stability-based mechanism for hysteresis in the walk–trot transition in quadruped locomotion. \JournalTitleJournal of The Royal Society Interface 10, 20120908 (2013).
  33. Why change gaits? dynamics of the walk-run transition. \JournalTitleJournal of Experimental Psychology: Human Perception and Performance 21, 183 (1995).
  34. Dynamics of human gait transitions. \JournalTitleTiming of behavior: Neural, psychological, and computational perspectives 323–343 (1998).
  35. Minimizing energy consumption leads to the emergence of gaits in legged robots. In Faust, A., Hsu, D. & Neumann, G. (eds.) Proceedings of the 5th Conference on Robot Learning, vol. 164 of Proceedings of Machine Learning Research, 928–937 (PMLR, 2022).
  36. Shao, Y. et al. Learning free gait transition for quadruped robots via phase-guided controller. \JournalTitleIEEE Robotics and Automation Letters 7, 1230–1237, DOI: 10.1109/LRA.2021.3136645 (2022).
  37. Robust high-speed running for quadruped robots via deep reinforcement learning. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 10364–10370, DOI: 10.1109/IROS47612.2022.9982132 (2022).
  38. Caluwaerts, K. et al. Barkour: Benchmarking animal-level agility with quadruped robots. \JournalTitlearXiv preprint arXiv:2305.14654 (2023).
  39. Opt-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors. \JournalTitlearXiv preprint arXiv:2210.01247 (2022).
  40. Li, C. et al. Learning agile skills via adversarial imitation of rough partial demonstrations. In Conference on Robot Learning, 342–352 (PMLR, 2023).
  41. Smith, L. et al. Learning and adapting agile locomotion skills by transferring experience. \JournalTitlearXiv preprint arXiv:2304.09834 (2023).
  42. Rl+ model-based control: Using on-demand optimal control to learn versatile legged locomotion. \JournalTitlearXiv preprint arXiv:2305.17842 (2023).
  43. Fast and efficient locomotion via learned gait transitions. In Conference on Robot Learning, 773–783 (PMLR, 2022).
  44. Wieber, P.-B. Viability and predictive control for safe locomotion. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, 1103–1108 (IEEE, 2008).
  45. Wieber, P.-B. On the stability of walking systems. In Proceedings of the international workshop on humanoid and human friendly robotics (2002).
  46. Puppeteer and marionette: Learning anticipatory quadrupedal locomotion based on interactions of a central pattern generator and supraspinal drive. \JournalTitle2023 IEEE International Conference on Robotics and Automation (2023).
  47. Contextual learning and obstacle memory in the walking cat. \JournalTitleIntegrative and Comparative Biology 47, 457–464 (2007).
  48. Learning, planning, and control for quadruped locomotion over challenging terrain. \JournalTitleThe International Journal of Robotics Research 30, 236–258 (2011).
  49. Magana, O. A. V. et al. Fast and continuous foothold adaptation for dynamic locomotion through cnns. \JournalTitleIEEE Robotics and Automation Letters 4, 2140–2147 (2019).
  50. Yu, W. et al. Visual-locomotion: Learning to walk on complex terrains with vision. In 5th Annual Conference on Robot Learning (2021).
  51. Lee, K.-H. et al. Pi-ars: Accelerating evolution-learned visual-locomotion with predictive information representations. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 1447–1454 (IEEE, 2022).
  52. Legged locomotion in challenging terrains using egocentric vision. \JournalTitlearXiv preprint arXiv:2211.07638 (2022).
  53. Glide: Generalizable quadrupedal locomotion in diverse environments with a centroidal model. In Algorithmic Foundations of Robotics XV: Proceedings of the Fifteenth Workshop on the Algorithmic Foundations of Robotics, 523–539 (Springer, 2022).
  54. Vision-aided dynamic quadrupedal locomotion on discrete terrain using motion libraries. In 2022 International Conference on Robotics and Automation (ICRA), 4708–4714 (IEEE, 2022).
  55. Margolis, G. B. et al. Learning to jump from pixels. In Proceedings of the 5th Conference on Robot Learning, vol. 164 of Proceedings of Machine Learning Research, 1025–1034 (PMLR, 2022).
  56. Advanced skills by learning locomotion and local navigation end-to-end. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2497–2503 (IEEE, 2022).
  57. Neural volumetric memory for visual locomotion control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1430–1440 (2023).
  58. The cost of transport in an extended trot. \JournalTitleEquine Veterinary Journal 34, 126–130 (2002).
  59. Hind limb stepping over obstacles in the horse guided by place-object memory. \JournalTitleBehavioural brain research 198, 372–379 (2009).
  60. The influence of visual, vestibular, and hindlimb proprioceptive ablations on landing preparation in cane toads. \JournalTitleIntegrative and Comparative Biology 58, 894–905 (2018).
  61. Biomechanical and energetic determinants of the walk–trot transition in horses. \JournalTitleJournal of Experimental Biology 207, 4215–4223 (2004).
  62. Computational modeling of spinal circuits controlling limb coordination and gaits in quadrupeds. \JournalTitleElife 6, e31050 (2017).
  63. Beyond basins of attraction: Quantifying robustness of natural dynamics. \JournalTitleIEEE Transactions on Robotics 35, 939–952 (2019).
  64. Viability, task switching, and fall avoidance of the simplest dynamic walker. \JournalTitleScientific reports 12, 8993 (2022).
  65. Modeling and control of legged robots. In Springer handbook of robotics, 1203–1234 (Springer, 2016).
  66. The condition for dynamic stability. \JournalTitleJournal of biomechanics 38, 1–8 (2005).
  67. Online dcm trajectory generation for push recovery of torque-controlled humanoid robots. In 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), 671–678 (IEEE, 2019).
  68. Robust walking based on mpc with viability guarantees. \JournalTitleIEEE Transactions on Robotics 38, 2389–2404 (2021).
  69. Local dynamic stability and variability of gait are associated with fall history in elderly subjects. \JournalTitleGait & posture 36, 527–531 (2012).
  70. Rohatgi, A. Webplotdigitizer: Version 4.6 (2022).
  71. Modelling spinal circuitry involved in locomotor pattern generation: insights from deletions during fictive locomotion. \JournalTitleThe Journal of physiology 577, 617–639 (2006).
  72. CPG-RL: Learning central pattern generators for quadruped locomotion. \JournalTitleIEEE Robotics and Automation Letters 7, 12547–12554 (2022).
  73. Spontaneous gait transition to high-speed galloping by reconciliation between body support and propulsion. \JournalTitleAdvanced robotics 32, 794–808 (2018).
  74. High-dimensional continuous control using generalized advantage estimation. \JournalTitleCoRR abs/1506.02438 (2015).
  75. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org (2016–2019).
  76. Learning to walk in minutes using massively parallel deep reinforcement learning. In Faust, A., Hsu, D. & Neumann, G. (eds.) Proceedings of the 5th Conference on Robot Learning, vol. 164 of Proceedings of Machine Learning Research, 91–100 (PMLR, 2022).
  77. Makoviychuk, V. et al. Isaac gym: High performance gpu-based physics simulation for robot learning. \JournalTitlearXiv preprint arXiv:2108.10470 (2021).
  78. Unitree Robotics. A1 (2021).
  79. Angular momentum in human walking. \JournalTitleJournal of experimental biology 211, 467–481 (2008).
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now