A Combined Learning and Optimization Framework to Transfer Human Whole-body Loco-manipulation Skills to Mobile Manipulators (2402.13915v1)
Abstract: Humans' ability to smoothly switch between locomotion and manipulation is a remarkable feature of sensorimotor coordination. Leaning and replication of such human-like strategies can lead to the development of more sophisticated robots capable of performing complex whole-body tasks in real-world environments. To this end, this paper proposes a combined learning and optimization framework for transferring human's loco-manipulation soft-switching skills to mobile manipulators. The methodology departs from data collection of human demonstrations for a locomotion-integrated manipulation task through a vision system. Next, the wrist and pelvis motions are mapped to mobile manipulators' End-Effector (EE) and mobile base. A kernelized movement primitive algorithm learns the wrist and pelvis trajectories and generalizes to new desired points according to task requirements. Next, the reference trajectories are sent to a hierarchical quadratic programming controller, where the EE and the mobile base reference trajectories are provided as the first and second priority tasks, generating the feasible and optimal joint level commands. A locomotion-integrated pick-and-place task is executed to validate the proposed approach. After a human demonstrates the task, a mobile manipulator executes the task with the same and new settings, grasping a bottle at non-zero velocity. The results showed that the proposed approach successfully transfers the human loco-manipulation skills to mobile manipulators, even with different geometry.
- M. A. Roa, M. R. Dogar, J. Pages, C. Vivas, A. Morales, N. Correll, M. Gorner, J. Rosell, S. Foix, R. Memmesheimer et al., “Mobile manipulation hackathon: Moving into real world applications,” IEEE Robotics & Automation Magazine, vol. 28, no. 2, pp. 112–124, 2021.
- J. Zhao, A. Giammarino, E. Lamon, J. M. Gandarias, E. D. Momi, and A. Ajoudani, “A hybrid learning and optimization framework to achieve physically interactive tasks with mobile manipulators,” IEEE Robotics and Automation Letters, vol. 7, pp. 8036–8043, 7 2022.
- T. Sandakalum and M. H. Ang Jr, “Motion planning for mobile manipulators—a systematic review,” Machines, vol. 10, no. 2, p. 97, 2022.
- N. Castaman, E. Pagello, E. Menegatti, and A. Pretto, “Receding horizon task and motion planning in changing environments,” Robotics and Autonomous Systems, vol. 145, p. 103863, 2021.
- N. Vahrenkamp, T. Asfour, and R. Dillmann, “Robot placement based on reachability inversion,” in 2013 IEEE International Conference on Robotics and Automation. IEEE, 2013, pp. 1970–1975.
- S. Jauhri, J. Peters, and G. Chalvatzaki, “Robot learning of mobile manipulation with reachability behavior priors,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8399–8406, 2022.
- S. Zimmermann, R. Poranne, and S. Coros, “Go fetch!-dynamic grasps using boston dynamics spot with external robotic arm,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 4488–4494.
- D. Honerkamp, T. Welschehold, and A. Valada, “Learning kinematic feasibility for mobile manipulation through deep reinforcement learning,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 6289–6296, 2021.
- P. Ferrari, M. Cognetti, and G. Oriolo, “Humanoid whole-body planning for loco-manipulation tasks,” in 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017, pp. 4741–4746.
- B. Burgess-Limerick, C. Lehnert, J. Leitner, and P. Corke, “An architecture for reactive mobile manipulation on-the-move,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 1623–1629.
- A. Mészáros, G. Franzese, and J. Kober, “Learning to pick at non-zero-velocity from interactive demonstrations,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6052–6059, 2022.
- J. Zhao, G. J. Lahr, F. Tassi, A. Santopaolo, E. De Momi, and A. Ajoudani, “Impact-friendly object catching at non-zero velocity based on combined optimization and learning,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 4428–4435.
- T. Welschehold, C. Dornhege, and W. Burgard, “Learning mobile manipulation actions from human demonstrations,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 3196–3201.
- Y. Wu, E. Lamon, F. Zhao, W. Kim, and A. Ajoudani, “Unified approach for hybrid motion control of moca based on weighted whole-body cartesian impedance formulation,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3505–3512, 2021.
- F. Tassi, F. Iodice, E. De Momi, and A. Ajoudani, “Sociable and ergonomic human-robot collaboration through action recognition and augmented hierarchical quadratic programming,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 10 712–10 719.
- Y. Huang, L. Rozo, J. Silvério, and D. G. Caldwell, “Kernelized movement primitives,” The International Journal of Robotics Research, vol. 38, no. 7, pp. 833–852, 2019.
- F. Tassi, E. De Momi, and A. Ajoudani, “Augmented hierarchical quadratic programming for adaptive compliance robot control,” in 2021 IEEE international conference on robotics and automation (ICRA). IEEE, 2021, pp. 3568–3574.
- O. Kanoun, F. Lamiraux, and P.-B. Wieber, “Kinematic control of redundant manipulators: Generalizing the task-priority framework to inequality task,” IEEE Transactions on Robotics, vol. 27, no. 4, pp. 785–792, 2011.
- Jianzhuang Zhao (4 papers)
- Francesco Tassi (5 papers)
- Yanlong Huang (14 papers)
- Elena De Momi (40 papers)
- Arash Ajoudani (63 papers)