2000 character limit reached
Robust Body Exposure (RoBE): A Graph-based Dynamics Modeling Approach to Manipulating Blankets over People (2304.04822v2)
Published 10 Apr 2023 in cs.RO
Abstract: Robotic caregivers could potentially improve the quality of life of many who require physical assistance. However, in order to assist individuals who are lying in bed, robots must be capable of dealing with a significant obstacle: the blanket or sheet that will almost always cover the person's body. We propose a method for targeted bedding manipulation over people lying supine in bed where we first learn a model of the cloth's dynamics. Then, we optimize over this model to uncover a given target limb using information about human body shape and pose that only needs to be provided at run-time. We show how this approach enables greater robustness to variation relative to geometric and reinforcement learning baselines via a number of generalization evaluations in simulation and in the real world. We further evaluate our approach in a human study with 12 participants where we demonstrate that a mobile manipulator can adapt to real variation in human body shape, size, pose, and blanket configuration to uncover target body parts without exposing the rest of the body. Source code and supplementary materials are available online.
- K. Puthuveetil, C. C. Kemp, and Z. Erickson, “Bodies uncovered: Learning to manipulate real blankets around people via physics simulations,” IEEE Robotics and Automation Letters, 2022.
- L. Sun, G. Aragon-Camarasa, S. Rogers, and J. P. Siebert, “Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening,” in IEEE ICRA, 2015.
- A. Ramisa, G. Alenyà, F. Moreno-Noguer, and C. Torras, “Using depth and appearance features for informed robot grasping of highly wrinkled clothes,” in IEEE ICRA, 2012.
- K. Yamazaki, “Grasping point selection on an item of crumpled clothing based on relational shape description,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014.
- H. Yuba, S. Arnold, and K. Yamazaki, “Unfolding of a rectangular cloth from unarranged starting shapes by a dual-armed robot with a mechanism for managing recognition error and uncertainty,” Advanced Robotics, vol. 31, no. 10, pp. 544–556, 2017.
- J. Qian, T. Weng, L. Zhang, B. Okorn, and D. Held, “Cloth region segmentation for robust grasp selection,” IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020.
- D. Seita, N. Jamali, M. Laskey, A. K. Tanwani, R. Berenstein, P. Baskaran, S. Iba, J. Canny, and K. Goldberg, “Deep Transfer Learning of Pick Points on Fabric for Robot Bed-Making,” in International Symposium on Robotics Research (ISRR), 2019.
- D. Seita, A. Ganapathi, R. Hoque, M. Hwang, E. Cen, A. K. Tanwani, A. Balakrishna, B. Thananjeyan, J. Ichnowski, N. Jamali, et al., “Deep imitation learning of sequential fabric smoothing from an algorithmic supervisor,” in IEEE/RSJ IROS, 2020.
- A. Ganapathi, P. Sundaresan, B. Thananjeyan, A. Balakrishna, D. Seita, J. Grannen, M. Hwang, R. Hoque, J. E. Gonzalez, N. Jamali, et al., “Learning dense visual correspondences in simulation to smooth and fold real fabrics,” in IEEE ICRA, 2021.
- J. Matas, S. James, and A. J. Davison, “Sim-to-real reinforcement learning for deformable object manipulation,” in Conference on Robot Learning. PMLR, 2018, pp. 734–743.
- Y. Wu, W. Yan, T. Kurutach, L. Pinto, and P. Abbeel, “Learning to manipulate deformable objects without demonstrations,” in RSS, 2020.
- R. Lee, D. Ward, V. Dasagi, A. Cosgun, J. Leitner, and P. Corke, “Learning arbitrary-goal fabric folding with one hour of real robot experience,” in Conference on Robot Learning, 2021.
- H. Ha and S. Song, “Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding,” in Conference on Robot Learning. PMLR, 2022, pp. 24–33.
- F. Ebert, C. Finn, S. Dasari, A. Xie, A. Lee, and S. Levine, “Visual foresight: Model-based deep reinforcement learning for vision-based robotic control,” arXiv preprint arXiv:1812.00568, 2018.
- R. Hoque, D. Seita, A. Balakrishna, A. Ganapathi, A. K. Tanwani, N. Jamali, K. Yamane, S. Iba, and K. Goldberg, “Visuospatial foresight for multi-step, multi-task fabric manipulation,” arXiv preprint arXiv:2003.09044, 2020.
- W. Yan, A. Vangipuram, P. Abbeel, and L. Pinto, “Learning predictive representations for deformable objects using contrastive estimation,” in Conference on Robot Learning. PMLR, 2021, pp. 564–574.
- Y. Li, J. Wu, R. Tedrake, J. B. Tenenbaum, and A. Torralba, “Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids,” in ICLR, 2019.
- A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. Battaglia, “Learning to simulate complex physics with graph networks,” in Proceedings of the 37th ICML, 2020.
- T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. W. Battaglia, “Learning mesh-based simulation with graph networks,” arXiv preprint arXiv:2010.03409, 2020.
- X. Lin, Y. Wang, Z. Huang, and D. Held, “Learning visible connectivity dynamics for cloth smoothing,” in CoRL, 2021.
- Z. Huang, X. Lin, and D. Held, “Mesh-based dynamics with occlusion reasoning for cloth manipulation,” in RSS, 2022.
- A. S. Kapusta, P. M. Grice, H. M. Clever, Y. Chitalia, D. Park, and C. C. Kemp, “A system for bedside assistance that integrates a robotic bed and a mobile manipulator,” Plos one, vol. 14, no. 10, 2019.
- Y. Wang, D. Held, and Z. Erickson, “Visual haptic reasoning: Estimating contact forces by observing deformable object interactions,” IEEE Robotics and Automation Letters, 2022.
- E. Coumans and Y. Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” 2016.
- J. Stępień, “Physics-based animation of articulated rigid body systems for virtual environments,” Ph.D. dissertation, Silesian University of Technology, 2013.
- Z. Erickson, V. Gangaram, A. Kapusta, C. K. Liu, and C. C. Kemp, “Assistive gym: A physics simulation framework for assistive robotics,” in IEEE ICRA, 2020.
- R. B. Rusu and S. Cousins, “3d is here: Point cloud library (pcl),” in IEEE international conference on robotics and automation, 2011.
- N. Hansen, S. D. Müller, and P. Koumoutsakos, “Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es),” Evolutionary computation, 2003.
- C. C. Kemp, A. Edsinger, H. M. Clever, and B. Matulevich, “The design of stretch: A compact, lightweight mobile manipulator for indoor human environments,” in IEEE ICRA, 2022.
- V. Bazarevsky, I. Grishchenko, K. Raveendran, T. Zhu, F. Zhang, and M. Grundmann, “Blazepose: On-device real-time body pose tracking,” in Computer Vision and Pattern Recognition, ser. Workshop on Computer Vision for Augmented and Virtual Reality, 2020.
- H. M. Clever, A. Kapusta, D. Park, Z. Erickson, Y. Chitalia, and C. C. Kemp, “3d human pose estimation on a configurable bed from a pressure image,” in IEEE/RSJ IROS, 2018.
- H. M. Clever, P. Grady, G. Turk, and C. C. Kemp, “Bodypressure–inferring body pose and contact pressure from a depth image,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
- G. Pavlakos, V. Choutas, N. Ghorbani, T. Bolkart, A. A. Osman, D. Tzionas, and M. J. Black, “Expressive body capture: 3d hands, face, and body from a single image,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
- Z. Erickson, A. Clegg, W. Yu, G. Turk, C. K. Liu, and C. C. Kemp, “What does the person feel? learning to infer applied forces during robot-assisted dressing,” in IEEE ICRA, 2017.
- Z. Erickson, H. M. Clever, G. Turk, C. K. Liu, and C. C. Kemp, “Deep haptic model predictive control for robot-assisted dressing,” in IEEE International Conference on Robotics and Automation, 2018.