Extending 3D body pose estimation for robotic-assistive therapies of autistic children (2402.08006v1)
Abstract: Robotic-assistive therapy has demonstrated very encouraging results for children with Autism. Accurate estimation of the child's pose is essential both for human-robot interaction and for therapy assessment purposes. Non-intrusive methods are the sole viable option since these children are sensitive to touch. While depth cameras have been used extensively, existing methods face two major limitations: (i) they are usually trained with adult-only data and do not correctly estimate a child's pose, and (ii) they fail in scenarios with a high number of occlusions. Therefore, our goal was to develop a 3D pose estimator for children, by adapting an existing state-of-the-art 3D body modelling method and incorporating a linear regression model to fine-tune one of its inputs, thereby correcting the pose of children's 3D meshes. In controlled settings, our method has an error below $0.3m$, which is considered acceptable for this kind of application and lower than current state-of-the-art methods. In real-world settings, the proposed model performs similarly to a Kinect depth camera and manages to successfully estimate the 3D body poses in a much higher number of frames.
- S. Ali, F. Mehmood, D. Dancey, Y. Ayaz, M. J. Khan, N. Naseer, R. D. C. Amadeu, H. Sadia, and R. Nawaz, “An adaptive multi-robot therapy for improving joint attention and imitation of asd children,” IEEE Access, vol. 7, pp. 81 808–81 825, 2019.
- Z. Zheng, E. M. Young, A. R. Swanson, A. S. Weitlauf, Z. E. Warren, and N. Sarkar, “Robot-mediated imitation skill training for children with autism,” IEEE Trans. Neural Syst., vol. 24, no. 6, pp. 682–691, 2016.
- E. Marinoiu, M. Zanfir, V. Olaru, and C. Sminchisescu, “3d human sensing, action and emotion recognition in robot assisted therapy of children with autism,” in CVPR, 2018.
- J. M. Rehg, G. D. Abowd, A. Rozga, M. Romero, M. A. Clements, S. Sclaroff, I. Essa, O. Y. Ousley, Y. Li, C. Kim, H. Rao, J. C. Kim, L. L. Presti, J. Zhang, D. Lantsman, J. Bidwell, and Z. Ye, “Decoding children’s social behavior,” in CVPR, 2013, pp. 3414–3421.
- M.-H. Nguyen, C.-C. Hsiao, W.-H. Cheng, and C.-C. Huang, “Practical 3d human skeleton tracking based on multi-view and multi-kinect fusion,” Multimedia Systems, vol. 28, no. 2, pp. 529–552, Apr 2022.
- A. Mustafa, A. Caliskan, L. Agapito, and A. Hilton, “Multi-person implicit reconstruction from a single image,” in CVPR, 2021, pp. 14 474–14 483.
- W. Jiang, N. Kolotouros, G. Pavlakos, X. Zhou, and K. Daniilidis, “Coherent reconstruction of multiple humans from a single image,” in CVPR, 2020.
- Y. Sun, Q. Bao, W. Liu, Y. Fu, M. J. Black, and T. Mei, “Monocular, one-stage, regression of multiple 3d people,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 11 179–11 188.
- Y. Sun, W. Liu, Q. Bao, Y. Fu, T. Mei, and M. J. Black, “Putting people in their place: Monocular regression of 3d people in depth,” in CVPR, 2022, pp. 13 243–13 252.
- N. Hesse, S. Pujades, J. Romero, M. J. Black, C. Bodensteiner, M. Arens, U. G. Hofmann, U. Tacke, M. Hadders-Algra, R. Weinberger, W. Müller-Felber, and A. S. Schroeder, “Learning an infant body model from RGB-D data for accurate full body motion analysis,” in MICCAI. Springer, 2018.
- L. Santos, A. Geminiani, P. Schydlo, I. Olivieri, J. Santos-Victor, and A. Pedrocchi, “Design of a robotic coach for motor, social and cognitive skills training toward applications with ASD children,” IEEE Trans. Neural Syst. Rehabil. Eng., 2021.
- G. Nagymáté and R. M. Kiss, “Application of optitrack motion capture systems in human movement analysis: A systematic literature review,” Recent innov. mechatron., vol. 5, no. 1., p. 1–9., Apr. 2018.
- A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, P. Dollár, and R. Girshick, “Segment anything,” 2023.
- Laura Santos (2 papers)
- Bernardo Carvalho (19 papers)
- Catarina Barata (13 papers)
- José Santos-Victor (22 papers)