Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 167 tok/s
Gemini 2.5 Pro 53 tok/s Pro
GPT-5 Medium 31 tok/s Pro
GPT-5 High 31 tok/s Pro
GPT-4o 106 tok/s Pro
Kimi K2 187 tok/s Pro
GPT OSS 120B 443 tok/s Pro
Claude Sonnet 4.5 37 tok/s Pro
2000 character limit reached

Human-mediated Large Language Models for Robotic Intervention in Children with Autism Spectrum Disorders (2402.00260v3)

Published 1 Feb 2024 in cs.RO and cs.AI

Abstract: The robotic intervention for individuals with Autism Spectrum Disorder (ASD) has generally used pre-defined scripts to deliver verbal content during one-to-one therapy sessions. This practice restricts the use of robots to limited, pre-mediated instructional curricula. In this paper, we increase robot autonomy in one such robotic intervention for children with ASD by implementing perspective-taking teaching. Our approach uses LLMs (LLM) to generate verbal content as texts and then deliver it to the child via robotic speech. In the proposed pipeline, we teach perspective-taking through which our robot takes up three roles: initiator, prompter, and reinforcer. We adopted the GPT-2 + BART pipelines to generate social situations, ask questions (as initiator), and give options (as prompter) when required. The robot encourages the child by giving positive reinforcement for correct answers (as a reinforcer). In addition to our technical contribution, we conducted ten-minute sessions with domain experts simulating an actual perspective teaching session, with the researcher acting as a child participant. These sessions validated our robotic intervention pipeline through surveys, including those from NASA TLX and GodSpeed. We used BERTScore to compare our GPT-2 + BART pipeline with an all GPT-2 and found the performance of the former to be better. Based on the responses by the domain experts, the robot session demonstrated higher performance with no additional increase in mental or physical demand, temporal demand, effort, or frustration compared to a no-robot session. We also concluded that the domain experts perceived the robot as ideally safe, likable, and reliable.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. A. Lambert, N. Norouzi, G. Bruder, and G. Welch, “A systematic review of ten years of research on human interaction with social robots,” International Journal of Human–Computer Interaction, vol. 36, no. 19, pp. 1804–1817, 2020.
  2. J. Forlizzi and C. DiSalvo, “Service robots in the domestic environment: a study of the roomba vacuum in the home,” in Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, 2006, pp. 258–265.
  3. C. D. Kidd and C. Breazeal, “Robots at home: Understanding long-term human-robot interaction,” in 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.   IEEE, 2008, pp. 3230–3235.
  4. K. M. Tsui, M. Desai, H. A. Yanco, and C. Uhlik, “Exploring use cases for telepresence robots,” in Proceedings of the 6th international conference on Human-robot interaction, 2011, pp. 11–18.
  5. A. M. Okamura, “Methods for haptic feedback in teleoperated robot-assisted surgery,” Industrial Robot: An International Journal, vol. 31, no. 6, pp. 499–508, 2004.
  6. I. El Rassi and J.-M. El Rassi, “A review of haptic feedback in tele-operated robotic surgery,” Journal of medical engineering & technology, vol. 44, no. 5, pp. 247–254, 2020.
  7. K. D. Bartl-Pokorny, M. Pykała, P. Uluer, D. E. Barkana, A. Baird, H. Kose, T. Zorcec, B. Robins, B. W. Schuller, and A. Landowska, “Robot-based intervention for children with autism spectrum disorder: a systematic literature review,” IEEE Access, vol. 9, pp. 165 433–165 450, 2021.
  8. D. Saral, S. Olcay, and H. Ozturk, “Autism spectrum disorder: When there is no cure, there are countless of treatments,” Journal of Autism and Developmental Disorders, pp. 1–16, 2022.
  9. D. A. Regier, E. A. Kuhl, and D. J. Kupfer, “The dsm-5: Classification and criteria changes,” World psychiatry, vol. 12, no. 2, pp. 92–98, 2013.
  10. D. Saral, S. Olcay, and H. Ozturk, “Autism spectrum disorder: When there is no cure, there are countless of treatments,” Journal of Autism and Developmental Disorders, vol. 53, no. 12, pp. 4901–4916, 2023.
  11. J. R. Steinbrenner, K. Hume, S. L. Odom, K. L. Morin, S. W. Nowell, B. Tomaszewski, S. Szendrey, N. S. McIntyre, S. Yücesoy-Özkan, and M. N. Savage, “Evidence-based practices for children, youth, and young adults with autism.” FPG child development institute, 2020.
  12. P. Ntaountaki, G. Lorentzou, A. Lykothanasi, P. Anagnostopoulou, V. Alexandropoulou, and A. Drigas, “Robotics in autism intervention.” Int. J. Recent Contributions Eng. Sci. IT, vol. 7, no. 4, pp. 4–17, 2019.
  13. B. Scassellati, L. Boccanfuso, C.-M. Huang, M. Mademtzi, M. Qin, N. Salomons, P. Ventola, and F. Shic, “Improving social skills in children with asd using a long-term, in-home social robot,” Science Robotics, vol. 3, no. 21, p. eaat7544, 2018.
  14. M. Begum, R. W. Serna, D. Kontak, J. Allspaw, J. Kuczynski, H. A. Yanco, and J. Suarez, “Measuring the efficacy of robots in autism therapy: How informative are standard hri metrics’,” in Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, 2015, pp. 335–342.
  15. H.-L. Cao, P. G. Esteban, M. Bartlett, P. Baxter, T. Belpaeme, E. Billing, H. Cai, M. Coeckelbergh, C. Costescu, D. David et al., “Robot-enhanced therapy: Development and validation of supervised autonomous robotic system for autism spectrum disorders therapy,” IEEE robotics & automation magazine, vol. 26, no. 2, pp. 49–58, 2019.
  16. I. van den Berk-Smeekens, M. van Dongen-Boomsma, M. W. De Korte, J. C. Den Boer, I. J. Oosterling, N. C. Peters-Scheffer, J. K. Buitelaar, E. I. Barakova, T. Lourens, W. G. Staal et al., “Adherence and acceptability of a robot-assisted pivotal response treatment protocol for children with autism spectrum disorder,” Scientific reports, vol. 10, no. 1, p. 8110, 2020.
  17. K. Takata, Y. Yoshikawa, T. Muramatsu, Y. Matsumoto, H. Ishiguro, M. Mimura, and H. Kumazaki, “Social skills training using multiple humanoid robots for individuals with autism spectrum conditions,” Frontiers in Psychiatry, vol. 14, 2023.
  18. L. Boccanfuso, S. Scarborough, R. K. Abramson, A. V. Hall, H. H. Wright, and J. M. O’Kane, “A low-cost socially assistive robot and robot-assisted intervention for children with autism spectrum disorder: field trials and lessons learned,” Autonomous Robots, vol. 41, pp. 637–655, 2017.
  19. P. Pennisi, A. Tonacci, G. Tartarisco, L. Billeci, L. Ruta, S. Gangemi, and G. Pioggia, “Autism and social robotics: A systematic review,” Autism Research, vol. 9, no. 2, pp. 165–183, 2016.
  20. M. Stolarz, A. Mitrevski, M. Wasil, and P. G. Plöger, “Personalized behaviour models: A survey focusing on autism therapy applications,” arXiv preprint arXiv:2205.08975, 2022.
  21. J.-J. Cabibihan, H. Javed, M. Ang, and S. M. Aljunied, “Why robots? a survey on the roles and benefits of social robots in the therapy of children with autism,” International journal of social robotics, vol. 5, pp. 593–618, 2013.
  22. V. Holeva, V. Nikopoulou, C. Lytridis, C. Bazinas, P. Kechayas, G. Sidiropoulos, M. Papadopoulou, M. Kerasidou, C. Karatsioras, N. Geronikola et al., “Effectiveness of a robot-assisted psychological intervention for children with autism spectrum disorder,” Journal of Autism and Developmental Disorders, pp. 1–17, 2022.
  23. Z. Salimi, E. Jenabi, and S. Bashirian, “Are social robots ready yet to be used in care and therapy of autism spectrum disorder: A systematic review of randomized controlled trials,” Neuroscience & Biobehavioral Reviews, vol. 129, pp. 1–16, 2021.
  24. E. S. Kim, L. D. Berkovits, E. P. Bernier, D. Leyzberg, F. Shic, R. Paul, and B. Scassellati, “Social robots as embedded reinforcers of social behavior in children with autism,” Journal of autism and developmental disorders, vol. 43, pp. 1038–1049, 2013.
  25. C. A. Huijnen, M. A. Lexis, R. Jansens, and L. P. de Witte, “Mapping robots to therapy and educational objectives for children with autism spectrum disorder,” Journal of autism and developmental disorders, vol. 46, pp. 2100–2114, 2016.
  26. C. A. Huijnen, M. A. Lexis, R. Jansens, and L. P. de Witte, “Roles, strengths and challenges of using robots in interventions for children with autism spectrum disorder (asd),” Journal of autism and developmental disorders, vol. 49, pp. 11–21, 2019.
  27. B. Scassellati, H. Admoni, and M. Matarić, “Robots for use in autism research,” Annual review of biomedical engineering, vol. 14, pp. 275–294, 2012.
  28. S. Baron-Cohen, A. M. Leslie, and U. Frith, “Does the autistic child have a “theory of mind”?” Cognition, vol. 21, no. 1, pp. 37–46, 1985.
  29. E. Gould, J. Tarbox, D. O’Hora, S. Noone, and R. Bergstrom, “Teaching children with autism a basic component skill of perspective-taking,” Behavioral Interventions, vol. 26, no. 1, pp. 50–66, 2011.
  30. F. Welsh, A. C. Najdowski, D. Strauss, L. Gallegos, and J. A. Fullen, “Teaching a perspective-taking component skill to children with autism in the natural environment,” Journal of Applied Behavior Analysis, vol. 52, no. 2, pp. 439–450, 2019.
  31. Y. Kimhi, “Theory of mind abilities and deficits in autism spectrum disorders,” Topics in Language Disorders, vol. 34, no. 4, pp. 329–343, 2014.
  32. E. Hoddenbach, H. M. Koot, P. Clifford, C. Gevers, C. Clauser, F. Boer, and S. Begeer, “Individual differences in the efficacy of a short theory of mind intervention for children with autism spectrum disorder: a randomized controlled trial,” Trials, vol. 13, no. 1, pp. 1–7, 2012.
  33. S. Begeer, P. Howlin, E. Hoddenbach, C. Clauser, R. Lindauer, P. Clifford, C. Gevers, F. Boer, and H. M. Koot, “Effects and moderators of a short theory of mind intervention for children with autism spectrum disorder: A randomized controlled trial,” Autism Research, vol. 8, no. 6, pp. 738–748, 2015.
  34. O. Rudovic, J. Lee, M. Dai, B. Schuller, and R. W. Picard, “Personalized machine learning for robot perception of affect and engagement in autism therapy,” Science Robotics, vol. 3, no. 19, p. eaao6760, 2018.
  35. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  36. A. Ezen-Can, “A comparison of lstm and bert for small corpus,” arXiv preprint arXiv:2009.05451, 2020.
  37. A. Rahali and M. A. Akhloufi, “Deeppress: guided press release topic-aware text generation using ensemble transformers,” Neural Computing and Applications, vol. 35, no. 17, pp. 12 847–12 874, 2023.
  38. P. Le and W. Zuidema, “Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive lstms,” arXiv preprint arXiv:1603.00423, 2016.
  39. S. Hochreiter, “The vanishing gradient problem during learning recurrent neural nets and problem solutions,” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 6, no. 02, pp. 107–116, 1998.
  40. D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014.
  41. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  42. Y.-H. Chan and Y.-C. Fan, “BERT for question generation,” in Proceedings of the 12th International Conference on Natural Language Generation.   Tokyo, Japan: Association for Computational Linguistics, Oct.–Nov. 2019, pp. 173–177. [Online]. Available: https://aclanthology.org/W19-8624
  43. Y.-H. Chan and Y.-C. Fan, “A recurrent bert-based model for question generation,” in Proceedings of the 2nd workshop on machine reading for question answering, 2019, pp. 154–162.
  44. R. Luo, L. Sun, Y. Xia, T. Qin, S. Zhang, H. Poon, and T.-Y. Liu, “Biogpt: generative pre-trained transformer for biomedical text generation and mining,” Briefings in Bioinformatics, vol. 23, no. 6, p. bbac409, 2022.
  45. B. Chintagunta, N. Katariya, X. Amatriain, and A. Kannan, “Medically aware gpt-3 as a data generator for medical dialogue summarization,” in Machine Learning for Healthcare Conference.   PMLR, 2021, pp. 354–372.
  46. C. Li, X. Zhang, D. Chrysostomou, and H. Yang, “Tod4ir: A humanised task-oriented dialogue system for industrial robots,” IEEE Access, vol. 10, pp. 91 631–91 649, 2022.
  47. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” arXiv preprint arXiv:1910.13461, 2019.
  48. Y. Ye, H. You, and J. Du, “Improved trust in human-robot collaboration with chatgpt,” IEEE Access, 2023.
  49. W. Huang, F. Xia, D. Shah, D. Driess, A. Zeng, Y. Lu, P. Florence, I. Mordatch, S. Levine, K. Hausman et al., “Grounded decoding: Guiding text generation with grounded models for robot control,” arXiv preprint arXiv:2303.00855, 2023.
  50. Z. Mandi, S. Jain, and S. Song, “Roco: Dialectic multi-robot collaboration with large language models,” arXiv preprint arXiv:2307.04738, 2023.
  51. S. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor, “Chatgpt for robotics: Design principles and model abilities,” Microsoft Auton. Syst. Robot. Res, vol. 2, p. 20, 2023.
  52. A. Lekova, P. Tsvetkova, T. Tanev, P. Mitrouchev, and S. Kostova, “Making humanoid robots teaching assistants by using natural language processing (nlp) cloud-based services,” Journal of Mechatronics and Artificial Intelligence in Engineering, vol. 3, no. 1, pp. 30–39, 2022.
  53. B. Xie, X. Xi, X. Zhao, Y. Wang, W. Song, J. Gu, and S. Zhu, “Chatgpt for robotics: A new approach to human-robot interaction and task planning,” in International Conference on Intelligent Robotics and Applications.   Springer, 2023, pp. 365–376.
  54. F. Bertacchini, F. Demarco, C. Scuro, P. Pantano, and E. Bilotta, “A social robot connected with chatgpt to improve cognitive functioning in asd subjects,” Frontiers in Psychology, vol. 14, 2023.
  55. M. Sap, H. Rashkin, D. Chen, R. LeBras, and Y. Choi, “Socialiqa: Commonsense reasoning about social interactions,” arXiv preprint arXiv:1904.09728, 2019.
  56. scikit learn. Tfidfvectorizer. [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html
  57. T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” arXiv preprint arXiv:1904.09675, 2019.
  58. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318.
  59. T. Tianyi. Bertscore. [Online]. Available: https://github.com/Tiiiger/bert_score
  60. M. L. McHugh, “Interrater reliability: the kappa statistic,” Biochemia medica, vol. 22, no. 3, pp. 276–282, 2012.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 2 tweets and received 0 likes.

Upgrade to Pro to view all of the tweets about this paper:

Start a free 7-day Pro trial