Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How Can Large Language Models Enable Better Socially Assistive Human-Robot Interaction: A Brief Survey (2404.00938v2)

Published 1 Apr 2024 in cs.HC, cs.CL, cs.CV, and cs.RO

Abstract: Socially assistive robots (SARs) have shown great success in providing personalized cognitive-affective support for user populations with special needs such as older adults, children with autism spectrum disorder (ASD), and individuals with mental health challenges. The large body of work on SAR demonstrates its potential to provide at-home support that complements clinic-based interventions delivered by mental health professionals, making these interventions more effective and accessible. However, there are still several major technical challenges that hinder SAR-mediated interactions and interventions from reaching human-level social intelligence and efficacy. With the recent advances in LLMs, there is an increased potential for novel applications within the field of SAR that can significantly expand the current capabilities of SARs. However, incorporating LLMs introduces new risks and ethical concerns that have not yet been encountered, and must be carefully be addressed to safely deploy these more advanced systems. In this work, we aim to conduct a brief survey on the use of LLMs in SAR technologies, and discuss the potentials and risks of applying LLMs to the following three major technical challenges of SAR: 1) natural language dialog; 2) multimodal understanding; 3) LLMs as robot policies.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Large language models show human-like content biases in transmission chain experiments. Proceedings of the National Academy of Sciences, 120(44): e2313790120.
  2. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774.
  3. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. arXiv:2204.01691.
  4. Reinforcement learning approaches in social robotics. Sensors, 21(4): 1292.
  5. A social robot connected with chatGPT to improve cognitive functioning in ASD subjects. Frontiers in Psychology, 14.
  6. Language models for human-robot interaction. In ACM/IEEE International Conference on Human-Robot Interaction, March 13–16, 2023, Stockholm, Sweden, 905–906. ACM Digital Library.
  7. Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. International journal of social robotics, 5: 593–618.
  8. Escaping oz: Autonomy in socially assistive robotics. Annual Review of Control, Robotics, and Autonomous Systems, 2: 33–61.
  9. A survey on multimodal large language models for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 958–979.
  10. A systematic literature review of experiments in socially assistive robotics using humanoid robots. arXiv preprint arXiv:1711.05379.
  11. Socially assistive robotics. IEEE Robotics & Automation Magazine, 18(1): 24–31.
  12. Understanding social reasoning in language models with language models. arXiv preprint arXiv:2306.15448.
  13. MLLM-Bench, Evaluating Multi-modal LLMs using GPT-4V. arXiv preprint arXiv:2311.13951.
  14. Knowledge-grounded dialogue flow management for social robots and conversational agents. International Journal of Social Robotics, 14(5): 1273–1293.
  15. A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation. arXiv preprint arXiv:2305.11391.
  16. Between Reality and Delusion: Challenges of Applying Large Language Models to Companion Robots for Open-Domain Dialogues with Older Adults.
  17. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, 4904–4916. PMLR.
  18. Impacts of low-cost robotic pets for older adults and people with dementia: scoping review. JMIR rehabilitation and assistive technologies, 8(1): e25340.
  19. The usability and impact of a low-cost pet robot for older adults and people with dementia: qualitative content analysis of user experiences and perceptions on consumer websites. JMIR aging, 5(1): e29224.
  20. Developing Social Robots with Empathetic Non-Verbal Cues Using Large Language Models. arXiv preprint arXiv:2308.16529.
  21. Intention understanding in human–robot interaction based on visual-NLP semantics. Frontiers in Neurorobotics, 14: 610139.
  22. A Sign Language Recognition System with Pepper, Lightweight-Transformer, and LLM. arXiv preprint arXiv:2309.16898.
  23. Ethical use of electronic health record data and artificial intelligence: recommendations of the primary care informatics working group of the international medical informatics association. Yearbook of medical informatics, 29(01): 051–057.
  24. Socially assistive robotics. Springer handbook of robotics, 1973–1994.
  25. Real-time emotion generation in human-robot dialogue using large language models. Frontiers in Robotics and AI, 10.
  26. Physical human-robot interaction influence in ASD therapy through an affordable soft social robot. Journal of Intelligent & Robotic Systems, 105(3): 67.
  27. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748–8763. PMLR.
  28. Randall, N. 2019. A survey of robot-assisted language learning (RALL). ACM Transactions on Human-Robot Interaction (THRI), 9(1): 1–36.
  29. Robotic vision for human-robot interaction and collaboration: A survey and systematic review. ACM Transactions on Human-Robot Interaction, 12(1): 1–66.
  30. Social IQa: Commonsense Reasoning about Social Interactions. In Inui, K.; Jiang, J.; Ng, V.; and Wan, X., eds., Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 4463–4473. Hong Kong, China: Association for Computational Linguistics.
  31. Use of social robots in mental health and well-being research: systematic review. Journal of medical Internet research, 21(7): e13322.
  32. Toward personalized affect-aware socially assistive robot tutors for long-term interventions with children with autism. ACM Transactions on Human-Robot Interaction (THRI), 11(4): 1–28.
  33. A systematic review of research into how robotic technology can help older people. Smart Health, 7: 1–18.
  34. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA), 11523–11530. IEEE.
  35. Skantze, G. 2021. Turn-taking in conversational systems and human-robot interaction: a review. Computer Speech & Language, 67: 101178.
  36. VITA: A Multi-modal LLM-based System for Longitudinal, Autonomous, and Adaptive Robotic Mental Well-being Coaching. arXiv preprint arXiv:2312.09740.
  37. GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat System. arXiv preprint arXiv:2306.01741.
  38. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering.
  39. GPT4Vis: What Can GPT-4 Do for Zero-shot Visual Recognition? arXiv preprint arXiv:2311.15732.
  40. A systematic evaluation of large language models of code. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, 1–10.
  41. A survey on recent advances in social robotics. Robotics, 11(4): 75.
  42. Vision-language models for vision tasks: A survey. arXiv preprint arXiv:2304.00685.
  43. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models. arXiv preprint arXiv:2309.01219.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Zhonghao Shi (14 papers)
  2. Ellen Landrum (1 paper)
  3. Amy O' Connell (1 paper)
  4. Mina Kian (4 papers)
  5. Leticia Pinto-Alva (2 papers)
  6. Kaleen Shrestha (3 papers)
  7. Xiaoyuan Zhu (5 papers)
  8. Maja J Matarić (11 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets