Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion? (2401.05302v2)

Published 10 Jan 2024 in cs.RO, cs.AI, and cs.HC

Abstract: LLMs have shown exceptional generative abilities in various natural language and generation tasks. However, possible anthropomorphization and leniency towards failure cases have propelled discussions on emergent abilities of LLMs especially on Theory of Mind (ToM) abilities in LLMs. While several false-belief tests exists to verify the ability to infer and maintain mental models of another entity, we study a special application of ToM abilities that has higher stakes and possibly irreversible consequences : Human Robot Interaction. In this work, we explore the task of Perceived Behavior Recognition, where a robot employs a LLM to assess the robot's generated behavior in a manner similar to human observer. We focus on four behavior types, namely - explicable, legible, predictable, and obfuscatory behavior which have been extensively used to synthesize interpretable robot behaviors. The LLMs goal is, therefore to be a human proxy to the agent, and to answer how a certain agent behavior would be perceived by the human in the loop, for example "Given a robot's behavior X, would the human observer find it explicable?". We conduct a human subject study to verify that the users are able to correctly answer such a question in the curated situations (robot setting and plan) across five domains. A first analysis of the belief test yields extremely positive results inflating ones expectations of LLMs possessing ToM abilities. We then propose and perform a suite of perturbation tests which breaks this illusion, i.e. Inconsistent Belief, Uninformative Context and Conviction Test. We conclude that, the high score of LLMs on vanilla prompts showcases its potential use in HRI settings, however to possess ToM demands invariance to trivial or irrelevant perturbations in the context which LLMs lack.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Palm 2 technical report. arXiv preprint arXiv:2305.10403 (2023).
  2. Janet Wilde Astington and Jodie A Baird. 2005. Why language matters for theory of mind. Oxford University Press.
  3. Janet Wilde Astington and Jennifer M Jenkins. 1995. Theory of mind development and social understanding. Cognition & Emotion 9, 2-3 (1995), 151–165.
  4. Bayesian theory of mind: Modeling joint belief-desire attribution. In Proceedings of the annual meeting of the cognitive science society, Vol. 33.
  5. Ali Borji. 2023. A categorical archive of chatgpt failures. arXiv preprint arXiv:2302.03494 (2023).
  6. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023).
  7. Planning for serendipity. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 5300–5306.
  8. Explicability? Legibility? Predictability? Transparency? Privacy? Security? The Emerging Landscape of Interpretable Agent Behavior. arXiv:1811.09722 [cs.AI]
  9. Explicability? legibility? predictability? transparency? privacy? security? the emerging landscape of interpretable agent behavior. In Proceedings of the international conference on automated planning and scheduling, Vol. 29. 86–96.
  10. Plan explanations as model reconciliation: Moving beyond explanation as soliloquy. arXiv preprint arXiv:1701.08317 (2017).
  11. Sandra Devin and Rachid Alami. 2016. An implemented theory of mind to improve human-robot shared plans execution. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 319–326.
  12. PaLM-E: An Embodied Multimodal Language Model. In arXiv preprint arXiv:2303.03378.
  13. Connor Esterwood and Lionel P Robert. 2023. The theory of mind and human–robot trust repair. Scientific Reports 13, 1 (2023), 9877.
  14. Chris Frith and Uta Frith. 2005. Theory of mind. Current biology 15, 17 (2005), R644–R645.
  15. David Gunning. 2018. Machine Common Sense Concept Paper. arXiv:1810.07528 [cs.AI]
  16. Thilo Hagendorff. 2023. Deception abilities emerged in large language models. arXiv preprint arXiv:2307.16513 (2023).
  17. Fledgling theories of mind: Deception as a marker of three-year-olds’ understanding of false belief. Child development 62, 1 (1991), 83–97.
  18. Accommodating human variability in human-robot teams through theory of mind. In Twenty-Second International Joint Conference on Artificial Intelligence.
  19. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608 (2022).
  20. ChatGPT: Jack of all trades, master of none. Information Fusion (2023), 101861.
  21. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213.
  22. Michal Kosinski. 2023a. Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083 (2023).
  23. Michal Kosinski. 2023b. Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083 (2023).
  24. Signaling friends and head-faking enemies simultaneously: Balancing goal obfuscation and goal legibility. arXiv preprint arXiv:1905.10672 (2019).
  25. Planning for proactive assistance in environments with partial observability. arXiv preprint arXiv:2105.00525 (2021).
  26. Explicable planning as minimizing distance from expected behavior. In AAMAS Conference proceedings.
  27. A Bayesian theory of mind approach to nonverbal communication. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 487–496.
  28. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477 (2023).
  29. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. Comput. Surveys 55, 9 (2023), 1–35.
  30. Heidemarie Lohmann and Michael Tomasello. 2003. The role of language in the development of false belief understanding: A training study. Child development 74, 4 (2003), 1130–1144.
  31. ” Would I Feel More Secure With a Robot?”: Understanding Perceptions of Security Robots in Public Spaces. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1–34.
  32. PsychSim: Agent-based modeling of social interactions and influence. In Proceedings of the international conference on cognitive modeling, Vol. 36. 243–248.
  33. Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve. arXiv:2309.13638 [cs.CL]
  34. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]
  35. Stefan Palan and Christian Schitter. 2018. Prolific. ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance 17 (2018), 22–27.
  36. David V Pynadath and Stacy C Marsella. 2005. PsychSim: Modeling theory of mind with decision-theoretic agents. In IJCAI, Vol. 5. 1181–1186.
  37. Event2mind: Commonsense inference on events, intents, and reactions. arXiv preprint arXiv:1805.06939 (2018).
  38. Neural theory-of-mind? on the limits of social intelligence in large lms. arXiv preprint arXiv:2210.13312 (2022).
  39. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv:1904.09728 (2019).
  40. Brian M Scassellati. 2001. Foundations for a Theory of Mind for a Humanoid Robot. Ph. D. Dissertation. Massachusetts Institute of Technology.
  41. Clever hans or neural theory of mind? stress testing social reasoning in large language models. arXiv preprint arXiv:2305.14763 (2023).
  42. Large Language Models Can Be Easily Distracted by Irrelevant Context. In Proceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, 31210–31227. https://proceedings.mlr.press/v202/shi23a.html
  43. Llm-planner: Few-shot grounded planning for embodied agents with large language models. arXiv preprint arXiv:2212.04088 (2022).
  44. Balancing explicability and explanation in human-aware planning. In 2017 AAAI Fall Symposium Series.
  45. Explainable Human-AI Interaction: A Planning Perspective. Springer Nature.
  46. Varsha Suresh and Desmond C Ong. 2021. Using knowledge-embedded attention to augment pre-trained language models for fine-grained emotion recognition. In 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 1–8.
  47. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023).
  48. Tomer Ullman. 2023a. Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks. arXiv:2302.08399 [cs.AI]
  49. Tomer Ullman. 2023b. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399 (2023).
  50. Large Language Models Still Can’t Plan (A Benchmark for LLMs on Planning and Reasoning about Change). arXiv:2206.10498 [cs.CL]
  51. Mudit Verma and Katherine Metcalf. 2022. Symbol Guided Hindsight Priors for Reward Learning from Human Preferences. arXiv preprint arXiv:2210.09151 (2022).
  52. Would a robot trust you? Developmental robotics model of trust and theory of mind. Philosophical Transactions of the Royal Society B 374, 1771 (2019), 20180032.
  53. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022).
  54. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837.
  55. Henry M Wellman. 1992. The child’s theory of mind. The MIT Press.
  56. Heinz Wimmer and Josef Perner. 1983. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition 13, 1 (1983), 103–128.
  57. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. arXiv:2211.05100 [cs.CL]
  58. Emotionx-ku: Bert-max based contextual emotion classifier. arXiv preprint arXiv:1906.11565 (2019).
  59. Human Security Robot Interaction and Anthropomorphism: An Examination of Pepper, RAMSEE, and Knightscope Robots. (2023).
  60. A Mental-Model Centric Landscape of Human-AI Symbiosis. arXiv:2202.09447 [cs.AI]
  61. A Mental Model Based Theory of Trust. arXiv:2301.12569 [cs.AI]
  62. Modeling the Interplay between Human Trust and Monitoring. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 1119–1123. https://doi.org/10.1109/HRI53351.2022.9889475
  63. Trust-Aware Planning: Modeling Trust Evolution in Longitudinal Human-Robot Interaction. CoRR abs/2105.01220 (2021). arXiv:2105.01220 https://arxiv.org/abs/2105.01220
  64. Plan explicability and predictability for robot task planning. In 2017 IEEE international conference on robotics and automation (ICRA). IEEE, 1313–1320.
  65. A Survey of Large Language Models. arXiv:2303.18223 [cs.CL]
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Mudit Verma (25 papers)
  2. Siddhant Bhambri (16 papers)
  3. Subbarao Kambhampati (126 papers)
Citations (14)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

HackerNews

  1. The Emergence of ToM in LLMs (3 points, 1 comment)