Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MindDial: Belief Dynamics Tracking with Theory-of-Mind Modeling for Situated Neural Dialogue Generation (2306.15253v4)

Published 27 Jun 2023 in cs.CL

Abstract: Humans talk in daily conversations while aligning and negotiating the expressed meanings or common ground. Despite the impressive conversational abilities of the large generative LLMs, they do not consider the individual differences in contextual understanding in a shared situated environment. In this work, we propose MindDial, a novel conversational framework that can generate situated free-form responses with theory-of-mind modeling. We introduce an explicit mind module that can track the speaker's belief and the speaker's prediction of the listener's belief. Then the next response is generated to resolve the belief difference and take task-related action. Our framework is applied to both prompting and fine-tuning-based models, and is evaluated across scenarios involving both common ground alignment and negotiation. Experiments show that models with mind modeling can achieve higher task outcomes when aligning and negotiating common ground. The ablation study further validates the three-level belief design can aggregate information and improve task outcomes in both cooperative and negotiating settings.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. Annual Meeting of the Association for Computational Linguistics (ACL), 2017a.
  2. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  3. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 33:1877–1901, 2020.
  4. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. URL http://jmlr.org/papers/v21/20-074.html.
  5. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems (NeurIPS), 35:27730–27744, 2022.
  6. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375, 2022.
  7. OpenAI. Gpt-4 technical report, 2023.
  8. Brant R Burleson. Constructivism: A general theory of communication skill. Explaining communication: Contemporary theories and exemplars, pages 105–128, 2007.
  9. The constructivist approach to communication. In Human communication theory: Comparative essays, pages 147–191. Harper and Row, 1982.
  10. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022.
  11. In situ bidirectional human-robot value alignment. Science Robotics, 7(68):eabm4183, 2022. doi: 10.1126/scirobotics.abm4183. URL https://www.science.org/doi/abs/10.1126/scirobotics.abm4183.
  12. Michal Kosinski. Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083, 2023.
  13. Tomer Ullman. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399, 2023.
  14. Mindgames: Targeting theory of mind in large language models with dynamic epistemic modal logic. arXiv preprint arXiv:2305.03353, 2023.
  15. Learning triadic belief dynamics in nonverbal communication from videos. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 7312–7321, 2021.
  16. The social sense: Susceptibility to others’ beliefs in human infants and adults. Science, 330(6012):1830–1834, 2010.
  17. Development of the social brain from age three to twelve years. Nature communications, 9(1):1027, 2018.
  18. Bayesian theory of mind: Modeling joint belief-desire attribution. In Annual Meeting of the Cognitive Science Society (CogSci), volume 33, 2011.
  19. How much does it help to know what she knows you know? an agent-based simulation study. Artificial Intelligence, 199:67–92, 2013.
  20. Modeling recursive reasoning by humans using empirically informed interactive pomdps. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 1223–1230, 2010.
  21. Learning others’ intentional models in multi-agent settings using interactive pomdps. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.
  22. A cognitive hierarchy model of games. The Quarterly Journal of Economics, 119(3):861–898, 2004.
  23. Using iterated reasoning to predict opponent strategies. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 593–600, 2011.
  24. Machine theory of mind. In International Conference on Machine Learning (ICML), pages 4218–4227. PMLR, 2018.
  25. Coordinate to cooperate or compete: abstract goals and joint intentions in social interaction. In Annual Meeting of the Cognitive Science Society (CogSci), 2016.
  26. Feature-based joint planning and norm learning in collaborative games. In Annual Meeting of the Cognitive Science Society (CogSci), 2016.
  27. Joint inference of states, robot knowledge, and human (false-) beliefs. In International Conference on Robotics and Automation (ICRA), pages 5972–5978. IEEE, 2020.
  28. A framework for endowing an interactive robot with reasoning capabilities about perspective-taking and belief management. In The 23rd IEEE international symposium on robot and human interactive communication, pages 1103–1109. IEEE, 2014.
  29. Who is mistaken? arXiv preprint arXiv:1612.01175, 2016.
  30. Towards socially intelligent agents with mental state transition and human value. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 146–158, Edinburgh, UK, September 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.sigdial-1.16.
  31. Evaluating theory of mind in question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2392–2400, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1261. URL https://aclanthology.org/D18-1261.
  32. Does the autistic child have a “theory of mind”? Cognition, 21(1):37–46, 1985.
  33. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019.
  34. DIALOGPT : Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270–278, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-demos.30. URL https://aclanthology.org/2020.acl-demos.30.
  35. Controllable dialogue generation with disentangled multi-grained style specification and attribute consistency reward. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:188–199, 2022.
  36. A personalized dialogue generator with implicit user persona detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 367–377, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics. URL https://aclanthology.org/2022.coling-1.29.
  37. Commonsense and named entity aware knowledge grounded dialogue generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1322–1335, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.95. URL https://aclanthology.org/2022.naacl-main.95.
  38. Empathetic dialogue generation with pre-trained roberta-gpt2 and external knowledge. In Conversational AI for Natural Human-Centric Interaction: 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021, Singapore, pages 67–81. Springer, 2022.
  39. CHAI: A CHatbot AI for task-oriented dialogue with offline reinforcement learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4471–4491, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.332. URL https://aclanthology.org/2022.naacl-main.332.
  40. Gpt-critic: Offline reinforcement learning for end-to-end task-oriented dialogue systems. In International Conference on Learning Representations (ICLR), 2022.
  41. Integrating common ground and informativeness in pragmatic word learning. 2019.
  42. Carolyn Jane Anderson. Tell me everything you know: a conversation update system for the rational speech acts framework. In Proceedings of the Society for Computation in Linguistics 2021, pages 244–253, 2021.
  43. Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1766–1776, Vancouver, Canada, July 2017b. Association for Computational Linguistics. doi: 10.18653/v1/P17-1162. URL https://aclanthology.org/P17-1162.
  44. MindCraft: Theory of mind modeling for situated dialogue in collaborative tasks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1112–1125, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.85. URL https://aclanthology.org/2021.emnlp-main.85.
  45. CoDraw: Collaborative drawing as a testbed for grounded goal-driven communication. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6495–6513, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1651. URL https://aclanthology.org/P19-1651.
  46. The PhotoBook dataset: Building common ground through visually-grounded dialogue. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1895–1910, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1184. URL https://aclanthology.org/P19-1184.
  47. Maintaining common ground in dynamic environments. Transactions of the Association for Computational Linguistics, 9:995–1011, 2021. doi: 10.1162/tacl_a_00409. URL https://aclanthology.org/2021.tacl-1.59.
  48. On the properties of neural machine translation: Encoder–decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pages 103–111, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/v1/W14-4012. URL https://aclanthology.org/W14-4012.
  49. Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017.
  50. Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information theory, 37(1):145–151, 1991.
  51. Learning to copy coherent knowledge for response generation. In AAAI Conference on Artificial Intelligence (AAAI), volume 35, pages 12535–12543, 2021.
  52. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.703. URL https://aclanthology.org/2020.acl-main.703.
  53. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 10.3115/1073083.1073135. URL https://aclanthology.org/P02-1040.
  54. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013.
  55. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228–231, Prague, Czech Republic, June 2007. Association for Computational Linguistics. URL https://aclanthology.org/W07-0734.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Shuwen Qiu (4 papers)
  2. Song-Chun Zhu (216 papers)
  3. Zilong Zheng (63 papers)
  4. Mingdian Liu (2 papers)
  5. Hengli Li (7 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now