Collaborative Quest Completion with LLM-driven Non-Player Characters in Minecraft (2407.03460v1)
Abstract: The use of generative AI in video game development is on the rise, and as the conversational and other capabilities of LLMs continue to improve, we expect LLM-driven non-player characters (NPCs) to become widely deployed. In this paper, we seek to understand how human players collaborate with LLM-driven NPCs to accomplish in-game goals. We design a minigame within Minecraft where a player works with two GPT4-driven NPCs to complete a quest. We perform a user study in which 28 Minecraft players play this minigame and share their feedback. On analyzing the game logs and recordings, we find that several patterns of collaborative behavior emerge from the NPCs and the human players. We also report on the current limitations of language-only models that do not have rich game-state or visual understanding. We believe that this preliminary study and analysis will inform future game developers on how to better exploit these rapidly improving generative AI models for collaborative roles in games.
- Qi Chen Gao and Ali Emami. 2023. The turing quest: Can transformers make good npcs? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 93–103.
- A Kalbiyev. 2022. Affective dialogue generation for video games. Master’s thesis, University of Twente.
- Scenecraft: automating interactive narrative scene generation in digital games with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 19, pages 86–96.
- Muhammad U Nasir and Julian Togelius. 2023. Practical pcg through large language models. arXiv preprint arXiv:2305.18243.
- OpenAI. 2023. Gpt-4 technical report. ArXiv, abs/2303.08774.
- Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22.
- Player-driven emergence in llm-driven game narrative. arXiv preprint arXiv:2404.17027.
- Investigating agency of llms in human-ai collaboration tasks. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1968–1987.
- Level generation through large language models. In Proceedings of the 18th International Conference on the Foundations of Digital Games, pages 1–8.
- Generating role-playing game quests with gpt language models. IEEE Transactions on Games.
- Craft an iron sword: Dynamically generating interactive game characters by prompting large language models tuned on code. In Proceedings of the 3rd Wordplay: When Language Meets Games Workshop (Wordplay 2022), pages 25–43.
- From human-human collaboration to human-ai collaboration: Designing ai systems that can work together with people. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI EA ’20, page 1–6, New York, NY, USA. Association for Computing Machinery.
- Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291.
- Sudha Rao (23 papers)
- Weijia Xu (23 papers)
- Michael Xu (18 papers)
- Jorge Leandro (3 papers)
- Ken Lobb (2 papers)
- Gabriel DesGarennes (2 papers)
- Chris Brockett (37 papers)
- Bill Dolan (45 papers)