Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
51 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Large Language Models and Games: A Survey and Roadmap (2402.18659v5)

Published 28 Feb 2024 in cs.CL, cs.AI, and cs.HC
Large Language Models and Games: A Survey and Roadmap

Abstract: Recent years have seen an explosive increase in research on LLMs, and accompanying public engagement on the topic. While starting as a niche area within natural language processing, LLMs have shown remarkable potential across a broad range of applications and domains, including games. This paper surveys the current state of the art across the various applications of LLMs in and for games, and identifies the different roles LLMs can take within a game. Importantly, we discuss underexplored areas and promising directions for future uses of LLMs in games and we reconcile the potential and limitations of LLMs within the games domain. As the first comprehensive survey and roadmap at the intersection of LLMs and games, we are hopeful that this paper will serve as the basis for groundbreaking research and innovation in this exciting new field.

PDF HTML Abstract

Surveying the Landscape: LLMs' Roles and Impacts on Games

Introduction

The integration of LLMs into games marks a pivotal chapter in the evolution of game development and research. This comprehensive survey details the current applications, utilities, and roles of LLMs in video games, addressing both the theoretical underpinnings and practical implementations. With the rapid advancement of LLMs, including their expansion into the gaming field, understanding their capabilities, limitations, and potential ethical implications is crucial for future research directions.

Roles of LLMs in Games

LLMs assume various roles within the ecosystem of games, from players to designers, laying a foundation for innovative approaches to game interaction and creation. These roles can be broadly categorized into:

  • Players: LLMs have demonstrated aptitude in playing both turn-based and real-time games, showing potential for reaching or even surpassing human performance levels.
  • Non-Player Characters (NPCs): By driving dynamic, contextually appropriate dialogues, LLMs enhance the realism and depth of NPC interactions, enriching the player's experience.
  • Player Assistants: As an untapped area, the utilization of LLMs as player assistants holds promise for providing gameplay tips, guiding narratives, or managing game mechanics on the player’s behalf.
  • Commentators/Retellers: LLMs have the capability to narratively recount gameplay events, offering a novel way to engage audiences or provide players with reflective summaries of in-game actions.
  • Game Master: Beyond digital confines, LLMs also present opportunities in tabletop games, facilitating narrative progression and management of game dynamics in real-time.
  • Game Mechanic: Through generating content or driving game mechanics with text-based inputs, LLMs introduce unique gameplay elements that can continually evolve.
  • Automated Designer and Design Assistant: From generating game levels to assisting in the creative process, LLMs serve both as generators of game content and collaborators in the design process, pushing the boundaries of traditional game development.

Future Applications and Research Directions

While the existing applications of LLMs in games are broad, several areas remain relatively underexplored, offering fertile ground for future research:

  • Design Assistance: Bridging the gap between conceptual and procedural support, there exists potential for LLMs to play a more collaborative role in the game design process.
  • Player Assistance: The conceptualization and implementation of LLM-driven player assistants could significantly enhance gameplay, providing real-time support, strategic advice, and content management.
  • Commentary and Retelling: Expanding the use of LLMs as commentators and retellers to include streamer support and audience engagement opens new avenues for interactive entertainment.
  • Player Modeling: Addressing one of the less explored areas, applying LLMs for modeling player behavior and experiences could revolutionize personalized gameplay and adaptive game design.

Limitations and Ethical Concerns

Despite their potential, LLMs introduce challenges related to hallucinations, factual inaccuracies, context retention, and user intent capture. Furthermore, ethical concerns regarding sustainability, copyright, explainability, and biases necessitate careful consideration and mitigation strategies to ensure responsible use.

Conclusions

The intersection of LLMs and games presents an exciting frontier with the promise of transforming game experiences, development processes, and player engagement. By addressing technical and ethical challenges, future developments in LLM applications have the potential to enrich the gaming landscape, fostering innovation and inclusivity in interactive digital entertainment.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (118)
  1. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, 2019.
  2. T. Kaufmann, P. Weng, V. Bengs, and E. Hüllermeier, “A survey of reinforcement learning from human feedback,” arXiv preprint arXiv:2312.14925, 2023.
  3. S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, H. Nori, H. Palangi, M. T. Ribeiro, and Y. Zhang, “Sparks of artificial general intelligence: Early experiments with GPT-4,” arXiv preprint arXiv:2303.12712, 2023.
  4. R. T. McCoy, S. Yao, D. Friedman, M. Hardy, and T. L. Griffiths, “Embers of autoregression: Understanding large language models through the problem they are trained to solve,” arXiv preprint arXiv:2309.13638, 2023.
  5. J. Gwertzman and J. Soslow, “The generative AI revolution in games,” https://a16z.com/the-generative-ai-revolution-in-games/, 2022, accessed 25 Feb 2024.
  6. S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao, “Large language models: A survey,” arXiv preprint arXiv:2402.06196, 2024.
  7. C. E. Shannon, “A mathematical theory of communication,” The Bell system technical journal, vol. 27, no. 3, 1948.
  8. J. L. Elman, “Finding structure in time,” Cognitive science, vol. 14, no. 2, 1990.
  9. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proceedings of the Conference on Neural Information Processing Systems, 2017.
  10. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7B,” arXiv preprint arXiv:2310.06825, 2023.
  11. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023.
  12. Z. Yang, L. Li, K. Lin, J. Wang, C.-C. Lin, Z. Liu, and L. Wang, “The dawn of LMMs: Preliminary explorations with GPT-4V(ision),” arXiv preprint arXiv:2309.17421, 2023.
  13. H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” in Proceedings of the Neural Information Processing Systems Conference, 2023.
  14. A. Summerville, S. Snodgrass, M. Guzdial, C. Holmgård, A. K. Hoover, A. Isaksen, A. Nealen, and J. Togelius, “Procedural content generation via machine learning (PCGML),” IEEE Transactions on Games, vol. 10, no. 3, 2018.
  15. D. Noever, M. Ciolino, and J. Kalin, “The chess transformer: Mastering play using generative language models,” arXiv preprint arXiv:2008.04057, 2020.
  16. A. Stöckl, “Watching a language model learning chess,” in Proceedings of the Recent Advances in Natural Language Processing International Conference, 2021.
  17. S. Toshniwal, S. Wiseman, K. Livescu, and K. Gimpel, “Chess as a testbed for language model state tracking,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 10, 2022.
  18. M. Ciolino, J. Kalin, and D. Noever, “The Go transformer: Natural language modeling for game play,” in Proceedings of the Artificial Intelligence for Industries Internation Conference, 2020.
  19. K. Li, A. K. Hopkins, D. Bau, F. Viégas, H. Pfister, and M. Wattenberg, “Emergent world representations: Exploring a sequence model trained on a synthetic task,” in Proceedings of the International Conference on Learning Representations, 2023.
  20. S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y. Sulsky, J. Kay, J. T. Springenberg et al., “A generalist agent,” arXiv preprint arXiv:2205.06175, 2022.
  21. M. Xu, W. Dai, C. Liu, X. Gao, W. Lin, G.-J. Qi, and H. Xiong, “Spatial-temporal transformer networks for traffic flow forecasting,” arXiv preprint arXiv:2001.02908, 2020.
  22. H. Chang, H. Zhang, L. Jiang, C. Liu, and W. T. Freeman, “MaskGIT: Masked generative image transformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  23. S. Yao, R. Rao, M. Hausknecht, and K. Narasimhan, “Keep CALM and explore: Language models for action generation in text-based games,” in Proceedings of the Empirical Methods in Natural Language Processing Conference, 2020.
  24. C. F. Tsai, X. Zhou, S. S. Liu, J. Li, M. Yu, and H. Mei, “Can large language models play text games well? current state-of-the-art and open questions,” arXiv preprint arXiv:2304.02868, 2023.
  25. Meta Fundamental AI Research Diplomacy Team (FAIR), A. Bakhtin, N. Brown, E. Dinan, G. Farina, C. Flaherty, D. Fried, A. Goff, J. Gray, H. Hu et al., “Human-level play in the game of diplomacy by combining language models with strategic reasoning,” Science, vol. 378, no. 6624, 2022.
  26. G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An open-ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291, 2023.
  27. GoodAI, “Introducing our work on general-purpose llm agents,” https://www.goodai.com/introducing-general-purpose-llm-agents/, 2023, accessed 24 Feb 2024.
  28. M. Ç. Uludağlı and K. Oğuz, “Non-player character decision-making in computer games,” Artificial Intelligence Review, vol. 56, no. 12, 2023.
  29. M. Shanahan, K. McDonell, and L. Reynolds, “Role play with large language models,” Nature, vol. 623, p. 493–498, 2023.
  30. H. Warpefelt and H. Verhagen, “A model of non-player character believability,” Journal of Gaming & Virtual Worlds, vol. 9, no. 1, 2017.
  31. J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkhura, and K. Xu, “Shifting attention to relevance: Towards the uncertainty estimation of large language models,” arXiv preprint arXiv:2307.01379, 2023.
  32. A. Mehta, Y. Kunjadiya, A. Kulkarni, and M. Nagar, “Exploring the viability of conversational AI for Non-Playable Characters: A comprehensive survey,” in Proceedings of the 4th International Conference on Recent Trends in Computer Science and Technology, 2022.
  33. E. Akata, L. Schulz, J. Coda-Forno, S. J. Oh, M. Bethge, and E. Schulz, “Playing repeated games with large language models,” arXiv preprint arXiv:2305.16867, 2023.
  34. Y. Xu, S. Wang, P. Li, F. Luo, X. Wang, W. Liu, and Y. Liu, “Exploring large language models for communication games: An empirical study on Werewolf,” arXiv preprint arXiv:2309.04658, 2023.
  35. P. Maas, F. Carey, C. Wheeler, E. Saatchi, P. Billington, and J. Yaffa Shamash, “To infinity and beyond: SHOW-1 and showrunner agents in multi-agent simulations,” https://fablestudio.github.io/showrunner-agents/, 2023, accessed 27 Feb 2024.
  36. J. Urbanek, A. Fan, S. Karamcheti, S. Jain, S. Humeau, E. Dinan, T. Rocktäschel, D. Kiela, A. Szlam, and J. Weston, “Learning to speak and act in a fantasy text adventure game,” in Proceedings of the Empirical Methods in Natural Language Processing Conference and the Natural Language Processing International Joint Conference, 2019.
  37. M. P. Eladhari, “Re-tellings: The fourth layer of narrative as an instrument for critique,” in Proceedings of the Interactive Digital Storytelling International Conference, 2018.
  38. M. Guzdial, S. Shah, and M. Riedl, “Towards automated let’s play commentary,” in Proceedings of the AIIDE workshop on Experimental AI in Games, 2018.
  39. T. Ishigaki, G. Topić, Y. Hamazono, H. Noji, I. Kobayashi, Y. Miyao, and H. Takamura, “Generating racing game commentary from vision, language, and structured data,” in Proceedings of the Natural Language Generation International Conference, 2021.
  40. C. Li, S. Gandhi, and B. Harrison, “End-to-end let’s play commentary generation using multi-modal video representations,” in Proceedings of the Foundations of Digital Games International Conference, 2019.
  41. N. Renella and M. Eger, “Towards automated video game commentary using generative AI,” in Proceedings of the AIIDE workshop on Experimental AI in Games, 2023.
  42. A. Tychsen, M. Hitchens, T. Brolund, and M. Kavakli, “The game master,” in Proceedings of the Australasian conference on Interactive entertainment, 2005.
  43. A. Liapis and A. Denisova, “The challenge of evaluating player experience in tabletop role-playing games,” in Proceedings of the Foundations of Digital Games International Conference, 2023.
  44. A. Liapis, “The discourse on computational creativity in indie tabletop role-playing games,” in preparation.
  45. M. Hua and R. Raley, “Playing with unicorns: AI dungeon and citizen NLP,” Digital Humanities Quarterly, vol. 14, no. 4, 2020.
  46. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  47. A. Zhu, L. Martin, A. Head, and C. Callison-Burch, “CALYPSO: LLMs as Dungeon Masters’ assistants,” in Proceedings of the Nineteenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2023.
  48. J. Kelly, M. Mateas, and N. Wardrip-Fruin, “Towards computational support with language models for TTRPG game masters,” in Proceedings of the FDG workshop on Human-AI Interaction through Play, 2023.
  49. D. Acharya, J. Kelly, W. Tate, M. Joslyn, M. Mateas, and N. Wardrip-Fruin, “Shoelace: A storytelling assistant for GUMSHOE One-2-One,” in Proceedings of the Foundations of Digital Games International Conference, 2023.
  50. Y. Sun, Z. Li, K. Fang, C. H. Lee, and A. Asadipour, “Language as reality: A co-creative storytelling game experience in 1001 nights using generative AI,” in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2023.
  51. M. Treanor, A. Zook, M. P. Eladhari, J. Togelius, G. Smith, M. Cook, T. Thompson, B. Magerko, J. Levine, and A. Smith, “AI-based game design patterns,” in Proceedings of the Foundations of Digital Games International Conference, 2015.
  52. J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” in Proceedings of the Annual ACM Symposium on User Interface Software and Technology, 2023.
  53. Y. Liu, G. Deng, Z. Xu, Y. Li, Y. Zheng, Y. Zhang, L. Zhao, T. Zhang, and Y. Liu, “Jailbreaking ChatGPT via prompt engineering: An empirical study,” arXiv preprint arXiv:2305.13860, 2023.
  54. T. Litchfield, “This browser-based ‘endless crafting game’ starts you off with fire and water, but it quickly escalates to God, the Big Bang, and ‘Yin-Yoda’,” https://www.pcgamer.com/this-browser-based-endless-crafting-game-starts-you-off-with-fire-and-water-but-it-quickly-escalates-to-god-the-big-bang-and-yin-yoda/, 2024, accessed 28 Feburary 2024.
  55. A. J. Summerville, S. Snodgrass, M. Mateas, and S. Ontanon, “The VGLC: The video game level corpus,” in Proceedings of the FDG workshop on Procedural Content Generation, 2016.
  56. A. Summerville and M. Mateas, “Super Mario as a string: Platformer level generation via LSTMs,” in Proceedings of the Joint Conference of DIGRA and FDG, 2016.
  57. G. Todd, S. Earle, M. U. Nasir, M. C. Green, and J. Togelius, “Level generation through large language models,” in Proceedings of the Foundations of Digital Games International Conference, 2023.
  58. C. Bamford, “Griddly: A platform for AI research in games,” Software Impacts, vol. 8, 2021.
  59. S. Sudhakaran, M. González-Duque, M. Freiberger, C. Glanois, E. Najarro, and S. Risi, “MarioGPT: Open-ended text2level generation through large language models,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  60. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2019.
  61. A. Liapis, “Searching for sentient design tools for game development,” Ph.D. dissertation, Center for Computer Games, IT University of Copenhagen, Copenhagen, Denmark, 2014.
  62. G. Smith, J. Whitehead, and M. Mateas, “Tanagra: Reactive planning and constraint solving for mixed-initiative level design,” IEEE Transactions on Computational Intelligence and AI in Games, vol. 3, no. 3, 2011.
  63. A. Liapis, G. N. Yannakakis, and J. Togelius, “Sentient sketchbook: Computer-aided game level authoring,” in Proceedings of the Foundations of Digital Games Conference, 2013.
  64. P. Migkotzidis and A. Liapis, “SuSketch: Surrogate models of gameplay as a design assistant,” IEEE Transactions on Games, vol. 14, no. 2, 2021.
  65. M. Charity, I. Dave, A. Khalifa, and J. Togelius, “Baba is Y’all 2.0: Design and investigation of a collaborative mixed-initiative system,” IEEE Transactions on Games, 2022, early Access.
  66. R. Gallotta, K. Arulkumaran, and L. B. Soros, “Preference-learning emitters for mixed-initiative Quality-Diversity algorithms,” IEEE Transactions on Games, 2023, early Access.
  67. G. N. Yannakakis, A. Liapis, and C. Alexopoulos, “Mixed-initiative co-creativity,” in Proceedings of the Foundations of Digital Games Conference, 2014.
  68. A. Liapis, G. N. Yannakakis, and J. Togelius, “Sentient world: Human-based procedural cartography,” in Proceedings of Evolutionary and Biologically Inspired Music, Sound, Art and Design Conference, 2013, vol. 7834, LNCS.
  69. D. Novick and S. Sutton, “What is mixed-initiative interaction?” in Proceedings of the AAAI Spring Symposium on Computational Models for Mixed Initiative Interaction, 1997.
  70. A. Liapis, G. N. Yannakakis, and J. Togelius, “Designer modeling for personalized game content creation tools,” in Proceedings of the AIIDE Workshop on Artificial Intelligence & Game Aesthetics, 2013.
  71. V. Vimpari, A. Kultima, P. Hämäläinen, and C. Guckelsberger, ““An adapt-or-die type of situation”: Perception, adoption, and use of text-to-image-generation AI by game industry professionals,” in Proceedings of the ACM on Human-Computer Interaction, vol. 7, 2023.
  72. J. Boucher, G. Smith, and Y. Telliel, “Examining early professionals’ use of generative AI in the game development process,” in Proceedings of the AIIDE Workshop on Experimental Artificial Intelligence in Games, 2023.
  73. M. Charity, Y. Bhartia, D. Zhang, A. Khalifa, and J. Togelius, “A Preliminary Study on a Conceptual Game Feature Generation and Recommendation System,” in Proceedings of the IEEE Conference on Games, 2023.
  74. M. U. Nasir and J. Togelius, “Practical PCG through large language models,” in Proceedings of the IEEE Conference on Games, 2023.
  75. V. Kumaran, D. Carpenter, J. Rowe, B. Mott, and J. Lester, “End-to-end procedural level generation in educational games with natural language instruction,” in Proceedings of the IEEE Conference on Games, 2023.
  76. P. Taveekitworachai, F. Abdullah, M. F. Dewantoro, R. Thawonmas, J. Togelius, and J. Renz, “ChatGPT4PCG competition: Character-like level generation for science birds,” arXiv preprint arXiv:2303.15662, 2023.
  77. T. Brooks, A. Holynski, and A. A. Efros, “InstructPix2Pix: Learning to follow image editing instructions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  78. J. Zhu, A. Liapis, S. Risi, R. Bidarra, and G. M. Youngblood, “Explainable AI for designers: A human-centered perspective on mixed-initiative co-creation,” in Proceedings of the IEEE Conference on Computational Intelligence and Games, 2018.
  79. A. Liapis, M. Awiszus, A. J. Champandard, M. Cook, A. Denisova, A. Dockhorn, T. Thompson, and J. Zhu, “Artificial intelligence for audiences,” in Human-Game AI Interaction (Dagstuhl Seminar 22251), D. Ashlock, S. Maghsudi, D. P. Liebana, P. Spronck, and M. Eberhardinger, Eds., 2023.
  80. G. N. Yannakakis and D. Melhart, “Affective game computing: A survey,” Proceedings of the IEEE, 2023.
  81. G. N. Yannakakis and J. Togelius, “Experience-driven procedural content generation,” IEEE Transactions on Affective Computing, vol. 2, no. 3, pp. 147–161, 2011.
  82. K. Pinitas, D. Renaudie, M. Thomsen, M. Barthet, K. Makantasis, A. Liapis, and G. N. Yannakakis, “Predicting player engagement in Tom Clancy’s The Division 2: A multimodal approach via pixels and gamepad actions,” in Proceedings of the ACM International Conference on Multimodal Interaction, 2023.
  83. D. Melhart, A. Liapis, and G. N. Yannakakis, “The Arousal video Game AnnotatIoN (AGAIN) dataset,” IEEE Transactions on Affective Computing, vol. 13, no. 4, 2022.
  84. N. M. M. Shafiullah, Z. J. Cui, A. Altanzaya, and L. Pinto, “Behavior transformers: Cloning k𝑘kitalic_k modes with one stone,” in Proceedings of the Neural Information Processing Systems Conference, 2022.
  85. J. Pfau, A. Liapis, G. N. Yannakakis, and R. Malaka, “Dungeons & Replicants II: Automated game balancing across multiple difficulty dimensions via deep player behavior modeling,” IEEE Transactions on Games, 2022.
  86. S. Johnson-Bey, M. Mateas, and N. Wardrip-Fruin, “Toward using ChatGPT to generate theme-relevant simulated storyworlds,” in Proceedings of the AIIDE Workshop on Experimental Artificial Intelligence in Games, 2023.
  87. M. Kreminski, D. Acharya, N. Junius, E. Oliver, K. Compton, M. Dickinson, C. Focht, S. Mason, S. Mazeika, and N. Wardrip-Fruin, “Cozy mystery construction kit: Prototyping toward an AI-assisted collaborative storytelling mystery game,” in Proceedings of the Foundations of Digital Games International Conference, 2019.
  88. C. Fernández-Vara and A. Thomson, “Procedural generation of narrative puzzles in adventure games: The puzzle-dice system,” in Proceedings of the Workshop on Procedural Content Generation in Games, 2012.
  89. J. Dormans, “Adventures in level design: generating missions and spaces for action adventure games,” in Proceedings of the Workshop on Procedural Content Generation in Games, 2010.
  90. P. Manakul, A. Liusie, and M. J. F. Gales, “SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models,” in Proceedings of the Empirical Methods in Natural Language Processing Conference, 2023.
  91. A. Strasser, “On pitfalls (and advantages) of sophisticated large language models,” arXiv preprint arXiv:2303.17511, 2023.
  92. N. Bian, H. Lin, P. Liu, Y. Lu, C. Zhang, B. He, X. Han, and L. Sun, “Influence of external information on large language models mirrors social cognitive patterns,” arXiv preprint arXiv:2305.04812, 2023.
  93. M. Karpinska and M. Iyyer, “Large language models effectively leverage document-level context for literary translation, but critical errors persist,” in Proceedings of the Machine Translation Conference, 2023.
  94. Z. Gekhman, J. Herzig, R. Aharoni, C. Elkind, and I. Szpektor, “TrueTeacher: Learning factual consistency evaluation with large language models,” in Proceedings of the Empirical Methods in Natural Language Processing Conference, 2023.
  95. J. Zhou, “An evaluation of state-of-the-art large language models for sarcasm detection,” arXiv preprint arXiv:2312.03706, 2023.
  96. Y. Liu, T. Han, S. Ma, J. Zhang, Y. Yang, J. Tian, H. He, A. Li, M. He, Z. Liu, Z. Wu, L. Zhao, D. Zhu, X. Li, N. Qiang, D. Shen, T. Liu, and B. Ge, “Summary of chatgpt-related research and perspective towards the future of large language models,” Meta-Radiology, vol. 1, no. 2, 2023.
  97. D. Li, A. S. Rawat, M. Zaheer, X. Wang, M. Lukasik, A. Veit, F. Yu, and S. Kumar, “Large language models with controllable working memory,” in Findings of the Association for Computational Linguistics, 2023.
  98. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Proceedings of the Conference on Neural Information Processing Systems, 2020.
  99. X. Miao, G. Oliaro, Z. Zhang, X. Cheng, H. Jin, T. Chen, and Z. Jia, “Towards efficient generative large language model serving: A survey from algorithms to systems,” arXiv preprint arXiv:2312.15234, 2023.
  100. F. de la Torre, C. M. Fang, H. Huang, A. Banburski-Fahey, J. A. Fernandez, and J. Lanier, “LLMR: Real-time prompting of interactive worlds using large language models,” arXiv preprint arXiv:2309.12276, 2023.
  101. D. Melhart, J. Togelius, B. Mikkelsen, C. Holmgård, and G. N. Yannakakis, “The ethics of AI in games,” IEEE Transactions on Affective Computing, 2023.
  102. A. A. Chien, L. Lin, H. Nguyen, V. Rao, T. Sharma, and R. Wijayawardana, “Reducing the carbon impact of generative AI inference (today and in 2035),” in Proceedings of the SCS Workshop on Sustainable Computer Systems, 2023.
  103. S. A. Khowaja, P. Khuwaja, and K. Dev, “ChatGPT needs SPADE (sustainability, PrivAcy, digital divide, and ethics) evaluation: A review,” arXiv preprint arXiv:2305.03123, 2023.
  104. Free Law Project, “Authors Guild v. OpenAI Inc.” https://www.courtlistener.com/docket/67810584/authors-guild-v-openai-inc/, accessed 27 Feb 2024.
  105. J. A. Rothchild, “Copyright implications of the use of code repositories to train a machine learning model,” Free Software Foundation, 2022.
  106. HackerNews, “Github copilot,” https://news.ycombinator.com/item?id=27676266, 2021, accessed 27 Feb 2024.
  107. P. Zhang, G. Zeng, T. Wang, and W. Lu, “Tinyllama: An open-source small language model,” 2024.
  108. U.S. Copyright Office Review Board, “Decision affirming refusal of registration of A Recent Entrance to Paradise,” 2022.
  109. ——, “Registration decision on Zarya of the Dawn,” 2023.
  110. C. E. Lamb and D. G. Brown, “Should we have seen the coming storm? Transformers, society, and CC,” in Proceedings of the International Conference on Computational Creativity, 2023.
  111. Y. Liu, Y. Yao, J.-F. Ton, X. Zhang, R. Guo, H. Cheng, Y. Klochkov, M. F. Taufiq, and H. Li, “Trustworthy LLMs: a survey and guideline for evaluating large language models’ alignment,” in Proceedings of the NeurIPS Workshop on Socially Responsible Language Modelling Research, 2023.
  112. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” in Proceedings of the Advances in Neural Information Processing Systems Conference, 2022.
  113. S. Chen, B. Li, and D. Niu, “Boosting of thoughts: Trial-and-error problem solving with large language models,” arXiv preprint arXiv:2402.11140, 2024.
  114. D. Mondal, S. Modi, S. Panda, R. Singh, and G. S. Rao, “KAM-CoT: Knowledge augmented multimodal Chain-of-Thoughts reasoning,” arXiv preprint arXiv:2401.12863, 2024.
  115. M. Turpin, J. Michael, E. Perez, and S. R. Bowman, “Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting,” in Proceedings of the Neural Information Processing Systems Conference, 2023.
  116. S. Frieder, L. Pinchetti, A. Chevalier, R.-R. Griffiths, T. Salvatori, T. Lukasiewicz, P. C. Petersen, and J. Berner, “Mathematical Capabilities of ChatGPT,” in Proceedings of the Neural Information Processing Systems Conference, 2023.
  117. Y. Jia, W. Wu, F. Cao, and S. C. Han, “In-game toxic language detection: Shared task and attention residuals,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
  118. J. Thomas, N. Chorakhalikar, A. Dantrey, R. R. Nalla, and P. Mehta, “Automatic classification and reporting of inappropriate language in online applications,” US Patent 20 210 370 188, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Roberto Gallotta (8 papers)
  2. Graham Todd (9 papers)
  3. Marvin Zammit (3 papers)
  4. Sam Earle (25 papers)
  5. Antonios Liapis (55 papers)
  6. Julian Togelius (154 papers)
  7. Georgios N. Yannakakis (59 papers)
Citations (52)
View on Semantic Scholar

HackerNews

  1. Large Language Models and Games: A Survey and Roadmap (3 points, 0 comments)