Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Open-Ended Wargames with Large Language Models (2404.11446v1)

Published 17 Apr 2024 in cs.CL, cs.AI, and cs.CY

Abstract: Wargames are a powerful tool for understanding and rehearsing real-world decision making. Automated play of wargames using AI enables possibilities beyond those of human-conducted games, such as playing the game many times over to see a range of possible outcomes. There are two categories of wargames: quantitative games, with discrete types of moves, and qualitative games, which revolve around open-ended responses. Historically, automation efforts have focused on quantitative games, but LLMs make it possible to automate qualitative wargames. We introduce "Snow Globe," an LLM-powered multi-agent system for playing qualitative wargames. With Snow Globe, every stage of a text-based qualitative wargame from scenario preparation to post-game analysis can be optionally carried out by AI, humans, or a combination thereof. We describe its software architecture conceptually and release an open-source implementation alongside this publication. As case studies, we simulate a tabletop exercise about an AI incident response and a political wargame about a geopolitical crisis. We discuss potential applications of the approach and how it fits into the broader wargaming ecosystem.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. S. Burns, D. DellaVolpe, R. Babb, N. Miller, and G. Muir, “War gamers’ handbook: A guide for professional war gamers,” tech. rep., Naval War College, Newport, RI, USA, November 2015.
  2. M. F. Cancian, M. Cancian, and E. Heginbotham, “The first battle of the next war: Wargaming a Chinese invasion of Taiwan,” report, Center for Strategic & International Studies, Washington, DC, USA, January 2023.
  3. P. K. Davis and P. J. E. Stan, “Concepts and models of escalation,” Report R-3235, The RAND Corp., Santa Monica, CA, USA, May 1984.
  4. J. Goodman, S. Risi, and S. Lucas, “AI and wargaming.” arXiv:2009.08922, 2020.
  5. P. K. Davis and P. Bracken, “Artificial intelligence for wargaming and modeling,” Journal of Defense Modeling and Simulation, February 2022.
  6. E. Lin-Greenberg, R. B. C. Pauly, and J. G. Schneider, “Wargaming for international relations research,” European Journal of International Relations, vol. 28, no. 1, pp. 83–109, 2022.
  7. H. Goldhamer and H. Speier, “Some observations on political gaming,” Report P-1679-RC, The RAND Corp., Santa Monica, CA, USA, April 1959.
  8. R. B. C. Pauly, “Would U.S. leaders push the button?,” International Security, vol. 43, Fall 2018.
  9. Dept. of Homeland Security, “Homeland Security Exercise and Evaluation Program (HSEEP).” https://www.fema.gov/sites/default/files/2020-04/Homeland-Security-Exercise-and-Evaluation-Program-Doctrine-2020-Revision-2-2-25.pdf, January 2020.
  10. L. Radow, “Tabletop exercise instructions for planned events and unplanned incidents/emergencies,” Report FHWA-HOP-15-004, Federal Highway Admin., Washington, DC, USA, November 2014.
  11. Cybersecurity and Infrastructure Security Agency, “CISA tabletop exercise packages.” https://www.cisa.gov/sites/default/files/2023-02/ctep_fact_sheet_v._11_16_2021_final.pdf, 2021.
  12. IQT Labs, “HAIWIRE.” https://github.com/IQTLabs/hAIwire, 2023.
  13. Development, Concepts, and Doctrine Centre, “Wargaming handbook,” tech. rep., Ministry of Defence, August 2017.
  14. R. T. Culkin, “Post-Cold War wargaming and the American military leadership challenge,” Master’s thesis, Air Command and Staff College, Maxwell Air Force Base, AL, USA, March 1999.
  15. E. M. Bartels, Building Better Games for National Security Policy Analysis. Dissertation, Pardee RAND Graduate School, Santa Monica, CA, USA, March 2020.
  16. E. Geist, A. B. Frank, and L. Menthe, “Understanding the limits of artificial intelligence for warfighters: Volume 4, wargames,” Research Report RRA1722-4, The RAND Corp., Santa Monica, CA, USA, January 2024.
  17. J. R. Emery, “Moral choices without moral language: 1950s political-military wargaming at the RAND Corporation,” Texas National Security Review, vol. 4, pp. 11–31, Fall 2021.
  18. R. Axelrod and W. D. Hamilton, “The evolution of cooperation,” Science, vol. 211, pp. 1390 – 1396, March 1981.
  19. A. Bravetti and P. Padilla, “An optimal strategy to solve the Prisoner’s Dilemma,” Scientific Reports, vol. 8, January 2018.
  20. B. O’Neill, “Game theory models of peace and war,” in Handbook of Game Theory, vol. 2, ch. 29, pp. 995 – 1053, Elsevier, 1994.
  21. A. Knack and R. Powell, “Artificial intelligence in wargaming: An evidence-based assessment of AI applications,” CETaS research report, The Alan Turing Institute, June 2023.
  22. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
  23. S. Bubeck et al., “Sparks of artificial general intelligence: Early experiments with GPT-4.” arXiv:2303.12712, 2023.
  24. J. Huang and K. C.-C. Chang, “Towards reasoning in large language models: A survey,” Findings of the Association for Computational Linguistics: ACL 2023, pp. 1049 – 1065, July 2023.
  25. M. D. Millot, R. Molander, and P. A. Wilson, “The day after… study: Nuclear proliferation in the Post-Cold War world: Volume III, exercise materials,” Monograph Report MR-267-AF, The RAND Corp., Santa Monica, CA, USA, 1993.
  26. T. Mouat, “Baltic challenge.” http://www.mapsymbs.com/BalticChallenge.pdf, October 2016.
  27. Z. Yang, L. Li, K. Lin, J. Wang, C.-C. Lin, Z. Liu, and L. Wang, “The dawn of LMMs: preliminary explorations with GPT-4V(ision).” arXiv:2309.17421, 2023.
  28. A. Bakhtin et al., “Human-level play in the game of Diplomacy by combining language models with strategic reasoning,” Science, vol. 378, pp. 1067 – 1074, November 2022.
  29. W. Hua, L. Fan, L. Li, K. Mei, J. Ji, Y. Ge, L. Hemphill, and Y. Zhang, “War and peace (WarAgent): large language model-based multi-agent simulation of world.” arXiv:2311.17227, 2023.
  30. J.-P. Rivera, G. Mukobi, A. Reuel, M. Lamparth, C. Smith, and J. Schneider, “Escalation risks from language models in military and diplomatic decision-making.” arXiv:2401.03408, 2024.
  31. M. Lamparth, A. Corso, J. Ganz, O. Skylar Mastro, J. Schneider, and H. Trinkunas, “Human vs. machine: Language models and wargames.” arXiv:2403.03407, 2024.
  32. L. D. Griffin, “AI & wargaming,” in Connections UK 2023, September 2023.
  33. R. Brynen, “ChatGPT plays a matrix game.” https://paxsims.wordpress.com/2024/02/28/chatgpt-plays-a-matrix-game/, 2024.
  34. R. Gallotta, G. Todd, M. Zammit, S. Earle, A. Liapis, J. Togelius, and G. N. Yannakakis, “Large language models and games: A survey and roadmap.” arXiv:2402.18659, 2024.
  35. G. Gerganov, “llama.cpp.” https://github.com/ggerganov/llama.cpp, 2024.
  36. Hugging Face, “Transformers.” https://github.com/huggingface/transformers, 2024.
  37. G. Brockman, M. Murati, and P. Welinder, “OpenAI API.” https://openai.com/blog/openai-api, June 2020.
  38. H. Chase, “LangChain.” https://github.com/langchain-ai/langchain, 2024.
  39. Mistral AI, “Mistral-7B-v0.1.” https://huggingface.co/mistralai/Mistral-7B-v0.1, 2023.
  40. Open-Orca, “Mistral-7B-OpenOrca.” https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca, 2023.
  41. T. Jobbins, “Mistral-7B-OpenOrca-GGUF.” https://huggingface.co/TheBloke/Mistral-7B-OpenOrca-GGUF, 2023.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets