Latent Combinational Game Design (2206.14203v3)
Abstract: We present latent combinational game design -- an approach for generating playable games that blend a given set of games in a desired combination using deep generative latent variable models. We use Gaussian Mixture Variational Autoencoders (GMVAEs) which model the VAE latent space via a mixture of Gaussian components. Through supervised training, each component encodes levels from one game and lets us define blended games as linear combinations of these components. This enables generating new games that blend the input games as well as controlling the relative proportions of each game in the blend. We also extend prior blending work using conditional VAEs and compare against the GMVAE and additionally introduce a hybrid conditional GMVAE (CGMVAE) architecture which lets us generate whole blended levels and layouts. Results show that these approaches can generate playable games that blend the input games in specified combinations. We use both platformers and dungeon-based games to demonstrate our results.
- A. Summerville, S. Snodgrass, M. Guzdial, C. Holmgård, A. K. Hoover, A. Isaksen, A. Nealen, and J. Togelius, “Procedural content generation via machine learning (PCGML),” IEEE Transactions on Games, 2018.
- A. Sarkar, M. Guzdial, S. Snodgrass, A. Summerville, T. Machado, and G. Smith, “Procedural content generation via knowledge transformation (PCG-KT),” IEEE Transactions on Games, 2023.
- J. Gow and J. Corneli, “Towards generating novel games using conceptual blending,” in Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2015.
- D. Kingma and M. Welling, “Auto-encoding variational Bayes,” in International Conference on Learning Representations (ICLR), 2013.
- A. Sarkar, Z. Yang, and S. Cooper, “Controllable level blending between games using variational autoencoders,” in EXAG Workshop, 2019.
- A. Sarkar, A. Summerville, S. Snodgrass, G. Bentley, and J. Osborn, “Exploring level blending across platformers via paths and affordances,” in Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2020.
- A. Sarkar, Z. Yang, and S. Cooper, “Conditional level generation and game blending,” in EXAG Workshop, 2020.
- A. Sarkar and S. Cooper, “Dungeon and platformer level blending and generation using conditional vaes,” in IEEE Conference on Games (CoG), 2021, pp. 1–8.
- M. A. Boden, “The creative mind: Myths and mechanisms,” 2004.
- N. Dilokthanakul, P. A. Mediano, M. Garnelo, M. C. Lee, H. Salimbeni, K. Arulkumaran, and M. Shanahan, “Deep unsupervised clustering with gaussian mixture variational autoencoders,” arXiv preprint arXiv:1611.02648, 2016.
- R. Shu, “Gaussian mixture VAE: Lessons in variational inference, generative models, and deep nets,” 2016. [Online]. Available: http://ruishu.io/2016/12/25/gmvae/
- K. Sohn, H. Lee, and X. Yan, “Learning structured output representation using deep conditional generative models,” in NeurIPS, 2015.
- M. Guzdial and M. O. Riedl, “Conceptual game expansion,” IEEE Transactions on Games, vol. 14, no. 1, pp. 93–106, 2021.
- S. Snodgrass and S. Ontanon, “An approach to domain transfer in procedural content generation of two-dimensional videogame levels,” in Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2016.
- Z. Yang, A. Sarkar, and S. Cooper, “Game level clustering and generation using gaussian mixture VAEs,” in Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2020.
- A. Cao, Y. Luo, and D. Klabjan, “Open-set recognition with gaussian mixture variational autoencoders,” in AAAI Conference on Artificial Intelligence, 2021.
- A. Abdulaziz, J. Zhou, A. Di Fulvio, Y. Altmann, and S. McLaughlin, “Semi-supervised gaussian mixture variational autoencoder for pulse shape discrimination,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022.
- X. Yan, J. Yang, K. Sohn, and H. Lee, “Attribute2image: Conditional image generation from visual attributes,” arXiv preprint arXiv:1512.00570, 2015.
- G. Fauconnier and M. Turner, “Conceptual integration networks,” Cognitive Science, vol. 22, no. 2, pp. 133–187, 1998.
- A. Sarkar and S. Cooper, “Towards game design via creative machine learning (GDCML),” in IEEE Conference on Games (CoG), 2020.
- S. Cooper, “Constraint-based 2D tile game blending in the Sturgeon system,” in EXAG Workshop, 2022.
- A. Summerville, A. Sarkar, S. Snodgrass, and J. Osborn, “Extracting physics from blended platformer game levels,” in EXAG Workshop, 2020.
- A. Summerville, J. Osborn, C. Holmgard, and D. W. Zhang, “Mechanics automatically recognized via interactive observation: Jumping,” in International Conference on Foundations of Digital Games, 2017.
- M. Mirza and S. Osindero, “Conditional generative adversarial networks,” arXiv preprint arXiv:1411.1784, 2014.
- A. Summerville, S. Snodgrass, M. Mateas, and S. Ontañón, “The VGLC: The video game level corpus,” in 7th Workshop on PCG at 1st Joint International Conference of DiGRA and FDG, 2016.
- A. Sarkar and S. Cooper, “Generating and blending game levels via quality-diversity in the latent space of a variational autoencoder,” in 16th International Conference on the Foundations of Digital Games, 2021.
- A. Summerville and M. Mateas, “Super Mario as a string: Platformer level generation via LSTMs,” 1st International Joint Conference of DiGRA and FDG, 2016.
- A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, 2019.
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg et al., “Scikit-learn: Machine learning in Python,” the Journal of machine Learning research, vol. 12, pp. 2825–2830, 2011.
- A. J. Summerville, S. Philip, and M. Mateas, “MCMCTS PCG 4 SMB: Monte Carlo tree search to guide platformer level generation,” in Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2015.
- G. R. Bentley and J. C. Osborn, “The videogame affordances corpus,” in EXAG Workshop, 2019.
- S. M. Lucas and V. Volz, “Tile pattern KL-divergence for analysing and evolving game levels,” in Genetic and Evolutionary Computation Conference, 2019.
- C. Biemer, A. Hervella, and S. Cooper, “Gram-elites: N-gram based quality-diversity search,” in 16th International Conference on the Foundations of Digital Games, 2021.
- M. Ben-Yosef and D. Weinshall, “Gaussian mixture generative adversarial networks for diverse datasets, and the unsupervised clustering of images,” arXiv preprint arXiv:1808.10356, 2018.
- A. Sarkar and S. Cooper, “tile2tile: learning game filters for platformer style transfer,” in Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2022.
- P. Bontrager, A. Roy, J. Togelius, N. Memon, and A. Ross, “Deepmasterprints: Generating masterprints for dictionary attacks via latent variable evolution,” in 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2018.
- D. Gravina, A. Khalifa, A. Liapis, J. Togelius, and G. N. Yannakakis, “Procedural content generation through quality diversity,” in IEEE Conference on Games (CoG), 2019.
- S. Cooper and A. Sarkar, “Pathfinding agents for platformer level repair,” in EXAG Workshop, 2020.