An Exploratory Case Study of LLM-Assisted Refactoring and Gameplay Feature Generation in an Endless Runner Game

Abstract: LLMs are increasingly used to support software development, but their practical usefulness in applied game-development settings remains underexplored, especially when generated code must be integrated into an existing game software system. This paper presents an exploratory empirical case study of GPT-4o in a custom Python/Pygame endless runner. The study examines six selected development tasks: three localized refactoring tasks and three tasks involving gameplay feature generation. The resulting implementations were evaluated using software metrics, unit tests, and manual gameplay assessments. In this case study, all three selected refactoring tasks were completed successfully in functional terms, whereas only one of the three selected gameplay feature generation tasks resulted in a correctly integrated feature. The findings suggest that, in this setting, GPT-4o handled localized transformations more reliably than tasks requiring new gameplay interactions across multiple existing systems. Given the exploratory single-case design, these results are best interpreted as indicative observations rather than as generalizable evidence of category-level model performance. Overall, the paper contributes a transparent case-based account of the opportunities and limitations of LLM-assisted refactoring and gameplay feature generation in an existing game software system.

• The paper shows that GPT-4o effectively completes localized refactoring tasks, improving maintainability and reducing code redundancy through iterative prompting.
• The paper reveals that LLM-assisted gameplay feature generation encounters integration challenges, with only pixel-perfect collision detection successful among evaluated features.
• The paper underscores that human oversight and rigorous playtesting are essential in game development to address semantic integration and runtime behavioral issues.

LLM-Assisted Refactoring and Gameplay Feature Generation in an Endless Runner: An Empirical Case Study

Background and Motivation

LLMs have increasingly been integrated into code-centric workflows, offering success in domains such as code generation, transformation, and developer assistance. Despite their adoption in general software engineering, empirical evidence regarding their efficacy in applied game development—particularly for tasks requiring integration with complex game architectures—is sparse. This paper addresses this gap through a focused empirical case study examining GPT-4o’s ability to aid both localized refactoring and gameplay feature generation in a custom Python/Pygame endless runner.

Game development presents a unique challenge because generated code must synchronize with gameplay logic, visual systems, event-driven programming, and interactive components. This interdependence means that code modifications—especially those affecting gameplay—demand evaluation not only through unit tests but also via playtesting to detect behavioral integration failures.

Methodology and Task Selection

The study utilized a well-defined experimental setup. The artifact was an author-developed endless runner: a 2D Pygame game featuring horizontal and vertical player movement, multiple enemy and obstacle types, power-up mechanics, and collision logic.

Figure 1: Example gameplay scene from the custom Python/Pygame endless runner used as the software artifact in this study.

Two distinct task classes were evaluated:

• Refactoring Tasks: Optimization of state management, elimination of redundant asset-loading, and unification of movement control.
• Gameplay Feature Generation Tasks: Pixel-perfect collision detection, shrink power-up, and car platform mechanic.

Each development task was executed in isolation, with the full codebase (31,947 tokens) accessible to GPT-4o, thereby avoiding context truncation or split-prompt artifacts. The experiments employed iterative prompting, leveraging test-guided, few-shot, and self-consistency strategies, and code generated by GPT-4o was validated through a combination of software metrics, unit tests (pytest), and manual gameplay assessment.

Results and Quantitative Evaluation

The study observed a pronounced dichotomy between task classes. All three refactoring tasks were successfully completed, exhibiting functional stability and often improved maintainability metrics. For example, the asset-loading refactoring led to a significant reduction in LOC and an increase in maintainability index, although complexity metrics rose modestly.

In contrast, only the pixel-perfect collision feature was integrated successfully among the gameplay feature additions. Both the shrink power-up and car platform mechanic failed despite passing unit tests—the shrink power-up proved incompatible due to scaling interactions, and the car platform mechanic exhibited flawed runtime behavior despite syntactic correctness.

Key numerical outcomes:

• Refactoring success: R1 (State management) required 3 iterations, R2 (Asset-loading) 7, R3 (Movement) 5; all passed unit tests and played as expected.
• Feature generation: F1 (Pixel-perfect collisions) was successful after 4 iterations, F2 (Shrink power-up) failed after 8, F3 (Car platform) failed after 6; failures stemmed from integration and runtime behavioral anomalies not detected by unit tests.

This illustrates that passing unit tests is insufficient for verifying gameplay feature correctness—manual playtesting remains vital in game development integration assessment.

Implications for Game Development and LLM-Assisted Coding

The results indicate that GPT-4o’s reliability is modulated by task scope and integration complexity. Localized refactoring tasks, which operate within well-defined code boundaries, are amenable to LLM assistance. For these tasks, LLM output can streamline structural improvements and reduce developer effort, provided iterative correction and validation are applied.

In contrast, gameplay feature generation demands cross-cutting changes spanning multiple subsystems. Integration failures often manifest only during interactive gameplay, where subtle semantic mismatches arise from generated code not fully understanding complex state and timing interactions. Therefore, human supervision, thorough playtesting, and iterative integration remain mandatory for safely adopting LLM-generated gameplay code.

These findings reinforce the necessity for LLM-assisted workflows to include both code-level and gameplay-level validation, especially in domains where functional correctness is dependent on real-time interactive behavior rather than isolated logic.

Limitations and Future Directions

Several methodological constraints limit the study’s generalizability. The experiments were conducted on a single game project with three tasks per class and only one LLM snapshot (GPT-4o). Thus, conclusions cannot be extended to other genres, codebases, or model variants without further empirical validation. Nondeterminism, run-to-run differences, and broader task variety remain unexplored.

Future work should address:

• Evaluating additional LLM models, specialized code LLMs, and agentic coding tools for broader applicability.
• Expanding task sets and project contexts, including calibrating task difficulty.
• Deeper exploration of automated validation protocols combining unit tests and playtesting frameworks for interactive systems.

Conclusion

This empirical study demonstrates that GPT-4o can reliably assist with localized refactoring in an existing game system, but struggles with integration-intensive gameplay feature generation. The challenges observed underscore the nontrivial nature of semantic integration in interactive domains and highlight the continued necessity for human oversight. LLMs offer practical benefits when bounded to structurally clear, well-defined tasks, but are insufficient for immediate production-ready gameplay functionality without rigorous evaluation and playtesting. Broader adoption in game development will require advances in integration-aware prompting, validation, and higher-fidelity modeling of interactive systems.