Strategic Test Selection for TDD-Guided LLM Code Generation

Determine a principled strategy for selecting a small, effective subset of test cases to guide large language models in repository-level code generation under the Test-Driven Development paradigm, subject to constraints of limited prompt context windows and computational budgets.

Background

The paper argues that Test-Driven Development (TDD) is particularly important for repository-level code generation with LLMs because tests serve as executable specifications that clarify intended behavior beyond natural language descriptions. However, real-world repositories often require validating complex functions against many diverse test cases, and providing all tests to an LLM is infeasible due to context window and efficiency constraints.

In this context, the authors explicitly state that how to strategically select a small yet effective subset of tests remains an open question. They later propose the Test Harness Mechanism (THM) as a concrete approach, but the foundational problem of devising principled, general strategies for test selection under TDD with limited resources is highlighted as a critical open issue.