TestWeaver: Execution-aware, Feedback-driven Regression Testing Generation with Large Language Models

Abstract: Regression testing ensures that code changes do not unintentionally break existing functionality. While recent advances in LLMs have shown promise in automating test generation for regression testing, they often suffer from limited reasoning about program execution, resulting in stagnated coverage growth - a phenomenon known as the coverage plateau. In this paper, we present TestWeaver, a novel LLM-based approach that integrates lightweight program analysis to guide test generation more effectively. TestWeaver introduces three key innovations: (1) it reduces hallucinations and improves focus by supplying the LLM with the backward slice from the target line instead of full program context; (2) it identifies and incorporates close test cases - those that share control-flow similarities with the path to the target line - to provide execution context within the LLM's context window; and (3) it enhances LLM's reasoning with execution in-line annotations that encode variable states as comments along executed paths. By equipping LLMs with these targeted and contextualized inputs, TestWeaver improves coverage-guided test generation and mitigates redundant explorations. Empirical results demonstrate that TestWeaver accelerates code coverage growth and generates more effective regression test cases than existing LLM-based approaches.