On the risk of coding before testing: An empirical study on LLM-based test generation workflow

Abstract: LLMs are increasingly used in software engineering workflows to generate both source code and test suites. This dual capability has enabled emerging development paradigms, including test-first and agentic workflows, where a single model is producing and validating implementations. However, these approaches assume that generated tests act as independent and reliable oracles - a fundamental requirement for effective software testing. In this paper, we challenge this assumption and investigate whether LLM-generated code biases the generation of subsequent tests. We introduce and empirically study the phenomenon of error propagation, where faults in generated code are systematically replicated in associated test artifacts. This leads to cases where incorrect implementations and tests are mutually consistent, masking defects rather than revealing them. We evaluate this effect across a range of programming tasks and agentic workflows, analyzing the consistency between generated code and test assertions, with particular focus on scenarios of aligned failures. Our study examines (i) whether erroneous code artifacts bias test generation, (ii) whether such bias persists under different prompting strategies, including chain-of-thought reasoning, and (iii) how errors propagate across multi-step workflows in which intermediate outputs are reused as context. The results show that error propagation is prevalent and impactful: generating tests after faulty code significantly reduces fault detection effectiveness compared to generating tests independently (14% vs. 25%). These findings highlight a fundamental limitation of current workflows, where lack of independence between generated artifacts undermines the reliability of automated testing. Furthermore, our results expose a previously underexplored threat to validity in empirical studies relying on coupled generation pipelines.

• The paper shows that specification-only test generation detects about 25% of faults, compared with roughly 14% when the model sees the implementation, across five models and three benchmarks.
• Implementation exposure reduces fault detection by 13.2% on average, while chain-of-thought, chain-of-verification, and summarization strategies fail to mitigate this error-propagation effect.
• The study finds that fresh-session, test-driven workflows outperform code-then-test agentic workflows by 11.7%, supporting independent test generation to avoid tests that validate shared misunderstandings.
• What mechanisms cause LLM-generated assertions to inherit implementation errors rather than follow the task specification?
• Can using different models for implementation and test generation restore the independence needed for reliable fault detection?
• How do error propagation and test quality change in longer, multi-step software engineering agent workflows?
• What evaluation methods can directly measure oracle correctness in LLM-generated test suites?
• Find recent papers about independent test generation for LLM-based software engineering.

Overview

This paper presents an empirical study of a structural weakness in LLM-based software engineering workflows: when the same model generates both an implementation and the tests intended to validate it, faults in the implementation systematically propagate into the test artifacts, producing tests that are mutually consistent with the faulty code rather than fault-revealing. The authors term this phenomenon error propagation and argue that it undermines the independence between system under test and oracle that effective testing requires (2607.05139). The study is motivated by the growing prevalence of agentic, "vibe-coding" style workflows in which intermediate outputs—including erroneous ones—are reused as context for subsequent generation steps.

The central empirical claim is stark: generating tests from the task description alone detects substantially more faults than any workflow that exposes the implementation to the model, whether explicitly (code in the prompt) or implicitly (shared conversational history). Across five models and three benchmarks, the headline result is a fault detection rate of 25% for a test-driven workflow versus approximately 14% when tests follow code generation.

Methodology

The experimental design proceeds in three stages. First, the authors generate candidate implementations for each programming task using temperature 0.8 with ten samples per task, retaining only implementations that fail at least one reference test while excluding those failing due to runtime errors. Second, they filter these faulty implementations by difficulty: implementations failing more than 50% of reference tests are discarded as trivially detectable, and per task they keep the single hardest fault (fewest failing reference tests), scored via a detectability metric $D(i) = 1 - |F_i|/|T|$. Third, they apply various test generation workflows to each retained faulty implementation and measure fault detection—defined as a test failing on the faulty implementation while passing on the reference—with fault triggering as a weaker precondition that isolates wrongly expressed assertions.

Three benchmarks are used: HumanEval+ (164 Python tasks), MBPP (974 tasks), and BigCodeBench (1,140 tasks spanning 139 libraries). Five models are evaluated: GPT-5-mini, GPT-4.1-mini, DeepSeek-V4-Flash, Claude Haiku 4.5, and Llama 3.3 Instruct (70B), covering reasoning-enabled and non-reasoning models as well as open-weight and commercial systems. The base workflow is LLM-Plain, a lightweight iterative refinement loop that repairs only compilation errors; this choice deliberately minimizes confounding factors so that differences can be attributed to the input configuration rather than tooling sophistication. Statistical significance is assessed with Mann-Whitney U tests at $p < 0.05$.

Implementation exposure degrades fault detection (RQ1)

The first research question compares three prompt configurations: task description only (P), description plus implementation (P+C), and implementation only (C). The results contradict the common practice of feeding the code under test to the model. Providing both sources reduces fault detection by 13.2% on average relative to the description alone, and providing only the implementation reduces it by 15.1%. Per-model reductions for P+C range from 9.1% (Claude Haiku 4.5) to 18.2% (GPT-4.1-mini). All P vs. P+C and P vs. C comparisons are statistically significant across all five models.

A notable secondary finding concerns how the model allocates attention once code is present: the difference between C and P+C is small (1.9% on average) and not statistically significant for any model, whereas P alone outperforms P+C by 13.2%. This indicates that once the implementation enters the context, the model anchors on it and largely ignores the specification—the mechanism through which implementation faults bias the generated oracles.

Prompting strategies do not mitigate the effect (RQ2)

The second question asks whether established prompting techniques—Test via Summarization (as in ChatAssert), Chain-of-Thought, and Chain-of-Verification—counteract the bias induced by code exposure. They do not. Generating tests directly from the task description outperforms summarization-based generation by 15.5% on average, and CoT and CoVe by 13.4% each, with all pairwise comparisons statistically significant for all models. Per-model gaps reach 25.8% for CoT on GPT-4.1-mini.

This is one of the paper's stronger claims: structured reasoning techniques, widely adopted to improve LLM reliability, provide no protection against error propagation because they still condition generation on the faulty artifact. The implication for practitioners is that prompt engineering cannot substitute for workflow-level independence between specification and validation.

Agentic workflows amplify the risk (RQ3)

The third question simulates a realistic conversational session: the model generates an implementation, the full conversation history is preserved, and the same model is then asked for unit tests—a faithful reproduction of contemporary coding assistants. This "Agentic Workflow" is compared against a "Test-Driven Workflow" in which tests are generated in a fresh session containing only the task description. The Test-Driven Workflow detects 11.7% more faults on average, with per-model improvements of 7.9% (Llama 3.3) to 17.7% (GPT-4.1-mini), all statistically significant.

Two aspects of this result deserve emphasis. First, the effect persists regardless of reasoning capabilities: GPT-5-mini and DeepSeek V4 show nearly identical improvements (~14%) despite differing reasoning support, suggesting the phenomenon is architectural rather than model-specific. Second, because the comparison holds the faulty implementation fixed across both workflows, the deficit is attributable purely to contextual contamination from the earlier code-generation step—not to differences in the code being tested.

Diagnostic analysis: what does not explain the gap

To rule out alternative explanations, the authors examine three confounds. Test count: CoVe and CoT consistently produce larger suites than Prompt-only (e.g., CoVe yields nearly double the tests of Prompt-only for Claude Haiku 4.5), yet detect fewer faults, so volume does not explain effectiveness. Statement coverage: coverage is uniformly high (mostly above 95%) across configurations and shows no consistent relationship with fault detection. Fault triggering: triggering and detection rates are not correlated in the expected direction—Prompt-only often exhibits the lowest triggering rates but the highest detection rates, indicating that the decisive factor is assertion quality rather than execution breadth. The authors concede that they do not directly measure oracle correctness, leaving the precise causal channel—inherited misinterpretations encoded in assertions—as an inference supported indirectly by these diagnostics.

Robustness to under-specified specifications

A natural objection is that well-specified benchmark prompts may overstate the value of specification-driven testing. To address this, the authors repeat the MBPP experiments using three progressively under-specified prompt variants from prior work on prompt underspecification. Even with weakened descriptions, specification-only generation outperforms implementation-exposed generation across all models, with under-specified lexical, vague, and syntax-modified variants exceeding the P+C configuration by 14.0%, 11.6%, and 10.6% respectively. In some cases the degraded prompts even outperform the original descriptions, consistent with prior findings that verbose specifications can confuse models. This strengthens the paper's central conclusion considerably: the advantage of specification-driven testing survives substantial information loss in the specification.

Limitations and open questions

The authors identify several constraints on generalizability. The benchmarks consist of self-contained programming tasks that do not reflect industrial-scale codebases, evolving requirements, or domain-specific constraints. The study evaluates a fixed set of models and prompting strategies; alternative designs—test-first generation, iterative refinement with agent verification, or multi-model pipelines where tests come from a different model—are not evaluated and may behave differently. Non-determinism is mitigated by low temperatures and ten-fold sampling where higher temperatures are required, but stochastic variation remains a threat. The diagnostic analysis stops short of directly measuring oracle correctness, so the assertion-quality hypothesis remains unverified. Open questions include whether cross-model diversity (generating tests with a different model than the code) restores independence, and how error propagation behaves in longer multi-step agentic chains beyond the two-step code-then-test setting studied here.

Conclusion

This study provides systematic evidence that the ordering of code and test generation materially affects testing effectiveness in LLM-driven development. Exposing the implementation—explicitly or through shared conversational context—consistently and significantly reduces fault detection, and neither chain-of-thought reasoning nor chain-of-verification mitigates the effect. The practical consequence is that high pass rates and high coverage on LLM-generated tests do not certify conformance to the specification when the same model produced both artifacts; such tests may validate a shared misinterpretation rather than reveal defects. The results also flag a methodological hazard: empirical evaluations built on coupled generation pipelines may inherit this bias, distorting reported performance for both proposed techniques and baselines. The actionable recommendation is to preserve separation between specification, implementation, and validation—for example via specification-first or independently generated tests—when relying on LLM-generated artifacts for quality assurance.