Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows

Abstract: Recently, the emergence of LLMs has spurred a surge of research into automated unit test generation, yielding impressive performance and reducing manual effort. However, existing LLM-based approaches still suffer from two major limitations: (1) they follow rigid, procedural workflows that underutilize the autonomous reasoning potential of LLMs, making it difficult to dynamically adapt testing strategies based on real-time feedback; and (2) they rely on rule-based context extraction that is not tailored to test generation, failing to capture fine-grained code dependencies and test-specific knowledge required for deriving test requirements. In this paper, we propose TestAgent, an LLM-based test generation approach that addresses the above limitations by emulating human testing practices via a multi-agent collaboration mechanism. Particularly, TestAgent designs three specialized agents, namely a requirement planner, a test generator, and a test reviewer, to simulate how developers understand, construct, and validate unit tests. To unleash the autonomous capabilities of LLMs, we equip TestAgent with a set of tool APIs that can be invoked dynamically in an on-demand and adaptive manner. To further support repository-level reasoning, TestAgent constructs a test-specialized knowledge graph via static analysis, which captures code entities and their dependencies across the project and persistently stores testing artifacts (e.g., test reports and failure analyses) produced during generation. Experimental results show that TestAgent achieves 97.46% execution rate, 92.34% line coverage, 90.24% branch coverage, and 83.69% mutation score on six Java projects, outperforming LLM-based baselines across all metrics and achieving substantially higher mutation scores than search-based tools.

• The paper introduces TestAgent, a multi-agent LLM system that emulates human-testing workflows to significantly improve unit test quality and fault detection.
• It details a modular architecture with requirement planner, test generator, and reviewer agents that operate over a specialized repository knowledge graph.
• Empirical evaluations on Java and Python projects demonstrate high execution rates, coverage metrics, and cost-efficiency, underscoring practical industrial applicability.

Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows

Introduction

The paper "Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows" (2607.09101) addresses two central limitations of existing LLM-based automatic unit test generation methodologies: (1) their inherently rigid, procedural, non-adaptive workflows; and (2) reliance on rule-based, task-agnostic context extraction, which typically fails to provide fine-grained, test-specific code dependencies essential for high-quality test suite construction. The authors introduce TestAgent, a multi-agent, LLM-driven system that emulates human developer processes for unit test planning, synthesis, and validation by decomposing the test generation workflow into semantically specialized agents—requirement planner, generator, and reviewer. This collaboration is orchestrated with a fine-grained, test-specialized repository knowledge graph and flexible tool API integration.

Background and Context

Traditional approaches to unit test automation, including search-based (SBST), random-based, and symbolic execution methods, optimize for structural coverage but frequently generate low-utility tests with poor readability and practical maintainability. With the advent of LLMs, prompt-based and fine-tuned neural methods have demonstrated improvements in test code quality and usability but continue to exhibit procedural rigidity and lack of adaptivity. Prior LLM-based agents in software engineering are limited by coarse code representation and single-agent pipelines, not capturing the iterative, feedback-rich human test writing loop.

TestAgent Architecture

TestAgent operationalizes a multi-agent collaborative workflow, illustrated as a close emulation of common developer test-writing processes.

Figure 1: A typical developer workflow for manual unit test writing, emphasizing iterative understanding, generation, and reviewing.

The system orchestrates three agent types: (1) a requirement planner that infers formalized test requirements via semantic analysis; (2) a test generator that iteratively synthesizes, refines, and validates executable tests through chain-of-thought reasoning and sandbox tool calls; and (3) a test reviewer responsible for critical coverage analysis and actionable feedback. All agents operate over a test-specialized knowledge graph that encodes static semantic relationships, code and test entities, and tracks artifact provenance and coverage states.

Figure 2: High-level schematic of TestAgent, showing agent composition, tool interfaces, and knowledge graph interactions.

The knowledge graph, produced via AST-based extraction and inter-procedural dependency analysis, encodes fine-grained relationships and acts as a persistent, queryable substrate across task iterations. The toolkit encompasses graph manipulation, context-aware retrieval, and runtime test validation APIs, enabling agents to adaptively acquire context, verify generated artifacts, and update state. Similarity-based test recommendation leverages relation-set Jaccard similarity for semantic-aligned retrieval, a capability empirically validated in its performance boost.

Experimental Analysis

Effectiveness and Comparative Performance

Empirical evaluation across six diverse Java projects decisively demonstrates TestAgent's superior performance on both functional and adequacy metrics. TestAgent attains an execution rate of 97.46%, line coverage of 92.34%, branch coverage of 90.24%, and mutation score of 83.69%, surpassing both LLM-based (ChatUniTest, HITS) and search-based (EvoSuite) baselines. Notably, while EvoSuite achieves competitive structural coverage, its mutation score (43.59%) lags substantially behind TestAgent, indicating significantly weaker fault-detection capacity. TestAgent's residual compilation failures are isolated to highly complex targets and treated as best-effort outcomes.

Ablation and Component Impact

Ablation studies indicate that removal of the knowledge graph degrades execution and coverage metrics most severely (execution rate -12.1pp, line coverage -22.3pp, mutation score -26.8pp), underscoring the essential contribution of fine-grained, persistent context representations. The test reviewer and similarity retrieval modules provide incremental but measurable improvements in feedback-guided refinement and context-driven reuse, respectively.

Model Generality and Cost-Effectiveness

TestAgent exhibits robust performance across different LLM backends (GPT-4o, DeepSeek-V3, and the open-source Qwen3-30B-A3B), maintaining the highest adequacy when deployed over GPT-4o but still outperforming baselines even with smaller or locally deployed models. DeepSeek-V3 achieves 91.4% of GPT-4o’s peak aggregate utility at only 5.35% of the monetary inference cost, illustrating model-agnostic extensibility and the practical tradeoff space between performance and deployment cost.

Figure 3: Pareto frontier of cost-utility for TestAgent and variants (utility: mean of line, branch, and mutation scores; cost: relative dollar and time budget).

Industrial Applicability and Human Factors

When deployed on the industrial, closed-source UTXXX dataset (658 methods), TestAgent achieves 84.2% line coverage and a 55.4% mutation score, outperforming LLM-based and heuristic baselines, even running only on Qwen3-30B-A3B. User studies verify that its generated tests are measurably more readable and actionable for developers (higher scores in naming, layout, assertion quality, and adoption effort)—a persistent weakness of SBST-derived methods such as EvoSuite.

Language Generalization

TestAgent’s core methodology is validated on Python projects, necessitating only AST/graph extraction adaptation. On nine Python modules, TestAgent exceeds both line and branch coverage of specialized baselines, with an average line coverage of 88.85% (+26.86pp relative to CoverUp).

Bug Detection and Failure Analysis

TestAgent distinguishes itself by its ability to generate and preserve non-regression, failure-inducing tests, confirmed through a manual audit: of 167 suspected faulty cases, 154 are true positives (92.2% precision), spanning input validation, boundary, and state inconsistency errors. Most detected faults fall under robustness (70.1%) and functional correctness bugs, enabling TestAgent-generated tests to directly enhance fault exposure in CI/CD pipelines.

Implications and Future Outlook

Practical implications include (1) test suites that are simultaneously high coverage, high mutation, and developer-aligned, (2) modular extensibility to multi-language software stacks; and (3) adaptive, cost-effective deployment driven by the LLM and toolkit mix most appropriate for project constraints. Theoretically, this work demonstrates that multi-agent LLM coordination, underpinned by fine-grained, persistent code representations and tool leverage, unlocks higher levels of adaptive automated software engineering—beyond what fixed-pipeline or unstructured-context approaches can achieve.

Sustained future directions include: hybridizing TestAgent with search-based approaches (e.g., EvoSuite as seed generation); extending agent specialization (e.g., advanced feedback interpreters, meta-agents for defect localization); domain languages beyond Python/Java; and industrial-scale integration with CI ecosystems.

Conclusion

TestAgent establishes a new operational paradigm for LLM-based unit test generation—one that fuses human-centric, feedback-driven workflows with multi-agent reasoning, fine-grained repository modeling, and extensible tool-chains. Empirically, it consistently outperforms prior SOTA on correctness, coverage, mutation, readability, and fault detection. The system’s architecture, algorithmic design, and empirical results collectively argue for the efficacy of human-testing-inspired, multi-agent LLM systems as a general approach to practical, scalable, and intelligent software quality assurance.