ChatDev: Multi-Agent Software Framework

• ChatDev is a multi-agent software engineering framework leveraging role-specialized LLM agents to autonomously perform all phases of the SDLC.
• It operationalizes a virtual software company using iterative instructor–assistant chat chains for design, coding, testing, and documentation.
• The framework enhances efficiency and safety through role clarity, experiential co-learning, and resource-optimized communication protocols.

ChatDev is a multi-agent software engineering framework in which specialized LLM agents communicate exclusively via natural language to autonomously design, implement, test, and document software. Structurally, ChatDev operationalizes a virtual software “company” employing a workflow inspired by the waterfall model, but realized entirely as a sequence of instructor–assistant chat chains spanning all phases of the SDLC (Software Development Life Cycle). Unlike monolithic LLM approaches or phasewise script-wrapping, ChatDev achieves end-to-end generation, iterative refinement, and error correction through granular dialogue among role-specialized agents. Since its introduction, ChatDev has served as a canonical reference in LLM-based multi-agent research, anchoring studies in autonomy, efficiency, safety, and collaboration (Qian et al., 2023).

1. System Architecture and Semantic Workflow

ChatDev instantiates a set of distinct LLM agents mapped to canonical software roles—CEO, CPO, CTO, programmer, reviewer, tester, designer—each configured with inception-style system prompts. The workflow is organized along four classical phases, each realized as a succession of two-party “atomic” chat sessions:

• Designing: The CEO, CPO, and CTO decompose a user requirement into architecture, technology choices, and module specifications through iterative NL dialogue.
• Coding: The CTO, programmer, and designer generate code skeletons, GUI artifacts, and concrete implementations.
• Testing: The programmer, reviewer, and tester conduct static code review and dynamic test runs, exchanging code and interpreter outputs.
• Documenting: Agents collaborate on requirements, dependency files, and user manuals.

A session is formalized as a “chat chain,” a strictly ordered sequence of paired messages, with each subtask’s progress tracked by an evolving memory stream:

$$\mathcal{M}_t = \langle (\mathcal{I}_1,\mathcal{A}_1), \ldots, (\mathcal{I}_t,\mathcal{A}_t) \rangle$$

Task-specific termination tokens and summarization (self-reflection) steps enforce convergence within bounded rounds. The entire pipeline runs autonomously, producing, on average, multicomponent software systems (code, GUI, docs) in under 7 minutes and at API costs below $1 (Qian et al., 2023).

2. Agent Specialization, Role Consistency, and Failure Modes

Role fidelity in ChatDev is achieved by distributing high-level and subtask-level responsibilities to each agent through role-conditioned prompts, ensuring that, for example, testers do not design features and CEOs do not write code. Despite these constraints, empirical studies have documented frequent “role overstepping” and behavioral drift, leading to inefficiencies and errors.

Recent advances formalize role clarity through a role assignment matrix:

$S(\phi)=[s_{ij}(\phi)] \quad s_{ij} = \cos(\mathbf{b}_i, \mathbf{r}_j)$</p> <p>where$\mathbf{b}_i$encodes the$i$-th agent’s behavior trajectory and$\mathbf{r}_j$$its written role description. The <strong>role clarity matrix</strong>$$M(\phi) = \text{softmax}(S(\phi)) - I$quantifies alignment; the Frobenius norm$\|M(\phi)\|_F$and derived score$C(\phi)$$make role adherence differentiable. Fine-tuning with a role clarity regularizer has reduced overstepping rates from up to 46.4<h2 class='paper-heading' id='co-learning-experiential-reuse-and-resource-optimization'>3. Co-Learning, Experiential Reuse, and Resource Optimization</h2> <p>To address ChatDev’s resource inefficiency—multiple redundant loops, unbounded review rounds, and context sprawl—recent extensions harvest “experiential trajectories” across completed sessions. Each interaction is modeled as a graph$$G = (N, E)$$of code states and instructional <a href="https://www.emergentmind.com/topics/tool-relationships-edges" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">edges</a>. By mining historical transitions, shortcut pseudo-instructions connecting non-adjacent high-reward states$$(r_i \Rightarrow r_j)$$are stored in key–value memories for both instructor and assistant roles.</p> <p>In co-learning execution, agents retrieve$$S(\phi)=[s_{ij}(\phi)] \quad s_{ij} = \cos(\mathbf{b}_i, \mathbf{r}_j)$0NN past transitions as context, enabling them to bypass trial-and-error subroutines. Integrating shortcut-based recall accelerates task completion, triples autonomy (from 0.3340 to 0.7100 in $S(\phi)=[s_{ij}(\phi)] \quad s_{ij} = \cos(\mathbf{b}_i, \mathbf{r}_j)$1 product), and reduces both wall-clock time and token consumption substantially (Qian et al., 2023).

Resource-aware frameworks such as Co-Saving further abstract these ideas by injecting trajectory “shortcuts” under explicit token/time budgets, decreasing token usage by 50.85% and boosting overall code quality by 10.06% over vanilla ChatDev, while maintaining budgeted completion rates above 70% (Qiu et al., 28 May 2025).

4. Communication Protocols and Heterogeneous Integration

ChatDev’s original implementation presumes homogeneous LLM backends and a unified orchestration runtime. The development of tap, a file-based, audit-friendly protocol, demonstrated the feasibility of supporting heterogeneous agent pools (e.g., OpenAI Codex, Anthropic Claude), each operating in isolated git worktrees and communicating via markdown+YAML message files. Tap decouples “at-least-once” delivery guarantees (disk-mediated) from push-based cross-runtime notification (webhooks, MCP), providing fault tolerance and native auditability. Heterogeneous agent review, coordinated via tap in a 27-day real-world experiment, achieved higher defect detection rates (69.8% for heterogeneous agent pairs vs. 53.1% for homogeneous), with operational artifacts providing persistent external memory (Kim, 12 Jun 2026).

5. Tokenomics and Overheads in Agentic Software Engineering

Systematic tracing of token flow in ChatDev reveals an extremely uneven distribution across SDLC phases:

Stage	% Total Tokens	Input %	Output %	Reasoning %
Code Review	59.4	51.4	24.7	23.9
Code Completion	26.8	47.7	41.7	10.5
Documentation	20.1	80.2	8.3	11.5
Testing	10.3	60.8	20.7	18.4
Coding	8.6	6.9	58.0	35.1
Design	2.4	60.4	3.6	36.0

Over half of all tokens are spent on input context, with review and documentation accounting for >50% input tokens—a clear indicator of inefficiency. The iterative, full-context review protocol is the dominant source of environmental cost and financial overhead (Salim et al., 20 Jan 2026).

Optimization strategies include context compression, delta-based codepass, and agent specialization; phase-level mapping and detailed token accounting aim to generate a generalized “cost map” for predictive resource planning.

6. Security and Attack Surface

Multi-agent LLM systems such as ChatDev present new, amplifying security risks relative to single-agent paradigms. Two adversarial models have been studied:

1. Malicious User, Benign Agents (MU-BA): Hidden prompt segments ($$S(\phi)=[s_{ij}(\phi)] \quad s_{ij} = \cos(\mathbf{b}_i, \mathbf{r}_j)$$2) embedded in the user request propagate undetected through agent rounds, producing malware code. Attack success rates (ASR) in ChatDev reach 93% in this threat model.
2. Benign User, Malicious Agents (BU-MA): One or more agents are prompt-tampered, injecting malicious logic into outputs even for honest user requests; ChatDev's ASR is 71% here.

Attacks exploiting review and testing-phase agents have the highest impact. Defensive interventions—such as embedding robust adversarial prompts, static analysis, and execution sandboxing—reduce ASR by more than 70% in MU-BA and by 45% in BU-MA, especially when targeted at critical (testing-phase) agents (Wang et al., 23 Nov 2025).

Additional studies (Evil Geniuses framework) highlight the cascading vulnerability of multi-agent role-passing: once one agent is jailbroken, harmful outputs propagate downstream, evading conventional LLM-level safeguards. System-level prompt filtering, inter-agent auditing, and cross-modal content review (code, doc, images) are essential mitigation layers (Tian et al., 2023).

7. Extensions, Benchmarks, and Future Directions

ChatDev stimulates benchmarking and cross-framework analysis in agentic SE:

• Comparative Agents: MetaGPT and AgileCoder adapt alternative role structures (agile/scrum, sprint-based sprints, dynamic code graphs), with AgileCoder yielding significant gains (pass@1 +7.7%, executability +25 points).
• Resource-Efficient Orchestration: Co-Saving and cross-team orchestration frameworks generalize memory, shortcutting, and collaborative solution merging, yielding higher code quality and completability (Du et al., 2024).
• Empirical Datasets: The DevGPT corpus provides a record-oriented dataset of 29,778 developer–ChatGPT interactions, supporting downstream analysis of real-world adoption, code reuse, conversational taxonomies, and LLM agent assessment (Xiao et al., 2023).
• Personalization and Context: User studies underpin design recommendations around developer-tailored assistance, context/tracking, and memory management, supporting both novice and expert developer flows (Melo et al., 13 May 2025).

Limitations remain in scaling to complex, non-linear workflows, modeling detailed non-technical aspects, and fully automating verification. Open questions include optimizing cross-modal context management, extending to new agentic paradigms (Kanban/Lean), and closing the loop on real-world deployment with robust, adaptive safety and audit layers.

ChatDev thus constitutes both a practical and a research-standard paradigm for LLM-driven multi-agent software development, with ongoing innovation in autonomy, efficiency, auditability, and safety (Qian et al., 2023, Salim et al., 20 Jan 2026, Qian et al., 2023, Zhou et al., 3 Apr 2026, Kim, 12 Jun 2026, Wang et al., 23 Nov 2025, Tian et al., 2023, Nguyen et al., 2024, Qiu et al., 28 May 2025, Du et al., 2024, Melo et al., 13 May 2025, Xiao et al., 2023).