---
title: 'ChatDev: Multi-Agent Software Framework'
url: https://www.emergentmind.com/topics/chatdev
type: topic
---

# ChatDev: Multi-Agent Software Framework

ChatDev is a multi-agent software engineering framework in which specialized large language model (LLM) agents communicate exclusively via natural language to autonomously design, implement, test, and document software. Structurally, ChatDev operationalizes a virtual software “company” employing a workflow inspired by the waterfall model, but realized entirely as a sequence of instructor–assistant chat chains spanning all phases of the SDLC (Software Development Life Cycle). Unlike monolithic LLM approaches or phasewise script-wrapping, ChatDev achieves end-to-end generation, iterative refinement, and error correction through granular dialogue among role-specialized agents. Since its introduction, ChatDev has served as a canonical reference in LLM-based multi-agent research, anchoring studies in autonomy, efficiency, safety, and collaboration [2307.07924].

## 1. System Architecture and Semantic Workflow

ChatDev instantiates a set of distinct LLM agents mapped to canonical software roles—CEO, CPO, CTO, programmer, reviewer, tester, designer—each configured with inception-style system prompts. The workflow is organized along four classical phases, each realized as a succession of two-party “atomic” chat sessions:

- **Designing**: The CEO, CPO, and CTO decompose a user requirement into architecture, technology choices, and module specifications through iterative NL dialogue.
- **Coding**: The CTO, programmer, and designer generate code skeletons, GUI artifacts, and concrete implementations.
- **Testing**: The programmer, reviewer, and tester conduct static code review and dynamic test runs, exchanging code and interpreter outputs.
- **Documenting**: Agents collaborate on requirements, dependency files, and user manuals.

A session is formalized as a “chat chain,” a strictly ordered sequence of paired messages, with each subtask’s progress tracked by an evolving memory stream:

\[
\mathcal{M}_t = \langle (\mathcal{I}_1,\mathcal{A}_1), \ldots, (\mathcal{I}_t,\mathcal{A}_t) \rangle
\]

Task-specific termination tokens and summarization (self-reflection) steps enforce convergence within bounded rounds. The entire pipeline runs autonomously, producing, on average, multicomponent software systems (code, GUI, docs) in under 7 minutes and at API costs below $1 [2307.07924].

## 2. Agent Specialization, Role Consistency, and Failure Modes

Role fidelity in ChatDev is achieved by distributing high-level and subtask-level responsibilities to each agent through role-conditioned prompts, ensuring that, for example, testers do not design features and CEOs do not write code. Despite these constraints, empirical studies have documented frequent “role overstepping” and behavioral drift, leading to inefficiencies and errors.

Recent advances formalize **role clarity** through a role assignment matrix:

\[
S(\phi)=[s_{ij}(\phi)] \quad s_{ij} = \cos(\mathbf{b}_i, \mathbf{r}_j)
\]

where $\mathbf{b}_i$ encodes the $i$-th agent’s behavior trajectory and $\mathbf{r}_j$ its written role description. The **role clarity matrix** $M(\phi) = \text{softmax}(S(\phi)) - I$ quantifies alignment; the Frobenius norm $\|M(\phi)\|_F$ and derived score $C(\phi)$ make role adherence differentiable. Fine-tuning with a role clarity regularizer has reduced overstepping rates from up to 46.4% to under 1% and lifted end-to-end task quality by several points [2604.02770].

## 3. Co-Learning, Experiential Reuse, and Resource Optimization

To address ChatDev’s resource inefficiency—multiple redundant loops, unbounded review rounds, and context sprawl—recent extensions harvest “experiential trajectories” across completed sessions. Each interaction is modeled as a graph $G = (N, E)$ of code states and instructional edges. By mining historical transitions, shortcut pseudo-instructions connecting non-adjacent high-reward states $(r_i \Rightarrow r_j)$ are stored in key–value memories for both instructor and assistant roles.

In co-learning execution, agents retrieve $k$NN past transitions as context, enabling them to bypass trial-and-error subroutines. Integrating shortcut-based recall accelerates task completion, triples autonomy (from 0.3340 to 0.7100 in $\alpha \beta \gamma$ product), and reduces both wall-clock time and token consumption substantially [2312.17025].

Resource-aware frameworks such as Co-Saving further abstract these ideas by injecting trajectory “shortcuts” under explicit token/time budgets, decreasing token usage by 50.85% and boosting overall code quality by 10.06% over vanilla ChatDev, while maintaining budgeted completion rates above 70% [2505.21898].

## 4. Communication Protocols and Heterogeneous Integration

ChatDev’s original implementation presumes homogeneous LLM backends and a unified orchestration runtime. The development of tap, a file-based, audit-friendly protocol, demonstrated the feasibility of supporting heterogeneous agent pools (e.g., OpenAI Codex, Anthropic Claude), each operating in isolated git worktrees and communicating via markdown+YAML message files. Tap decouples “at-least-once” delivery guarantees (disk-mediated) from push-based cross-runtime notification (webhooks, MCP), providing fault tolerance and native auditability. Heterogeneous agent review, coordinated via tap in a 27-day real-world experiment, achieved higher defect detection rates (69.8% for heterogeneous agent pairs vs. 53.1% for homogeneous), with operational artifacts providing persistent external memory [2606.14445].

## 5. Tokenomics and Overheads in Agentic Software Engineering

Systematic tracing of token flow in ChatDev reveals an extremely uneven distribution across SDLC phases:

| Stage           | % Total Tokens | Input % | Output % | Reasoning % |
|-----------------|---------------|---------|----------|-------------|
| Code Review     | 59.4          | 51.4    | 24.7     | 23.9        |
| Code Completion | 26.8          | 47.7    | 41.7     | 10.5        |
| Documentation   | 20.1          | 80.2    | 8.3      | 11.5        |
| Testing         | 10.3          | 60.8    | 20.7     | 18.4        |
| Coding          | 8.6           | 6.9     | 58.0     | 35.1        |
| Design          | 2.4           | 60.4    | 3.6      | 36.0        |

Over half of all tokens are spent on **input context**, with review and documentation accounting for >50% input tokens—a clear indicator of inefficiency. The iterative, full-context review protocol is the dominant source of environmental cost and financial overhead [2601.14470].

Optimization strategies include context compression, delta-based codepass, and agent specialization; phase-level mapping and detailed token accounting aim to generate a generalized “cost map” for predictive resource planning.

## 6. Security and Attack Surface

Multi-agent LLM systems such as ChatDev present new, amplifying security risks relative to single-agent paradigms. Two adversarial models have been studied:

1. **Malicious User, Benign Agents (MU-BA)**: Hidden prompt segments ($P_m$) embedded in the user request propagate undetected through agent rounds, producing malware code. Attack success rates (ASR) in ChatDev reach 93% in this threat model.
2. **Benign User, Malicious Agents (BU-MA)**: One or more agents are prompt-tampered, injecting malicious logic into outputs even for honest user requests; ChatDev's ASR is 71% here.

Attacks exploiting review and testing-phase agents have the highest impact. Defensive interventions—such as embedding robust adversarial prompts, static analysis, and execution sandboxing—reduce ASR by more than 70% in MU-BA and by 45% in BU-MA, especially when targeted at critical (testing-phase) agents [2511.18467].

Additional studies (Evil Geniuses framework) highlight the cascading vulnerability of multi-agent role-passing: once one agent is jailbroken, harmful outputs propagate downstream, evading conventional LLM-level safeguards. System-level prompt filtering, inter-agent auditing, and cross-modal content review (code, doc, images) are essential mitigation layers [2311.11855].

## 7. Extensions, Benchmarks, and Future Directions

ChatDev stimulates benchmarking and cross-framework analysis in agentic SE:

- **Comparative Agents**: MetaGPT and AgileCoder adapt alternative role structures (agile/scrum, sprint-based sprints, dynamic code graphs), with AgileCoder yielding significant gains (pass@1 +7.7%, executability +25 points).
- **Resource-Efficient Orchestration**: Co-Saving and cross-team orchestration frameworks generalize memory, shortcutting, and collaborative solution merging, yielding higher code quality and completability [2406.08979].
- **Empirical Datasets**: The DevGPT corpus provides a record-oriented dataset of 29,778 developer–ChatGPT interactions, supporting downstream analysis of real-world adoption, code reuse, conversational taxonomies, and LLM agent assessment [2309.03914].
- **Personalization and Context**: User studies underpin design recommendations around developer-tailored assistance, context/tracking, and memory management, supporting both novice and expert developer flows [2505.08648].

Limitations remain in scaling to complex, non-linear workflows, modeling detailed non-technical aspects, and fully automating verification. Open questions include optimizing cross-modal context management, extending to new agentic paradigms (Kanban/Lean), and closing the loop on real-world deployment with robust, adaptive safety and audit layers.

ChatDev thus constitutes both a practical and a research-standard paradigm for LLM-driven multi-agent software development, with ongoing innovation in autonomy, efficiency, auditability, and safety [2307.07924][2601.14470][2312.17025][2604.02770][2606.14445][2511.18467][2311.11855][2406.11912][2505.21898][2406.08979][2505.08648][2309.03914].

Source: https://www.emergentmind.com/topics/chatdev