ChatDev: Communicative Agents for Software Development
About
Software development is a complex task that necessitates cooperation among multiple members with diverse skills. Numerous studies used deep learning to improve specific phases in a waterfall model, such as design, coding, and testing. However, the deep learning model in each phase requires unique designs, leading to technical inconsistencies across various phases, which results in a fragmented and ineffective development process. In this paper, we introduce ChatDev, a chat-powered software development framework in which specialized agents driven by large language models (LLMs) are guided in what to communicate (via chat chain) and how to communicate (via communicative dehallucination). These agents actively contribute to the design, coding, and testing phases through unified language-based communication, with solutions derived from their multi-turn dialogues. We found their utilization of natural language is advantageous for system design, and communicating in programming language proves helpful in debugging. This paradigm demonstrates how linguistic communication facilitates multi-agent collaboration, establishing language as a unifying bridge for autonomous task-solving among LLM agents. The code and data are available at https://github.com/OpenBMB/ChatDev.
Key insights
What problem does ChatDev address?
Software development phases stay isolated with phase-specific models, causing inconsistency. ChatDev unifies design, coding, and testing using LLM agents guided by a chat chain and communicative dehallucination to reduce coding hallucinations.
How does it differ from prior work?
Unlike GPT-Engineer and MetaGPT, ChatDev uses cooperative multi-turn dialogue rather than single-step or static predefined instructions. On 1,200 tasks, ChatDev achieved 0.3953 quality versus MetaGPT's 0.1523.
What are the practical uses?
ChatDev suits prototype system generation from textual requirements. Agents autonomously produce executable software with features like GUIs. No real-world production deployment is documented in the paper.
What limitations do the authors note?
Agents overestimate capability, producing simple logic with vague requirements. Evaluation lacks factors like robustness and safety. Multi-agent approaches consume more tokens and time, increasing computational demands versus single-agent methods.