Role Consistency on SWE Dev hard (test)
6.8Overstepping Rate (<INFO>)ChatDev (Joint FT)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ChatDev (Joint FT)CEO agent training=FT, CPO agent training=FT, Backbone=Qwen, Role clarity regularization=true2026.04 | 6.8 | 7.6 | -42.9 | -60.6 | |
| ChatDev (Base CEO, FT CPO)CEO agent training=Base, CPO agent training=FT, Backbone=Qwen, Role clarity regularization=true2026.04 | 16.8 | 30.4 | -32.9 | -37.8 | |
| ChatDev (FT CEO, Base CPO)CEO agent training=FT, CPO agent training=Base, Backbone=Qwen, Role clarity regularization=true2026.04 | 44 | 44 | -5.7 | -24.2 | |
| ChatDev (Base-Base)CEO agent training=Base, CPO agent training=Base, Backbone=Qwen, Role clarity regularization=true2026.04 | 49.7 | 68.2 | — | — |