Role Consistency on SWE dev full set (test)
8.4Total Overstepping Rate (<INFO>)ChatDev (Joint FT)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ChatDev (Joint FT)CEO agent training=FT, CPO agent training=FT, Backbone=Qwen, Role clarity regularization=true2026.04 | 8.4 | 9 | -38 | -58 | |
| ChatDev (Base CEO, FT CPO)CEO agent training=Base, CPO agent training=FT, Backbone=Qwen, Role clarity regularization=true2026.04 | 18 | 32.4 | -28.4 | -34.6 | |
| ChatDev (FT CEO, Base CPO)CEO agent training=FT, CPO agent training=Base, Backbone=Qwen, Role clarity regularization=true2026.04 | 40.6 | 40.8 | -5.8 | -26.2 | |
| ChatDev (Base-Base)CEO agent training=Base, CPO agent training=Base, Backbone=Qwen, Role clarity regularization=true2026.04 | 46.4 | 67 | — | — |