Role Consistency on SWE easy subset dev (test)
10Overstepping Rate (<INFO>)ChatDev (Joint FT)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ChatDev (Joint FT)CEO agent training=FT, CPO agent training=FT, Backbone=Qwen, Role clarity regularization=true2026.04 | 10 | 10.4 | -33 | -55.4 | |
| ChatDev (Base CEO, FT CPO)CEO agent training=Base, CPO agent training=FT, Backbone=Qwen, Role clarity regularization=true2026.04 | 19.2 | 34.4 | -23.8 | -31.4 | |
| ChatDev (FT CEO, Base CPO)CEO agent training=FT, CPO agent training=Base, Backbone=Qwen, Role clarity regularization=true2026.04 | 37.2 | 37.6 | -5.8 | -28.2 | |
| ChatDev (Base-Base)CEO agent training=Base, CPO agent training=Base, Backbone=Qwen, Role clarity regularization=true2026.04 | 43 | 65.8 | — | — |