Role clarity on SWE (dev total)
90.79Total Role Clarity ScoreChatDev
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ChatDevCEO=FT, CPO=FT, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 90.79 | 0.3751 | |
| ChatDevCEO=FT, CPO=FT, Backbone=Llama, Role clarity regularization=with2026.04 | 85.3 | 0.3523 | |
| ChatDevCEO=FT, CPO=Base, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 65.11 | 0.1183 | |
| ChatDevCEO=Base, CPO=FT, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 60.29 | 0.0701 | |
| ChatDevCEO=FT, CPO=Base, Backbone=Llama, Role clarity regularization=with2026.04 | 57.35 | 0.0728 | |
| ChatDevCEO=Base, CPO=FT, Backbone=Llama, Role clarity regularization=with2026.04 | 56.45 | 0.0638 | |
| ChatDevCEO=Base, CPO=Base, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 53.28 | — | |
| ChatDevCEO=Base, CPO=Base, Backbone=Llama, Role clarity regularization=with2026.04 | 50.07 | — |