Role Clarity on SWE hard (dev)
90.76Role Clarity ScoreChatDev
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ChatDevCEO=FT, CPO=FT, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 90.76 | 0.3781 | |
| ChatDevCEO=FT, CPO=FT, Backbone=Llama, Role clarity regularization=with2026.04 | 85.35 | 0.3523 | |
| ChatDevCEO=FT, CPO=Base, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 64.4 | 0.1145 | |
| ChatDevCEO=Base, CPO=FT, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 60.44 | 0.0749 | |
| ChatDevCEO=Base, CPO=FT, Backbone=Llama, Role clarity regularization=with2026.04 | 56.55 | 0.0643 | |
| ChatDevCEO=FT, CPO=Base, Backbone=Llama, Role clarity regularization=with2026.04 | 54.96 | 0.0484 | |
| ChatDevCEO=Base, CPO=Base, Backbone=Qwen, Role clarity regularization=Enabled2026.04 | 52.95 | — | |
| ChatDevCEO=Base, CPO=Base, Backbone=Llama, Role clarity regularization=with2026.04 | 50.12 | — |