Multi-Agent Collaboration Role Overstepping on SWE hard subset (dev)
0Overstepping Rate (<INFO>)ChatDev
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ChatDevCEO Configuration=FT, CPO Configuration=FT, Backbone=Llama, System=ChatDev, Role Clarity Regularization=enabled2026.04 | 0 | 0 | -41.6 | -42.8 | |
| ChatDevCEO Configuration=FT, CPO Configuration=Base, Backbone=Llama, System=ChatDev, Role Clarity Regularization=enabled2026.04 | 0.5 | 0.5 | -41.1 | -42.3 | |
| ChatDevCEO Configuration=Base, CPO Configuration=FT, Backbone=Llama, System=ChatDev, Role Clarity Regularization=enabled2026.04 | 14.4 | 15.6 | -27.2 | -27.2 | |
| ChatDevCEO Configuration=Base, CPO Configuration=Base, Backbone=Llama, System=ChatDev, Role Clarity Regularization=enabled2026.04 | 41.6 | 42.8 | — | — |