Coding on SWE-bench Lite (test)
25.83AccuracyMAS-ZERO
Evaluation results
| Method | Links | |
|---|---|---|
| MAS-ZEROLLM Backbone=GPT-4o2025.05 | 25.83 | |
| MAS-ZEROLLM Backbone=Llama3.32025.05 | 16.74 | |
| AFlowLLM Backbone=GPT-4o2025.05 | 16.25 | |
| DebateLLM Backbone=GPT-4o2025.05 | 12.5 | |
| Self-RefineLLM Backbone=GPT-4o2025.05 | 11.67 | |
| MaASLLM Backbone=GPT-4o2025.05 | 10 | |
| CoTLLM Backbone=GPT-4o2025.05 | 9.17 | |
| DebateLLM Backbone=Llama3.32025.05 | 6.67 | |
| AFlowLLM Backbone=Llama3.32025.05 | 6.67 | |
| MaASLLM Backbone=Llama3.32025.05 | 5 | |
| CoTLLM Backbone=Llama3.32025.05 | 2.92 | |
| Self-RefineLLM Backbone=Llama3.32025.05 | 1.67 |