Software Engineering Task Resolution on SWE-BENCH LIVE
24.7Resolution RateQwen3-Coder 480B/A35B Instruct
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-Coder 480B/A35B InstructScale=Large, Scaffold=OpenHands2025.12 | 24.7 | |
| Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent + BOAD2025.12 | 20 | |
| GLM-4.5-AirScale=Large, Scaffold=SWE-Agent2025.12 | 17.7 | |
| Qwen3-Coder-30B-A3B-InstructScale=Small, Scaffold=SWE-agent2025.12 | 17 | |
| Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent + Evolutionary Search2025.12 | 17 | |
| Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent + Manual Sub-agent2025.12 | 14 | |
| Claude 3.7 SonnetScale=Large, Scaffold=SWE-agent2025.12 | 13.7 | |
| DeepSeek-V3Scale=Large, Scaffold=Agentless2025.12 | 13.3 | |
| Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent (baseline)2025.12 | 12.3 | |
| GPT-4oScale=Large, Scaffold=Agentless2025.12 | 11.7 | |
| GPT-4oScale=Large, Scaffold=SWE-agent2025.12 | 10 |