List your company
ResearchFormerly WizWand

Software Engineering Task Resolution on SWE-BENCH LIVE

24.7Resolution RateQwen3-Coder 480B/A35B Instruct
9.41213.38117.3521.319Dec 29, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Qwen3-Coder 480B/A35B InstructScale=Large, Scaffold=OpenHands2025.1224.7
Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent + BOAD2025.1220
GLM-4.5-AirScale=Large, Scaffold=SWE-Agent2025.1217.7
Qwen3-Coder-30B-A3B-InstructScale=Small, Scaffold=SWE-agent2025.1217
Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent + Evolutionary Search2025.1217
Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent + Manual Sub-agent2025.1214
Claude 3.7 SonnetScale=Large, Scaffold=SWE-agent2025.1213.7
DeepSeek-V3Scale=Large, Scaffold=Agentless2025.1213.3
Seed-OSS-36B-InstructScale=Small, Scaffold=SWE-agent (baseline)2025.1212.3
GPT-4oScale=Large, Scaffold=Agentless2025.1211.7
GPT-4oScale=Large, Scaffold=SWE-agent2025.1210