Software Engineering on SWE-Bench-Verified (50 cases)
72AccuracySWE-agent + Claude 3.7 Sonnet w/ Review Heavy
Evaluation results
| Method | Links | |
|---|---|---|
| SWE-agent + Claude 3.7 Sonnet w/ Review HeavyOpen/Close Source=Open Source2025.05 | 72 | |
| CodeStory Midwit Agent + swe-searchOpen/Close Source=Close Source2025.05 | 70 | |
| Openhands_04_15Open/Close Source=Open Source2025.05 | 68 | |
| InfantAgent-Next + Claude-3.7-SonnetOpen/Close Source=Open Source2025.05 | 66 | |
| AgentScopeOpen/Close Source=Close Source2025.05 | 66 | |
| CORTEXAOpen/Close Source=Close Source2025.05 | 62 | |
| Amazon Q Developer Agent_2024_12_02Open/Close Source=Close Source2025.05 | 54 | |
| AutoCodeRover-v2.0 (Claude-3.5-Sonnet-20241022)Open/Close Source=Open Source2025.05 | 52 | |
| devlo_2024_11_08Open/Close Source=Close Source2025.05 | 48 | |
| SWE-agent + SWE-agent-LM-32BOpen/Close Source=Open Source2025.05 | 46 | |
| AppMap Navie v2Open/Close Source=Open Source2025.05 | 12 | |
| Agentless Lite + O3 Mini (20250214)Open/Close Source=Open Source2025.05 | 10 |