Software Engineering on SWE-Bench Pro 1.0 (test)
51.6Resolved RateClaude-Opus-4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude-Opus-4.5Size=?, Scaffold=SWE-Agent, Max turns=3002026.02 | 51.6 | |
| Claude-Sonnet-4.5Size=?, Scaffold=SWE-Agent, Max turns=3002026.02 | 50.5 | |
| Claude-Opus-4.5Size=?, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 50.2 | |
| Kimi-K2.5Size=1000A32, Scaffold=SWE-Agent, Max turns=3002026.02 | 47.3 | |
| DeepSeek-V3.2Size=671A37, Scaffold=SWE-Agent, Max turns=3002026.02 | 46 | |
| GLM-4.7Size=358A32, Scaffold=SWE-Agent, Max turns=3002026.02 | 45.1 | |
| Claude-Sonnet-4.5Size=?, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 43 | |
| Kimi-K2.5Size=1000A32, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 42.8 | |
| Qwen3-Coder-NextSize=80A3, Scaffold=SWE-Agent, Max turns=3002026.02 | 42.7 | |
| MiniMax-M2.1Size=230A10, Scaffold=SWE-Agent, Max turns=3002026.02 | 40.8 | |
| GLM-4.7Size=358A32, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 39.4 | |
| MiniMax-M2.1Size=230A10, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 39.1 | |
| Qwen3-Coder-NextSize=80A3, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 38.7 | |
| DeepSeek-V3.2Size=671A37, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 32.4 |