Software Engineering on SWE-Bench Multilingual 1.0 (test)
75.2Resolution RateClaude-Opus-4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude-Opus-4.5Size=?, Scaffold=OpenHands, Max turns=3002026.02 | 75.2 | |
| Claude-Opus-4.5Size=?, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 71.8 | |
| Claude-Opus-4.5Size=?, Scaffold=SWE-Agent, Max turns=3002026.02 | 71.7 | |
| MiniMax-M2.1Size=230A10, Scaffold=OpenHands, Max turns=3002026.02 | 67.5 | |
| Claude-Sonnet-4.5Size=?, Scaffold=SWE-Agent, Max turns=3002026.02 | 67.2 | |
| MiniMax-M2.1Size=230A10, Scaffold=SWE-Agent, Max turns=3002026.02 | 66.2 | |
| Claude-Sonnet-4.5Size=?, Scaffold=OpenHands, Max turns=3002026.02 | 66 | |
| Qwen3-Coder-NextSize=80A3, Scaffold=OpenHands, Max turns=3002026.02 | 64.3 | |
| GLM-4.7Size=358A32, Scaffold=SWE-Agent, Max turns=3002026.02 | 63.7 | |
| Kimi-K2.5Size=1000A32, Scaffold=SWE-Agent, Max turns=3002026.02 | 63.7 | |
| Qwen3-Coder-NextSize=80A3, Scaffold=SWE-Agent, Max turns=3002026.02 | 62.8 | |
| MiniMax-M2.1Size=230A10, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 62.5 | |
| DeepSeek-V3.2Size=671A37, Scaffold=SWE-Agent, Max turns=3002026.02 | 62.3 | |
| Kimi-K2.5Size=1000A32, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 62.3 | |
| DeepSeek-V3.2Size=671A37, Scaffold=OpenHands, Max turns=3002026.02 | 61.8 | |
| GLM-4.7Size=358A32, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 61.8 | |
| Claude-Sonnet-4.5Size=?, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 61.3 | |
| GLM-4.7Size=358A32, Scaffold=OpenHands, Max turns=3002026.02 | 60.8 | |
| Qwen3-Coder-NextSize=80A3, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 56.2 | |
| DeepSeek-V3.2Size=671A37, Scaffold=MiniSWE-Agent, Max turns=3002026.02 | 55.5 |