Agentic Software Engineering on SWE-bench Multilingual (SWE-M)
50Success RateClaude-Opus-4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude-Opus-4.52026.06 | 50 | |
| Gemini-3-Pro2026.06 | 42.7 | |
| DeepSeek-V3.22026.06 | 37.4 | |
| Kimi-K2-Instruct-09052026.06 | 33.5 | |
| Qwen3-Coder-480B-A35B-Instruct2026.06 | 32.7 | |
| LoopCoder-v2R (number of refinement loops)=22026.06 | 31 | |
| LoopCoder-v2R (number of refinement loops)=12026.06 | 14 | |
| LoopCoder-v2R (number of refinement loops)=32026.06 | 11 | |
| LoopCoder-v2R (number of refinement loops)=42026.06 | 9.3 | |
| DeepSeek-Coder-V2-Lite-Instruct2026.06 | 0 | |
| Qwen2.5-Coder-7B-Instruct2026.06 | 0 | |
| Seed-Coder-8B-Instruct2026.06 | 0 | |
| Qwen2.5-Coder-14B-Instruct2026.06 | 0 | |
| Qwen2.5-Coder-32B-Instruct2026.06 | 0 |