File-level localization on SWE-Bench Verified
89.6Hit@1MiniMax-M2.5
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MiniMax-M2.5Framework=SWE-Agent, Method=Baseline2026.06 | 89.6 | 86.4 | 88.8 | 86.3 | 489 | |
| MiniMax-M2.5Framework=SWE-Agent, Method=Masked2026.06 | 88.6 | 84.8 | 88 | 85.1 | 461 | |
| Qwen3-235B-A22B-ThinkingFramework=SHERLOC, Method=SHERLOC2026.06 | 88.4 | 81.3 | 87.6 | 83.5 | 498 | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Method=Baseline2026.06 | 86.8 | 83.4 | 84.4 | 82.1 | 491 | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Method=Masked2026.06 | 86.2 | 83.3 | 83.9 | 81.8 | 488 | |
| MiniMax-M2.5Framework=OpenHands, Method=Masked2026.06 | 83.4 | 85.5 | 82.3 | 81.6 | 487 | |
| MiniMax-M2.5Framework=OpenHands, Method=Baseline2026.06 | 82.4 | 84.6 | 81.8 | 80.9 | 493 | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Method=Baseline2026.06 | 81.6 | 83.7 | 77.3 | 77.6 | 490 | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Method=Masked2026.06 | 81.2 | 83.7 | 78.5 | 78.5 | 490 | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Method=Masked2026.06 | 78.8 | 73.5 | 78.1 | 74.3 | 457 | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Method=Baseline2026.06 | 77.2 | 73 | 76.3 | 73.4 | 437 | |
| Qwen3-Next-80B-A3B-InstructFramework=SWE-Agent, Method=Masked2026.06 | 76.6 | 72.6 | 76.6 | 73.1 | 447 | |
| Qwen3-Next-80B-A3B-InstructFramework=SWE-Agent, Method=Baseline2026.06 | 76 | 72.1 | 76.1 | 72.8 | 441 | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Method=Baseline2026.06 | 75.2 | 75.9 | 74.8 | 73 | 461 | |
| Qwen3-Next-80B-A3B-InstructFramework=OpenHands, Method=Baseline2026.06 | 75.2 | 77.6 | 75.2 | 73.8 | 480 | |
| Qwen3-Next-80B-A3B-InstructFramework=OpenHands, Method=Masked2026.06 | 74.4 | 77.5 | 73.4 | 73.1 | 479 | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Method=Masked2026.06 | 73.4 | 73.8 | 72.5 | 70.8 | 456 | |
| Qwen3-Coder-NextFramework=SWE-Agent, Method=Baseline2026.06 | 56.2 | 55.3 | 55.1 | 54.2 | 305 | |
| Qwen3-Coder-NextFramework=OpenHands, Method=Baseline2026.06 | 56.2 | 58.4 | 54.3 | 54.7 | 357 | |
| Qwen3-Coder-NextFramework=OpenHands, Method=Masked2026.06 | 56.2 | 57.3 | 54 | 53.8 | 350 | |
| Qwen3-Coder-NextFramework=SWE-Agent, Method=Masked2026.06 | 49.4 | 48.3 | 48.5 | 47.4 | 271 |