Software Engineering Issue Resolution on SWE-bench Verified
89.2Resolution RateMiniMax-M2.5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MiniMax-M2.5Framework=SWE-Agent, Localization Setting=Oracle GPT-5.22026.06 | 89.2 | — | — | |
| MiniMax-M2.5Framework=OpenHands, Localization Setting=Oracle GPT-5.22026.06 | 87.8 | — | — | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Localization Setting=Oracle GPT-5.22026.06 | 86.2 | — | — | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Localization Setting=Oracle GPT-5.22026.06 | 82.6 | — | — | |
| Simple Strands AgentBackbone=Opus 4.62026.06 | 80.8 | 82.54 | 80.8 | |
| Simple Strands AgentBackbone=Gemini 3.1 Pro2026.06 | 80.8 | 84.14 | 80.6 | |
| Simple Strands AgentBackbone=Opus 4.52026.06 | 80.48 | 83.3 | 80.9 | |
| Simple Strands AgentBackbone=GPT-5.42026.06 | 80.16 | 83.4 | — | |
| Simple Strands AgentBackbone=GPT-5.32026.06 | 79.52 | 82.2 | — | |
| Simple Strands AgentBackbone=GPT-5.22026.06 | 79.28 | 81.72 | 80 | |
| Simple Strands AgentBackbone=Sonnet 4.62026.06 | 79 | 82.06 | 79.6 | |
| Simple Strands AgentBackbone=Gemini 3 Flash2026.06 | 78.32 | 81.76 | 78 | |
| Simple Strands AgentBackbone=Grok 4.22026.06 | 77.56 | 80.76 | — | |
| Simple Strands AgentBackbone=GPT-5.2-codex2026.06 | 76.8 | 79.98 | — | |
| MiniMax-M2.5Framework=SWE-Agent, Localization Setting=QF SHERLOC2026.06 | 76.6 | — | — | |
| Simple Strands AgentBackbone=Sonnet 4.52026.06 | 76.4 | 79.94 | 77.2 | |
| Simple Strands AgentBackbone=Qwen3.6 27B2026.06 | 76.24 | 80.52 | 77.2 | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Localization Setting=Oracle GPT-5.22026.06 | 76 | — | — | |
| MiniMax-M2.5Framework=SWE-Agent, Localization Setting=Masked2026.06 | 74.5 | — | — | |
| MiniMax-M2.5Framework=SWE-Agent, Localization Setting=Baseline2026.06 | 74.4 | — | — | |
| Simple Strands AgentBackbone=Qwen3.5 27B2026.06 | 74.36 | 79.5 | 72.4 | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Localization Setting=Oracle GPT-5.22026.06 | 73.4 | — | — | |
| Simple Strands AgentBackbone=Qwen3.6 35B-A3B2026.06 | 72.92 | 77.76 | 73.4 | |
| MiniMax-M2.5Framework=OpenHands, Localization Setting=QF SHERLOC2026.06 | 72.8 | — | — | |
| Simple Strands AgentBackbone=Haiku 4.52026.06 | 72.6 | 77.3 | 73.3 | |
| Simple Strands AgentBackbone=Qwen3-Coder Next2026.06 | 72.44 | 78.2 | 71.3 | |
| MiniMax-M2.5Framework=OpenHands, Localization Setting=Baseline2026.06 | 72.2 | — | — | |
| Simple Strands AgentBackbone=Sonnet 4.02026.06 | 72 | 76.82 | 72.7 | |
| MiniMax-M2.5Framework=OpenHands, Localization Setting=Masked2026.06 | 71.8 | — | — | |
| Qwen3-Next-80B-A3B-InstructFramework=SWE-Agent, Localization Setting=Oracle GPT-5.22026.06 | 71 | — | — | |
| MiniMax-M2.5Framework=SWE-Agent, Localization Setting=All SHERLOC2026.06 | 70.4 | — | — | |
| Qwen3-Coder-NextFramework=OpenHands, Localization Setting=Oracle GPT-5.22026.06 | 68.4 | — | — | |
| Qwen3-Next-80B-A3B-InstructFramework=OpenHands, Localization Setting=Oracle GPT-5.22026.06 | 68.4 | — | — | |
| Simple Strands AgentBackbone=Qwen3-Coder 480B2026.06 | 67.56 | 73.66 | 69.6 | |
| Orchard-SWE (SFT+RL)Base Model=Qwen3-30B-A3B-Thinking, Harness=mini-swe-agent2026.05 | 67.5 | — | — | |
| MiniMax-M2.5Framework=OpenHands, Localization Setting=All SHERLOC2026.06 | 67.2 | — | — | |
| OpenSWE-72BBase Model=Qwen2.5-72B-Base, Harness=SWE-Agent2026.05 | 66 | — | — | |
| OpenSWE-72BBase Model=Qwen2.5-72B-Base, Harness=OpenHands2026.05 | 65 | — | — | |
| Simple Strands AgentBackbone=GPT-OSS 120B2026.06 | 64.64 | 71.9 | 62 | |
| Qwen3-Coder-NextFramework=SWE-Agent, Localization Setting=Oracle GPT-5.22026.06 | 64.6 | — | — | |
| Orchard-SWE (SFT)Base Model=Qwen3-30B-A3B-Thinking, Harness=mini-swe-agent2026.05 | 64.3 | — | — | |
| Scale-SWE-AgentBase Model=Qwen3-30B-A3B-Instruct, Harness=OpenHands2026.05 | 64 | — | — | |
| MiniMax-M2.5Framework=OpenHands, Localization Setting=Shuffled2026.06 | 64 | — | — | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Localization Setting=QF SHERLOC2026.06 | 64 | — | — | |
| Qwen3.5Model Size=35B-A3B2026.06 | 63.5 | — | — | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Localization Setting=Baseline2026.06 | 63 | — | — | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Localization Setting=All SHERLOC2026.06 | 63 | — | — | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Localization Setting=Masked2026.06 | 62.8 | — | — | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Localization Setting=QF SHERLOC2026.06 | 62.8 | — | — | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Localization Setting=All SHERLOC2026.06 | 62.6 | — | — | |
| OpenSWE-32BBase Model=Qwen2.5-32B-Base, Harness=SWE-Agent2026.05 | 62.4 | — | — | |
| Orchard-SWE (SFT)Base Model=Qwen3-30B-A3B-Thinking, Harness=OpenHands2026.05 | 62.1 | — | — | |
| Keye-VL-2.0-30B-A3BModel Size=30B-A3B2026.06 | 62 | — | — | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Localization Setting=Masked2026.06 | 61.8 | — | — | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Localization Setting=Baseline2026.06 | 61.6 | — | — | |
| SWE-Master-32B-RLBase Model=Qwen2.5-Coder-32B-Inst., Harness=R2E-Gym2026.05 | 61.4 | — | — | |
| MiniMax-M2.5Framework=SWE-Agent, Localization Setting=Shuffled2026.06 | 61 | — | — | |
| Kimi-DevBase Model=Qwen2.5-72B-Base, Harness=Agentless2026.05 | 60.6 | — | — | |
| Simple Strands AgentBackbone=GPT-OSS 20B2026.06 | 59.84 | 68.54 | 60 | |
| OpenSWE-32BBase Model=Qwen2.5-32B-Base, Harness=OpenHands2026.05 | 59.8 | — | — | |
| CoderForge-32BBase Model=Qwen3-32B, Harness=OpenHands2026.05 | 59.4 | — | — | |
| GLM-4.7-Flash-30A3B2026.05 | 59.2 | — | — | |
| Qwen3-Coder-480B-A35BFramework=SWE-Agent, Localization Setting=Shuffled2026.06 | 58.6 | — | — | |
| daVinci-Dev-72BBase Model=Qwen2.5-72B-Base, Harness=SWE-Agent2026.05 | 58.5 | — | — | |
| SWE-Master-32BBase Model=Qwen2.5-Coder-32B-Inst., Harness=R2E-Gym2026.05 | 57.8 | — | — | |
| Qwen3-Coder-480B-A35BFramework=OpenHands, Localization Setting=Shuffled2026.06 | 57.8 | — | — | |
| SWE-CompressorBase Model=Qwen2.5-32B-Base, Harness=OpenHands2026.05 | 57.6 | — | — | |
| daVinci-Dev-32BBase Model=Qwen2.5-32B-Base, Harness=SWE-Agent2026.05 | 56.1 | — | — | |
| GPT-5-miniRelease Date/Version=2025-08-072026.06 | 55.5 | — | — | |
| FrogBossBase Model=Qwen3-32B, Harness=R2E-Gym2026.05 | 54.6 | — | — | |
| Qwen3-Coder-NextFramework=OpenHands, Localization Setting=QF SHERLOC2026.06 | 54.6 | — | — | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Localization Setting=All SHERLOC2026.06 | 54 | — | — | |
| Qwen3-Coder-NextFramework=SWE-Agent, Localization Setting=All SHERLOC2026.06 | 53.2 | — | — | |
| Qwen3-Coder-NextFramework=OpenHands, Localization Setting=All SHERLOC2026.06 | 53.2 | — | — | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Localization Setting=All SHERLOC2026.06 | 53 | — | — | |
| SWE-Lego-Qwen3-32BBase Model=Qwen3-32B, Harness=OpenHands2026.05 | 52.6 | — | — | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Localization Setting=QF SHERLOC2026.06 | 52.4 | — | — | |
| SWE-Mirror-LMBase Model=Qwen2.5-Coder-32B-Inst., Harness=MOpenHands2026.05 | 52.2 | — | — | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Localization Setting=QF SHERLOC2026.06 | 52.2 | — | — | |
| Qwen3-Coder-NextFramework=SWE-Agent, Localization Setting=QF SHERLOC2026.06 | 52 | — | — | |
| Qwen3-Coder-30B-A3B-InstructHarness=OpenHands2026.05 | 51.6 | — | — | |
| Simple Strands AgentBackbone=Qwen3-Coder 30B2026.06 | 51.36 | 59.46 | 51.6 | |
| Qwen3-Next-80B-A3B-InstructFramework=SWE-Agent, Localization Setting=QF SHERLOC2026.06 | 50.6 | — | — | |
| Qwen3-Next-80B-A3B-InstructFramework=OpenHands, Localization Setting=All SHERLOC2026.06 | 50.4 | — | — | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Localization Setting=Baseline2026.06 | 49.4 | — | — | |
| Qwen3-Next-80B-A3B-InstructFramework=SWE-Agent, Localization Setting=All SHERLOC2026.06 | 49.4 | — | — | |
| Qwen3-Next-80B-A3B-InstructFramework=OpenHands, Localization Setting=QF SHERLOC2026.06 | 47.8 | — | — | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Localization Setting=Masked2026.06 | 46 | — | — | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Localization Setting=Masked2026.06 | 45.4 | — | — | |
| Qwen3-Coder-NextFramework=SWE-Agent, Localization Setting=Baseline2026.06 | 45.2 | — | — | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Localization Setting=Baseline2026.06 | 44.7 | — | — | |
| Qwen3-Coder-NextFramework=OpenHands, Localization Setting=Baseline2026.06 | 44.6 | — | — | |
| Qwen3-Coder-NextFramework=OpenHands, Localization Setting=Masked2026.06 | 44.6 | — | — | |
| Qwen3-Coder-NextFramework=SWE-Agent, Localization Setting=Shuffled2026.06 | 43.2 | — | — | |
| Qwen3-Next-80B-A3B-InstructFramework=SWE-Agent, Localization Setting=Masked2026.06 | 41.6 | — | — | |
| Qwen3-Coder-NextFramework=SWE-Agent, Localization Setting=Masked2026.06 | 41.2 | — | — | |
| SWE-Agent-LMBase Model=Qwen2.5-Coder-32B-Inst., Harness=SWE-Agent2026.05 | 40.2 | — | — | |
| Qwen3-Coder-30B-A3BFramework=SWE-Agent, Localization Setting=Shuffled2026.06 | 40.2 | — | — | |
| Qwen3-Coder-30B-A3BFramework=OpenHands, Localization Setting=Shuffled2026.06 | 40.2 | — | — | |
| Qwen3-Coder-NextFramework=OpenHands, Localization Setting=Shuffled2026.06 | 39.2 | — | — |