List your company
ResearchFormerly WizWand

Software Engineering on SWE-bench Verified (Success Rate (%))

80.9Success RateClaude-Opus-4.5
-3.23618.60740.4562.293Apr 13, 2026Apr 23, 2026May 4, 2026May 15, 2026May 25, 2026Jun 5, 2026Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Claude-Opus-4.52026.0680.9
GPT-5.12026.0676.3
Gemini-3-Pro2026.0676.2
GLM-4.72026.0673.8
DeepSeek-V3.22026.0673.1
OpenHandsBase Model=GPT-5, Scaffold=OpenHands, Data=-2026.0471.8
OpenHandsBase Model=Claude 4 Sonnet, Scaffold=OpenHands, Data=-2026.0470.4
Qwen3-CoderBase Model=Qwen3-Coder-480B, Scaffold=OpenHands, Data=-2026.0469.6
Kimi-K2-Instruct-09052026.0669.2
Qwen3-Coder-480B-A35B-Instruct2026.0667
SWE-agentBase Model=Claude 4 Sonnet, Scaffold=SWE-agent, Data=-2026.0466.6
Kimi-K2Base Model=Kimi-K2-1T, Scaffold=OpenHands, Data=-2026.0465.4
LoopCoder-v2R (number of refinement loops)=22026.0664.4
GLM-4.5Base Model=GLM-4.5-355B, Scaffold=OpenHands, Data=-2026.0464.2
Kimi-Dev-72B2026.0660.4
CodeFuse-CGMBase Model=Qwen2.5-72B, Scaffold=Graph RAG, Data=200k issue-patch pairs2026.0450.4
Best DGM agentAgent Stage=Best, Evaluation Protocol=Endpoint2026.0650
ENTROPO-KTOBase Model=Qwen3-Coder-30B, Scaffold=R2EGym, Data=-2026.0449.3
Kimi-DevBase Model=Qwen 2.5-72B, Scaffold=SWE-Agent, Data=150B tokens +2026.0448.6
Qwen3-235B-A22B-Instruct-25072026.0645.2
DeepSWE+AEMBase Model=Qwen3-32B2026.0543.7
LoopCoder-v2R (number of refinement loops)=12026.0643
DeepSWEBase Model=Qwen3-32B2026.0542.3
DeepSWEBase Model=Qwen3-32B, Scaffold=R2EGym, Data=4.5k SWE tasks2026.0442.2
SWE-RLBase Model=Llama-3.3-70B, Scaffold=Agentless Mini, Data=-2026.0441
SWE-smithBase Model=Qwen2.5-Coder-32B, Scaffold=SWE-agent, Data=5k trajectories2026.0440.2
Agentless-1.5Base Model=GPT-4o, Scaffold=Agentless, Data=-2026.0438.8
Skywork-SWEBase Model=Qwen2.5-Coder, Scaffold=OpenHands, Data=8k trajectories2026.0438
SWE-DevBase Model=Qwen2.5-Coder-32B, Scaffold=OpenHands, Data=19.3k trajectories2026.0436.6
R2EGymBase Model=Qwen2.5-Coder-32B, Scaffold=R2EGym, Data=3.3k trajectories2026.0434.4
SWE-fixerBase Model=Qwen2.5-72B, Scaffold=Pipline, Data=-2026.0432.8
SWE-AGILE (SFT)Base Model=Qwen3-14B, Scaffold=R2EGym, Data=2.2k trajectories2026.0430.06
LoopCoder-v2R (number of refinement loops)=32026.0627.6
R2EGymBase Model=Qwen2.5-Coder-14B, Scaffold=R2EGym, Data=3.3k trajectories2026.0426.8
Best DGM agentAgent Stage=Best, Evaluation Protocol=Transfer (from Polyglot)2026.0624.5
SWE-AGILE (SFT+RL)Base Model=Qwen3-8B, Scaffold=R2EGym, Data=+ 896 SWE tasks2026.0424.1
SWE-DevBase Model=Qwen2.5-Coder-7B, Scaffold=OpenHands, Data=19.3k trajectories2026.0423.4
LoopCoder-v2R (number of refinement loops)=42026.0622.4
SkyRL-Agent-v0Base Model=Qwen3-14B, Scaffold=OpenHands, Data=-2026.0421.6
SWE-AGILE (SFT)Base Model=Qwen3-8B, Scaffold=R2EGym, Data=2.2k trajectories2026.0421.45
SWE-GymBase Model=Qwen2.5-Coder-32B, Scaffold=OpenHands, Data=-2026.0420.6
Initial agentAgent Stage=Initial, Evaluation Protocol=Endpoint2026.0620
Initial agentAgent Stage=Initial, Evaluation Protocol=Transfer (from Polyglot)2026.0620
R2EGymBase Model=Qwen2.5-Coder-7B, Scaffold=R2EGym, Data=3.3k trajectories2026.0419
SWE-GymBase Model=Qwen2.5-Coder-14B, Scaffold=OpenHands, Data=-2026.0416.4
Qwen3Base Model=Qwen3-8B, Scaffold=R2EGym, Data=-2026.0415.83
SWE-smithBase Model=Qwen2.5-Coder-7B, Scaffold=SWE-agent, Data=5k trajectories2026.0415.2
SWE-GymBase Model=Qwen2.5-Coder-7B, Scaffold=OpenHands, Data=491 trajectories2026.0410.6
DeepSeek-Coder-V2-Lite-Instruct2026.060
Qwen2.5-Coder-7B-Instruct2026.060
Seed-Coder-8B-Instruct2026.060
Qwen2.5-Coder-14B-Instruct2026.060
Qwen2.5-Coder-32B-Instruct2026.060