List your company
ResearchFormerly WizWand

Automated Software Engineering on SWE-bench Verified

1,770Resolved RateYoutu-LLM 2B
-61.024414.338889.71,365.062Dec 9, 2025Dec 17, 2025Dec 25, 2025Jan 3, 2026Jan 11, 2026Jan 19, 2026Jan 28, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Youtu-LLM 2BNon-thinking Mode=true, Scaffold=R2E-Gym/Openhands, Maximum Step=502025.121,770
SmolLM3 3BNon-thinking Mode=true, Scaffold=R2E-Gym/Openhands, Maximum Step=502025.12720
Qwen3 4BNon-thinking Mode=true, Scaffold=R2E-Gym/Openhands, Maximum Step=502025.12570
AIRBackbone=Gemini 2.52025.1287.1
OpenHandsBackbone=GPT-52025.1279.8
Devstral-Small-2Open Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=24B, Teacher=—, Context=256K2026.0168
SWE-agentBackbone=Claude 4 Sonnet2025.1266.6
GLM-4.6Open Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=357B, Teacher=—, Context=32K2026.0160.8
Qwen3 1.7BNon-thinking Mode=true, Scaffold=R2E-Gym/Openhands, Maximum Step=502025.1260
GLM-4.7-FlashOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=30B, Teacher=—, Context=128K2026.0159.2
Devstral-Small-2Open Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=24B, Teacher=—, Context=64K2026.0159.1
GLM-4.5-AirOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=110B, Teacher=—, Context=64K2026.0157.4
FrogBoss-32BOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=Qwen 3-32B, Teacher=Claude 4 Sonnet, Context=64K2026.0154.6
SERA-32BOpen Source Code=true, Open Source Model=true, Open Source Data=true, Base Model=Qwen 3-32B, Teacher=GLM-4.6, Context=64K2026.0154.2
CWMOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=32B, Teacher=—, Context=128K2026.0153.9
OpenHandsBackbone=Claude 3.5 Sonnet2025.1252.8
SERA-32B-GAOpen Source Code=true, Open Source Model=true, Open Source Data=true, Base Model=Qwen 3-32B, Teacher=GLM-4.5-Air, Context=64K2026.0151.7
Qwen 3-Coder-30BOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=30B, Teacher=—, Context=256K2026.0151.6
GLM-4.5-AirOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=110B, Teacher=—, Context=32K2026.0150.5
Devstral-Small-2Open Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=24B, Teacher=—, Context=32K2026.0150
SERA-32BOpen Source Code=true, Open Source Model=true, Open Source Data=true, Base Model=Qwen 3-32B, Teacher=GLM-4.6, Context=32K2026.0149.5
Kimi-devOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=72B, Teacher=—, Context=32K2026.0148.6
SWE-agentBackbone=Claude 3.5 Sonnet2025.1247.9
SERA-32B-GAOpen Source Code=true, Open Source Model=true, Open Source Data=true, Base Model=Qwen 3-32B, Teacher=GLM-4.5-Air, Context=32K2026.0146.6
AutoCodeRoverBackbone=GPT-4o2025.1246.2
Qwen 3-Coder-30BOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=30B, Teacher=—, Context=32K2026.0145
SWE-SwissOpen Source Code=true, Open Source Model=true, Base Model=Qwen 2.5-32B, Teacher=—, Context=128K2026.0145
DeepSWEOpen Source Code=true, Open Source Model=true, Base Model=Qwen 3-32B, Teacher=—, Context=32K2026.0142.2
GLM-4.7-FlashOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=30B, Teacher=—, Context=64K2026.0139.7
SkyRL-AgentOpen Source Code=true, Open Source Model=true, Base Model=Qwen 3-32B, Teacher=—, Context=32K2026.0139.4
GLM-4.7-FlashOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=30B, Teacher=—, Context=32K2026.0137.3
FrogBoss-32BOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=Qwen 3-32B, Teacher=Claude 4 Sonnet, Context=32K2026.0135
SWE-smithOpen Source Code=true, Open Source Model=true, Open Source Data=true, Base Model=Qwen 2.5-32B, Teacher=Claude 3.7, Context=32K2026.0132.6
SERA-8B-GAOpen Source Code=true, Open Source Model=true, Open Source Data=true, Base Model=Qwen 3-8B, Teacher=GLM-4.5-Air, Context=32K2026.0131.7
SERA-8BOpen Source Code=true, Open Source Model=true, Open Source Data=true, Base Model=Qwen 3-8B, Teacher=GLM-4.6, Context=32K2026.0131.7
SWE-smithBase Model=Qwen 3-32B, Teacher=Claude 3.7, Context=32K2026.0125.6
Qwen 3-32BOpen Source Code=false, Open Source Model=true, Open Source Data=false, Base Model=32B, Teacher=—, Context=32K2026.0124.4
Nex-N1-8BOpen Source Code=true, Open Source Model=true, Base Model=InternLM3-8B, Teacher=—, Context=32K2026.0120.3
SkyRL-8BOpen Source Code=true, Open Source Model=true, Base Model=Qwen 3-8B, Teacher=—, Context=32K2026.019.4