List your company
ResearchFormerly WizWand

Code Generation on MBPP+ (Score)

94.2ScoreGLM-4.6
33.651249.370665.0980.8094Dec 15, 2025Dec 26, 2025Jan 6, 2026Jan 17, 2026Jan 28, 2026Feb 8, 2026Feb 19, 2026
Updated 8mo ago

Evaluation results

MethodLinks
GLM-4.6Parameters=357B-A32B, Non-Thinking Mode=true2026.0194.2
DeepSeek-V3.1Parameters=685B-A37B, Non-Thinking Mode=true2026.0192.6
Trinity Large Base2026.0288.62
A.X K1Parameters=519B-A33B, Non-Thinking Mode=true2026.0185.7
Qwen3-Coder-30B-A3B-Instruct (Base)Model=Qwen3-Coder-30B-A3B-Instruct, Strategy=Base2026.0275.13
DeepSeek V3.1Model Variant=Base, # Shots=0-shot, # Activated Params=37B, # Total Params=671B2026.0272.2
MiMo-V2 FlashModel Variant=Base, # Shots=0-shot, # Activated Params=15B, # Total Params=309B2026.0271.4
Step 3.5 FlashModel Variant=Base, # Shots=0-shot, # Activated Params=11B, # Total Params=196B2026.0270.6
Qwen 3 32BModel Family=Qwen 3, Parameter Count=32B, Thinking Capability=false2025.1270.6
DS-R1 32BModel Family=DeepSeek-R1, Parameter Count=32B, Thinking Capability=true2025.1270.1
DeepSeek V3.2Model Variant=Exp Base, # Shots=0-shot, # Activated Params=37B, # Total Params=671B2026.0269.8
Qwen 3 8B2025.1269.1
Olmo 3.1 Think 32BTraining Stage=Final Think 3.1, Model Family=Olmo 3.1, Parameter Count=32B, Thinking Capability=true2025.1268.3
Olmo 3 Think (Final 3.0)Training Stage=Final Think 3.0, Model Family=Olmo 3, Parameter Count=32B, Thinking Capability=true2025.1268
Olmo 3 Think (DPO)Training Stage=DPO, Model Family=Olmo 3, Parameter Count=32B, Thinking Capability=true2025.1267.2
Olmo 3 Think (SFT)Training Stage=SFT, Model Family=Olmo 3, Parameter Count=32B, Thinking Capability=true2025.1266.7
Qwen 3 VL 8B Inststage=Instruct2025.1266.3
Qwen 3 VL 32B ThinkModel Family=Qwen 3, Parameter Count=32B, Thinking Capability=true2025.1266.2
Nemotron Nano 9B v22025.1266.1
K2-V2 70B InstructModel Family=K2, Parameter Count=70B, Thinking Capability=false2025.1266
Olmo 3 7B ThinkStage=Final Think2025.1264.7
Qwen 3 8Bstage=Instruct2025.1264.4
DS-R1 Qwen 7B2025.1263.5
Olmo 3 7B ThinkStage=SFT2025.1263.2
Olmo 3 7B ThinkStage=DPO2025.1263
Qwen 3 VL 8B Think2025.1263
Qwen 2.5 7Bstage=Instruct2025.1262.6
OpenThinker3 7B2025.1261.4
OR Nemotron 7B2025.1261.2
Olmo 3 7B Instructstage=Final Instruct2025.1260.2
Qwen3-4B-Instruct-2507 (C/E Weighted)Model=Qwen3-4B-Instruct-2507, Strategy=C/E Weighted2026.0258.73
Qwen3-4B-Instruct-2507 (Base)Model=Qwen3-4B-Instruct-2507, Strategy=Base2026.0256.61
Qwen3-4B-Instruct-2507 (Edge Only)Model=Qwen3-4B-Instruct-2507, Strategy=Edge Only2026.0256.61
Olmo 3 7B Instructstage=SFT2025.1256.5
Qwen3-4B-Instruct-2507 (Basic Only)Model=Qwen3-4B-Instruct-2507, Strategy=Basic Only2026.0256.34
Olmo 3 7B Instructstage=DPO2025.1255.9
Granite 3.3 8B Inststage=Instruct2025.1254
Qwen3-4B-Instruct-2507 (Uniform)Model=Qwen3-4B-Instruct-2507, Strategy=Uniform2026.0253.7
Qwen3-4B-Instruct-2507 (B/I Weighted)Model=Qwen3-4B-Instruct-2507, Strategy=B/I Weighted2026.0252.38
Qwen3-4B-Instruct-2507 (Complex Only)Model=Qwen3-4B-Instruct-2507, Strategy=Complex Only2026.0251.85
Apertus 8B Inststage=Instruct2025.1242.1
OLMo 2 7B Inststage=Instruct2025.1240.7
Qwen3-4B-Instruct-2507 (C/E Weighted (Rev))Model=Qwen3-4B-Instruct-2507, Strategy=C/E Weighted (Rev)2026.0235.98