List your company
ResearchFormerly WizWand

Coding on MBPP (Accuracy)

98.4AccuracyGPT-5
56.2867.21578.1589.085May 20, 2025Jul 19, 2025Sep 17, 2025Nov 17, 2025Jan 16, 2026Mar 17, 2026May 17, 2026
Updated 4mo ago

Evaluation results

MethodLinks
GPT-5Evaluation Protocol=Closed-Source2026.0198.4
Qwen2.5-CoderModel Size=14B, GPU Hour=∼1.8M2025.0585.4
InfiGFusionModel Size=14B, GPU Hour=1952025.0585.2
Fine-tunedBase LLM=Qwen-2.5-7B-Instruct, Parameters=x4, Evaluation Protocol=0 shot2026.0584.8
Gemini2.5-ProEvaluation Protocol=Closed-Source2026.0183.7
ATLAS (cluster)Evaluation Protocol=In-Distribution2026.0183.6
ATLAS (cluster)Evaluation Protocol=Out-of-Distribution2026.0183.6
DiDi-Merg.-LBase LLM=Qwen-2.5-7B-Instruct, Parameters=x2.0, Evaluation Protocol=0 shot2026.0583.3
GPT-4oEvaluation Protocol=Closed-Source2026.0182.6
FREE-MergingBase LLM=Qwen-2.5-7B-Instruct, Parameters=x2.08, Evaluation Protocol=0 shot2026.0582.5
ATLAS (RL)Evaluation Protocol=Out-of-Distribution2026.0181.8
FuseChatModel Size=14B, GPU Hour=6502025.0581.8
Qwen2.5-InstructModel Size=14B, GPU Hour=∼1.8M2025.0581.7
Twin-MergingBase LLM=Qwen-2.5-7B-Instruct, Parameters=x2.25, Evaluation Protocol=0 shot2026.0581.6
Zero-shotBase LLM=Qwen-2.5-7B-Instruct, Parameters=x1, Evaluation Protocol=0 shot2026.0580.2
InfiFusionModel Size=14B, GPU Hour=1602025.0579.63
FuseLLMModel Size=14B, GPU Hour=2252025.0579.28
BertRouterEvaluation Protocol=Out-of-Distribution2026.0179
RouterDCEvaluation Protocol=Out-of-Distribution2026.0178.7
MiniLogitModel Size=14B, GPU Hour=2202025.0578.49
MAS-GPTBackbone=Qwen-2.5-72B-Instruct2025.0578
Pivot-SFTModel Size=14B, GPU Hour=1202025.0577.86
RouterDCEvaluation Protocol=In-Distribution2026.0177.7
DebateBackbone=Qwen-2.5-72B-Instruct2025.0577.6
AgentVerseBackbone=Qwen-2.5-72B-Instruct2025.0577.6
ESModel=Qwen2.5-3B-Inst2026.0377.2
GRPOModel=Qwen2.5-3B-Inst2026.0377
DyLANBackbone=Qwen-2.5-72B-Instruct2025.0576.8
SingleBackbone=Qwen-2.5-72B-Instruct2025.0576.5
PPOModel=Qwen2.5-3B-Inst2026.0376.3
ES + TT-MVModel=Qwen2.5-3B-Inst2026.0376.3
Yuan3.0-1T Base#Shots=3-shot, Architecture=MoE, # activated params=68.5B, # total params=1010B2026.0175.9
RandOptModel=Qwen2.5-3B-Inst2026.0375.9
CoTBackbone=Qwen-2.5-72B-Instruct2025.0575.8
SCBackbone=Qwen-2.5-72B-Instruct2025.0575.8
MacNetBackbone=Qwen-2.5-72B-Instruct2025.0575.5
AFlow-MathBackbone=Qwen-2.5-72B-Instruct2025.0575.5
DeepSeek-V3-Base#Shots=3-shot, Architecture=MoE, # activated params=37B, # total params=671B2026.0175.4
RandOptModel=OLMo3-7B-Inst2026.0375.1
TT-MV†Model=Qwen2.5-3B-Inst2026.0374.5
MAVBackbone=Qwen-2.5-72B-Instruct2025.0574
Best-of-N‡Model=Qwen2.5-3B-Inst2026.0373
Fine-tunedBase LLM=Llama-3.1-8B-Instruct, Parameters=x4, Evaluation Protocol=0 shot2026.0573
ES + TT-MVModel=OLMo3-7B-Inst2026.0372.5
AdaRASCategory=Steering2026.0172.22
BertRouterEvaluation Protocol=In-Distribution2026.0172.1
ESModel=OLMo3-7B-Inst2026.0372
DiDi-Merg.-LBase LLM=Llama-3.1-8B-Instruct, Parameters=x2.0, Evaluation Protocol=0 shot2026.0571.8
AgentVerseBackbone=Llama-3.3-70B-Instruct2025.0571.3
FREE-MergingBase LLM=Llama-3.1-8B-Instruct, Parameters=x2.08, Evaluation Protocol=0 shot2026.0571.2
GRPOModel=OLMo3-7B-Inst2026.0370.8
Phi-4Model Size=14B, GPU Hour=∼1.0M2025.0570.8
ProbingCategory=Steering2026.0170.37
Twin-MergingBase LLM=Llama-3.1-8B-Instruct, Parameters=x2.25, Evaluation Protocol=0 shot2026.0570.3
MAS-GPTBackbone=Llama-3.3-70B-Instruct2025.0570.3
GRPOModel=Qwen2.5-1.5B-Inst2026.0370.2
ES + TT-MVModel=Qwen2.5-1.5B-Inst2026.0370.2
DyLANBackbone=Llama-3.3-70B-Instruct2025.0570.1
ESModel=Qwen2.5-1.5B-Inst2026.0369.9
CoTBackbone=Llama-3.3-70B-Instruct2025.0569.7
SCBackbone=Llama-3.3-70B-Instruct2025.0569.7
DebateBackbone=Llama-3.3-70B-Instruct2025.0569.7
AFlow-MathBackbone=Llama-3.3-70B-Instruct2025.0569.7
RandOptModel=Qwen2.5-1.5B-Inst2026.0369.6
STMBackbone=Llama-3.2-3B-Instruct2026.0569.6
Best-of-N‡Model=Qwen2.5-1.5B-Inst2026.0369.5
BaseModel=Qwen2.5-3B-Inst2026.0369.5
MAVBackbone=Llama-3.3-70B-Instruct2025.0569.1
Mistral-SmallModel Size=24B, GPU Hour=∼1.6M2025.0568.8
CoTPrompting=Vanilla CoT, Base Model=Qwen3-1.7B2026.0168.78
MLPRouterEvaluation Protocol=In-Distribution2026.0168.7
PPOModel=Qwen2.5-1.5B-Inst2026.0368.5
LLaMA-3.1-405B Base#Shots=3-shot, Architecture=Dense, # activated params=405B, # total params=405B2026.0168.4
TT-MV†Model=Qwen2.5-1.5B-Inst2026.0368
SingleBackbone=Llama-3.3-70B-Instruct2025.0567.9
MLPRouterEvaluation Protocol=Out-of-Distribution2026.0167.7
PPOModel=OLMo3-7B-Inst2026.0367.7
MacNetBackbone=Llama-3.3-70B-Instruct2025.0567.1
Zero-shotBase LLM=Llama-3.1-8B-Instruct, Parameters=x1, Evaluation Protocol=0 shot2026.0566.9
MADBackbone=Qwen-2.5-72B-Instruct2025.0566.3
BaseModel=OLMo3-7B-Inst2026.0365.9
RandOptModel=Llama3.1-8B-Inst2026.0365.2
ESModel=Llama3.1-8B-Inst2026.0364.8
FS RouterEvaluation Protocol=Training-free, Prompting Strategy=Few-shot2026.0164.7
ZS RouterEvaluation Protocol=Training-free, Prompting Strategy=Zero-shot2026.0164.2
Best-of-N‡Model=OLMo3-7B-Inst2026.0364
ES + TT-MVModel=Llama3.1-8B-Inst2026.0363.1
AutoGenBackbone=Llama-3.3-70B-Instruct2025.0562.9
BaseBackbone=Llama-3.2-3B-Instruct2026.0562.7
BaseModel=Qwen2.5-1.5B-Inst2026.0362.3
DFTBackbone=Llama-3.2-3B-Instruct2026.0562.2
ESModel=OLMo3-7B2026.0361.8
GRPOModel=Llama3.1-8B-Inst2026.0361
TT-MV†Model=OLMo3-7B-Inst2026.0360.5
Best-of-N‡Model=Llama3.1-8B-Inst2026.0360
TT-MV†Model=Llama3.1-8B-Inst2026.0359.5
HySparse# Shots=3-shot, Model Architecture=80B MoE (Hybrid 1:11), Attention Variant=HySparse2026.0259.3
Anchored LearningBackbone=Llama-3.2-3B-Instruct2026.0558.7
GRPOModel=OLMo3-7B2026.0358.5
PPOModel=OLMo3-7B2026.0357.9