List your company
ResearchFormerly WizWand

Code Generation on MBPP+ (Pass@1, Avg, ΔScore)

76.42Pass@1Individual
48.39255.668562.94570.2215Mar 13, 2026Mar 26, 2026Apr 9, 2026Apr 22, 2026May 6, 2026May 19, 2026Jun 2, 2026
Updated 5mo ago

Evaluation results

MethodLinks
IndividualModel Architecture=Qwen3-30B-A3B, HARC Calibration=false2026.0676.42——
Weight Averaging w/ HARCMerging Strategy=Weight Averaging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=true2026.0676.36——
WUDI-Merging w/ HARCMerging Strategy=WUDI-Merging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=true2026.0676.12——
TIES-Merging w/ HARCMerging Strategy=TIES-Merging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=true2026.0676.05——
Weight AveragingMerging Strategy=Weight Averaging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=false2026.0676.03——
Fisher MergingMerging Strategy=Fisher Merging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=false2026.0675.94——
WUDI-MergingMerging Strategy=WUDI-Merging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=false2026.0675.61——
Fisher Merging w/ HARCMerging Strategy=Fisher Merging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=true2026.0675.38——
TIES-MergingMerging Strategy=TIES-Merging, Model Architecture=Qwen3-30B-A3B, HARC Calibration=false2026.0675.23——
DARE w/ HARCMerging Strategy=DARE, Model Architecture=Qwen3-30B-A3B, HARC Calibration=true2026.0675.18——
DAREMerging Strategy=DARE, Model Architecture=Qwen3-30B-A3B, HARC Calibration=false2026.0674.19——
R1-OracleBackbone=Qwen2.5-3B-Instruct, Supervision Paradigm=100% GT2026.0373.7579.120
Uni-OPDDistillation Scenario=Multi-Teacher Distillation, Model Scale=4B2026.0572.6——
ExPODistillation Scenario=Multi-Teacher Distillation, Model Scale=4B2026.0572——
Uni-OPDDistillation Scenario=Single-Teacher Distillation, Model Scale=4B2026.0571.6——
OPDDistillation Scenario=Multi-Teacher Distillation, Model Scale=4B2026.0570.9——
ExOPDDistillation Scenario=Single-Teacher Distillation, Model Scale=4B2026.0570.7——
ExOPDDistillation Scenario=Multi-Teacher Distillation, Model Scale=4B2026.0570.6——
ExPODistillation Scenario=Single-Teacher Distillation, Model Scale=4B2026.0570.2——
TeacherTraining Strategy=RL2026.0569.8——
SFTDistillation Scenario=Multi-Teacher Distillation, Model Scale=4B2026.0569.6——
OPDDistillation Scenario=Single-Teacher Distillation, Model Scale=4B2026.0568.8——
R-ZeroBackbone=Qwen3-4B2026.0568.3——
BaseBackbone=Qwen3-4B2026.0567.72——
AZRBackbone=Qwen3-4B2026.0567.5——
COSEBackbone=Qwen3-4B2026.0567.5——
Teacher2026.0566.75——
StudentModel Scale=4B2026.0565.3——
R1-pBackbone=Qwen2.5-3B-Instruct, Supervision Paradigm=20% GT2026.036575.67-3.45
MemRewardBackbone=Qwen2.5-3B-Instruct, Supervision Paradigm=20% GT + GNN2026.0363.7577.02-2.1
R1-OracleBackbone=Qwen2.5-1.5B-Instruct, Supervision Paradigm=100% GT2026.0356.2570.470
R1-pBackbone=Qwen2.5-1.5B-Instruct, Supervision Paradigm=20% GT2026.035562.72-7.75
MemRewardBackbone=Qwen2.5-1.5B-Instruct, Supervision Paradigm=20% GT + GNN2026.035568.1-2.37
AZRBackbone=Llama-3.2-3B-Instruct2026.0554.5——
Reverse KL + Ours2026.0551.95——
α-β divergence + Ours2026.0551.95——
COSEBackbone=Llama-3.2-3B-Instruct2026.0551.9——
Forward KL2026.0551.59——
Skew RKL + Ours2026.0551.17——
R-ZeroBackbone=Llama-3.2-3B-Instruct2026.0550.5——
Forward KL + Ours2026.0550.39——
Skew FKL + Ours2026.0550.39——
Skew RKL2026.0550.39——
Skew FKL2026.0550.26——
α-β divergence2026.0550.26——
Hard KD2026.0550——
Reverse KL2026.0550——
Total Variation + Ours2026.0549.97——
Student (No Distill)2026.0549.61——
Total Variation2026.0549.61——
BaseBackbone=Llama-3.2-3B-Instruct2026.0549.47——