List your company
ResearchFormerly WizWand

Reasoning on Big Bench Hard 3-shot

41.46AccuracyUltraMix
32.682434.961237.2439.5188Nov 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=3-shot2025.1141.46
UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=3-shot2025.1141.33
TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=3-shot2025.1140.32
UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=3-shot2025.1139.98
ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=3-shot2025.1139.31
SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=3-shot2025.1138.43
HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=3-shot2025.1137.35
UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=3-shot2025.1136.81
CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=3-shot2025.1135.33
UM-190kTraining Dataset=UM-190k2025.1135.31
UM-187kTraining Dataset=UM-187k2025.1135.2
UM-170kTraining Dataset=UM-170k2025.1134.79
TuluDPOTraining Dataset=TuluDPO2025.1134.14
CodePrefTraining Dataset=CodePref2025.1133.59
ORPOTraining Dataset=ORPO2025.1133.43
HelpSteerTraining Dataset=HelpSteer2025.1133.31
SFTTraining Dataset=SFT2025.1133.29
UltraFBTraining Dataset=UltraFB2025.1133.02