List your company
ResearchFormerly WizWand

Math on GSM8K (5-shot)

82.7ScoreUltraMix-190k
35.32847.626559.92572.2235Nov 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1182.7
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1182.48
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1181.26
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1180.93
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1179.98
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1179.48
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1179.19
UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.1178.1
UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.1178.01
UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.1177.71
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1176.84
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1176.19
TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.1173.84
UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.1173.24
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=5-shot2025.1172.74
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=5-shot2025.1172.57
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=5-shot2025.1171.95
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1171.27
ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.1171.05
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1170.66
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=5-shot2025.1170.58
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=5-shot2025.1169.85
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1168.31
HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.1168.05
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=5-shot2025.1167.93
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=5-shot2025.1167.55
UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=5-shot2025.1167.27
UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=5-shot2025.1166.59
HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1165.88
TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1165.17
CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.1165.01
UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=5-shot2025.1164.99
ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=5-shot2025.1163.73
SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.1162.77
UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=5-shot2025.1162.45
CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=5-shot2025.1161.14
SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=5-shot2025.1159.22
UM-190kTraining Dataset=UM-190k2025.1152.36
UM-187kTraining Dataset=UM-187k2025.1152.13
UM-170kTraining Dataset=UM-170k2025.1150.51
TuluDPOTraining Dataset=TuluDPO2025.1150.36
UltraFBTraining Dataset=UltraFB2025.1149.48
ORPOTraining Dataset=ORPO2025.1149.35
HelpSteerTraining Dataset=HelpSteer2025.1143.73
CodePrefTraining Dataset=CodePref2025.1140.44
SFTTraining Dataset=SFT2025.1137.15