List your company
ResearchFormerly WizWand

Knowledge on MMLU (5-shot)

74.01Knowledge (5-shot) ScoreUltraMix-190k
38.80647.945557.08566.2245Nov 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1174.01
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1173.92
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1173.53
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1173.1
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1172.41
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1164.61
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1164.19
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1163.47
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1163.27
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1162.3
UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.1161.92
UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.1161.51
TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.1161.13
ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.1161.07
UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.1161.04
UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.1160.95
CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.1160.85
SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.1160.63
HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.1160.62
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=5-shot2025.1158.96
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=5-shot2025.1158.28
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1158.19
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=5-shot2025.1157.93
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=5-shot2025.1157.7
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1157.54
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=5-shot2025.1157.43
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=5-shot2025.1157.39
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=5-shot2025.1157.14
UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=5-shot2025.1156.9
UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=5-shot2025.1156.57
TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1155.26
UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=5-shot2025.1155.16
ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=5-shot2025.1154.9
SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=5-shot2025.1154.24
CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=5-shot2025.1150.33
HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1150.3
UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=5-shot2025.1150.23
UM-190kBase Model=OLMo-2-1B-SFT, Training Dataset=UM-190k, Evaluation Protocol=5-shot2025.1142.17
UM-187kBase Model=OLMo-2-1B-SFT, Training Dataset=UM-187k, Evaluation Protocol=5-shot2025.1142.04
TuluDPOBase Model=OLMo-2-1B-SFT, Training Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1141.78
ORPOBase Model=OLMo-2-1B-SFT, Training Dataset=ORPO, Evaluation Protocol=5-shot2025.1141.66
UM-170kBase Model=OLMo-2-1B-SFT, Training Dataset=UM-170k, Evaluation Protocol=5-shot2025.1141.45
UltraFBBase Model=OLMo-2-1B-SFT, Training Dataset=UltraFB, Evaluation Protocol=5-shot2025.1141.21
SFTBase Model=OLMo-2-1B-SFT, Training Dataset=SFT, Evaluation Protocol=5-shot2025.1140.62
HelpSteerBase Model=OLMo-2-1B-SFT, Training Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1140.54
CodePrefBase Model=OLMo-2-1B-SFT, Training Dataset=CodePref, Evaluation Protocol=5-shot2025.1140.16