List your company
ResearchFormerly WizWand

Knowledge on MMLU-Pro (5-shot)

44.65Knowledge Score (5-shot)UltraMix-190k
11.557220.148628.7437.3314Nov 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1144.65
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1144.5
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1143.48
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1143.32
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1143.16
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1130.96
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1130.24
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1128.98
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1128.5
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1128.08
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=5-shot2025.1127.86
UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.1127.84
UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.1127.69
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=5-shot2025.1127.17
TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.1127.1
UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.1127.1
UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.1126.75
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=5-shot2025.1126.44
ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.1126.39
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=5-shot2025.1126.06
HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.1126.02
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=5-shot2025.1125.95
SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.1125.94
CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.1125.78
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=5-shot2025.1125.44
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=5-shot2025.1125.39
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=5-shot2025.1125.19
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=5-shot2025.1125.14
UM-190kTraining Dataset=UM-190k2025.1114.77
UM-187kTraining Dataset=UM-187k2025.1114.64
CodePrefTraining Dataset=CodePref2025.1114.28
TuluDPOTraining Dataset=TuluDPO2025.1114.07
ORPOTraining Dataset=ORPO2025.1114.01
UM-170kTraining Dataset=UM-170k2025.1114
UltraFBTraining Dataset=UltraFB2025.1113.45
SFTTraining Dataset=SFT2025.1113.02
HelpSteerTraining Dataset=HelpSteer2025.1112.83