List your company
ResearchFormerly WizWand

Reasoning on MuSR 0-shot

48.82Reasoning Score (0-shot)UltraMix-190k
33.126437.200741.27545.3493Nov 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1148.82
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1148.58
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1148.15
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1147.87
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1147.16
UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=0-shot2025.1146.19
TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=0-shot2025.1145.68
UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=0-shot2025.1145.43
ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=0-shot2025.1144.81
UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=0-shot2025.1144.38
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1144.02
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1143.42
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1143.25
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1142.43
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1141.93
SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=0-shot2025.1140.92
HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=0-shot2025.1139.39
UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=0-shot2025.1139.11
CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=0-shot2025.1138.58
UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.1138.19
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=0-shot2025.1137.99
UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.1137.98
UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.1137.86
UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.1137.83
ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.1137.7
TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.1137.64
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=0-shot2025.1137.59
HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.1137.43
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=0-shot2025.1137.39
SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.1137.04
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=0-shot2025.1136.92
UM-190kTraining Dataset=UM-190k2025.1136.85
UM-187kTraining Dataset=UM-187k2025.1136.52
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=0-shot2025.1136.24
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=0-shot2025.1136.19
UM-170kTraining Dataset=UM-170k2025.1136.07
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=0-shot2025.1135.98
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=0-shot2025.1135.98
CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.1135.85
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=0-shot2025.1135.71
TuluDPOTraining Dataset=TuluDPO2025.1135.41
SFTTraining Dataset=SFT2025.1134.52
CodePrefTraining Dataset=CodePref2025.1134.32
ORPOTraining Dataset=ORPO2025.1134.2
UltraFBTraining Dataset=UltraFB2025.1133.88
HelpSteerTraining Dataset=HelpSteer2025.1133.73