List your company
ResearchFormerly WizWand

Reasoning on ARC-C 25-shot

62.43ScoreUltraMix-190k
50.303653.451856.659.7482Nov 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
UltraMix-190kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1162.43
UltraMix-187kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1162.26
UltraMix-170kBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1162.12
TuluDPOBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1160.32
SFTBase Model=Qwen-2.5-7B-TuluSFT, Training Mixture/Protocol=SFT2025.1159.22
UltraMix-190kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-190k2025.1158.7
UltraMix-187kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-187k2025.1158.34
TuluDPOBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=TuluDPO2025.1157.93
UltraMix-170kBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=UM-170k2025.1157.63
UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.1157.57
UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.1157.3
UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.1157.17
UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.1156.48
TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.1156.23
ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.1156.06
HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.1155.55
SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.1155.29
SFTBase Model=Llama-3.1-8B-TuluSFT, Training Mixture/Protocol=SFT2025.1154.95
Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=25-shot2025.1153.92
Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=25-shot2025.1153.67
Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=25-shot2025.1153.16
Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=25-shot2025.1153.16
Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=25-shot2025.1152.56
CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.1152.47
Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=25-shot2025.1152.3
Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=25-shot2025.1152.22
Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=25-shot2025.1152.13
Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=25-shot2025.1150.77