List your company
ResearchFormerly WizWand

Reasoning on ARC-Challenge (25-shot)

57.51AccuracyUltraMix
37.365242.595147.82553.0549Nov 14, 2025
Updated 8mo ago

Evaluation results

MethodLinks
UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=25-shot2025.1157.51
UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=25-shot2025.1157.45
UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=25-shot2025.1156.77
TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=25-shot2025.1156.73
ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=25-shot2025.1155.03
SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=25-shot2025.1154.27
HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=25-shot2025.1152.72
UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=25-shot2025.1152.68
CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=25-shot2025.1151.71
UM-190kTraining Dataset=UM-190k2025.1144.3
UM-187kTraining Dataset=UM-187k2025.1144.17
TuluDPOTraining Dataset=TuluDPO2025.1143.6
UM-170kTraining Dataset=UM-170k2025.1143.34
ORPOTraining Dataset=ORPO2025.1142.02
UltraFBTraining Dataset=UltraFB2025.1141.81
HelpSteerTraining Dataset=HelpSteer2025.1138.65
CodePrefTraining Dataset=CodePref2025.1138.4
SFTTraining Dataset=SFT2025.1138.14