Reasoning on ARC-Challenge (25-shot)
57.51AccuracyUltraMix
Evaluation Results
| Method | Links | |
|---|---|---|
| UltraMixTraining Method=DPO, Dataset=UM-187k, Evaluation Protocol=25-shot2025.11 | 57.51 | |
| UltraMixTraining Method=DPO, Dataset=UM-190k, Evaluation Protocol=25-shot2025.11 | 57.45 | |
| UltraMixTraining Method=DPO, Dataset=UM-170k, Evaluation Protocol=25-shot2025.11 | 56.77 | |
| TuluDPOTraining Method=DPO, Dataset=TuluDPO, Evaluation Protocol=25-shot2025.11 | 56.73 | |
| ORPOTraining Method=ORPO, Dataset=ORPO, Evaluation Protocol=25-shot2025.11 | 55.03 | |
| SmolLM-3-3B-SFTTraining Method=SFT, Evaluation Protocol=25-shot2025.11 | 54.27 | |
| HelpSteerTraining Method=DPO, Dataset=HelpSteer, Evaluation Protocol=25-shot2025.11 | 52.72 | |
| UltraFBTraining Method=DPO, Dataset=UltraFB, Evaluation Protocol=25-shot2025.11 | 52.68 | |
| CodePrefTraining Method=DPO, Dataset=CodePref, Evaluation Protocol=25-shot2025.11 | 51.71 | |
| UM-190kTraining Dataset=UM-190k2025.11 | 44.3 | |
| UM-187kTraining Dataset=UM-187k2025.11 | 44.17 | |
| TuluDPOTraining Dataset=TuluDPO2025.11 | 43.6 | |
| UM-170kTraining Dataset=UM-170k2025.11 | 43.34 | |
| ORPOTraining Dataset=ORPO2025.11 | 42.02 | |
| UltraFBTraining Dataset=UltraFB2025.11 | 41.81 | |
| HelpSteerTraining Dataset=HelpSteer2025.11 | 38.65 | |
| CodePrefTraining Dataset=CodePref2025.11 | 38.4 | |
| SFTTraining Dataset=SFT2025.11 | 38.14 |