Downstream Task Evaluation on LMEval Harness (MMLU, ARC-C, HellaSwag, TruthfulQA, Winogrande, GSM8K)
65.8MMLUInitial Model
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Initial ModelBackbone=Llama-3, Model Type=Instruct, Parameter Scale=8B2026.02 | 65.8 | 56.4 | 59 | 36.1 | 77.3 | 74.8 | 61.6 | |
| DPOBackbone=Llama-3, Model Type=Instruct, Parameter Scale=8B2026.02 | 65.6 | 61.3 | 60.6 | 42.4 | 76.4 | 73.5 | 63.3 | |
| HyPOBackbone=Llama-3, Model Type=Instruct, Parameter Scale=8B2026.02 | 65 | 64.4 | 58.7 | 45.3 | 73.3 | 70.3 | 62.8 | |
| SimPOBackbone=Llama-3, Model Type=Instruct, Parameter Scale=8B2026.02 | 64.7 | 59.8 | 56 | 43.8 | 72.5 | 71.3 | 61.4 | |
| Initial ModelBackbone=Llama-3, Model Type=Base, Parameter Scale=8B2026.02 | 63.7 | 56.9 | 61.4 | 31.5 | 77.4 | 50.3 | 56.9 | |
| HyPOBackbone=Llama-3, Model Type=Base, Parameter Scale=8B2026.02 | 63.6 | 65.4 | 64.3 | 42.8 | 78.9 | 49.7 | 60.8 | |
| DPOBackbone=Llama-3, Model Type=Base, Parameter Scale=8B2026.02 | 63.3 | 61.6 | 64.7 | 37.9 | 78 | 54.7 | 60 | |
| SimPOBackbone=Llama-3, Model Type=Base, Parameter Scale=8B2026.02 | 62.1 | 63.4 | 64.5 | 38 | 77.2 | 50.4 | 59.3 | |
| HyPOBackbone=Mistral, Model Type=Instruct, Parameter Scale=7B2026.02 | 59.6 | 66.8 | 67.4 | 54 | 77.6 | 40.6 | 61 | |
| Initial ModelBackbone=Mistral, Model Type=Instruct, Parameter Scale=7B2026.02 | 59.2 | 58.7 | 66.1 | 52.6 | 78.1 | 44.1 | 59.8 | |
| Initial ModelBackbone=Mistral, Model Type=Base, Parameter Scale=7B2026.02 | 59 | 54.2 | 60.8 | 28 | 77.7 | 34.6 | 52.4 | |
| DPOBackbone=Mistral, Model Type=Instruct, Parameter Scale=7B2026.02 | 58.8 | 60.9 | 67.6 | 51.2 | 78.3 | 41.6 | 59.7 | |
| HyPOBackbone=Mistral, Model Type=Base, Parameter Scale=7B2026.02 | 58.5 | 60.2 | 63.5 | 39.1 | 78.9 | 31.8 | 55.3 | |
| SimPOBackbone=Mistral, Model Type=Base, Parameter Scale=7B2026.02 | 58.4 | 58.6 | 63.6 | 33.1 | 78.1 | 35 | 54.5 | |
| SimPOBackbone=Mistral, Model Type=Instruct, Parameter Scale=7B2026.02 | 58.1 | 62.5 | 67.5 | 55 | 77.4 | 39.8 | 60.1 | |
| DPOBackbone=Mistral, Model Type=Base, Parameter Scale=7B2026.02 | 57.6 | 58.7 | 64.4 | 40.3 | 77.4 | 31.5 | 55 |