General language understanding and reasoning on Open LLM Leaderboard v1 (test)
44.97MMLU-ProSimPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SimPOBackbone=Qwen2.5-7B-Instruct, Mode=short2025.05 | 44.97 | 73.5 | 54.94 | 38.6 | 31.46 | 48.69 | |
| DPOBackbone=Qwen2.5-7B-Instruct, Mode=short2025.05 | 44.91 | 75.06 | 54.99 | 39.12 | 31.21 | 49.06 | |
| DPOBackbone=Qwen2.5-7B-Instruct, Mode=long2025.05 | 44.91 | 75.3 | 55.06 | 37.99 | 31.8 | 49.01 | |
| SoLo-ORPOBackbone=Qwen2.5-7B-Instruct, Mode=SoLo2025.05 | 44.83 | 75.18 | 55.23 | 37.16 | 31.46 | 48.77 | |
| SFTBackbone=Qwen2.5-7B-Instruct, Mode=short2025.05 | 44.81 | 72.18 | 55.15 | 36.71 | 32.12 | 48.19 | |
| LongPO(reimp)Backbone=Qwen2.5-7B-Instruct2025.05 | 44.8 | 73.86 | 55.07 | 34.81 | 31.91 | 48.08 | |
| ORPOBackbone=Qwen2.5-7B-Instruct, Mode=short2025.05 | 44.78 | 74.7 | 55.26 | 38.21 | 30.95 | 48.78 | |
| SoLo-SimPOBackbone=Qwen2.5-7B-Instruct, Mode=SoLo2025.05 | 44.78 | 75.9 | 54.89 | 37.76 | 31.54 | 48.97 | |
| SFTBackbone=Qwen2.5-7B-Instruct, Mode=long2025.05 | 44.74 | 71.46 | 55.35 | 36.55 | 31.45 | 47.9 | |
| SimPOBackbone=Qwen2.5-7B-Instruct, Mode=long2025.05 | 44.74 | 73.5 | 55.29 | 37.61 | 32.05 | 48.64 | |
| LongPO(pub)Backbone=Qwen2.5-7B-Instruct2025.05 | 44.69 | 76.49 | 53.94 | 32.32 | 31.87 | 47.86 | |
| SoLo-DPOBackbone=Qwen2.5-7B-Instruct, Mode=SoLo2025.05 | 44.66 | 73.98 | 54.78 | 35.57 | 31.63 | 48.12 | |
| ORPOBackbone=Qwen2.5-7B-Instruct, Mode=long2025.05 | 44.64 | 73.61 | 55.37 | 37.16 | 32.12 | 48.58 | |
| InstructBackbone=Qwen2.5-7B-Instruct2025.05 | 44.63 | 74.22 | 55.25 | 36.86 | 31.88 | 48.56 | |
| InstructBackbone=Llama3.1-8B-Instruct2025.05 | 37.79 | 62.23 | 50.98 | 15.25 | 31.71 | 39.59 | |
| SoLo-ORPOBackbone=Llama3.1-8B-Instruct, Mode=SoLo2025.05 | 37.6 | 63.43 | 50.18 | 15.63 | 31.38 | 39.64 | |
| ORPOBackbone=Llama3.1-8B-Instruct, Mode=long2025.05 | 37.43 | 60.43 | 50.86 | 15.86 | 31.63 | 39.24 | |
| ORPOBackbone=Llama3.1-8B-Instruct, Mode=short2025.05 | 37.26 | 61.15 | 50.95 | 14.88 | 32.13 | 39.27 | |
| SFTBackbone=Llama3.1-8B-Instruct, Mode=long2025.05 | 37.04 | 60.67 | 49.64 | 16.16 | 32.38 | 39.17 | |
| SFTBackbone=Llama3.1-8B-Instruct, Mode=short2025.05 | 36.37 | 60.31 | 50.23 | 17.9 | 31.79 | 39.32 |