Large Language Model Evaluation on HuggingFace Open LLM Leaderboard (Full Suite)
55.37GSM8KSynPO
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SynPOBackbone=LLama3-8B, Variant=Instruct2025.06 | 55.37 | — | 83.19 | — | 54.35 | 79.71 | 36.06 | 31.98 | — | 42.08 | 55.42 | 60.61 | |
| SynPOBackbone=Mistral-7B, Variant=Instruct2025.06 | 55.11 | — | 83.76 | — | 54.51 | 79.92 | 35.84 | 32.87 | — | 40.08 | 55.32 | 60.43 | |
| SynPOBackbone=Mistral-7B, Variant=Base2025.06 | 54.95 | — | 81.26 | — | 50.58 | 77.14 | 30.83 | 28.99 | — | 37.84 | 52.1 | 55.24 | |
| DPOBackbone=LLama3-8B, Variant=Instruct2025.06 | 54.83 | — | 82.85 | — | 52.81 | 78.22 | 34.54 | 31.29 | — | 41.32 | 54.45 | 59.76 | |
| SynPOBackbone=LLama3-8B, Variant=Base2025.06 | 54.81 | — | 80.88 | — | 53.04 | 74.55 | 31.42 | 29.03 | — | 38.03 | 52.16 | 55.54 | |
| SimPOBackbone=LLama3-8B, Variant=Instruct2025.06 | 54.72 | — | 82.29 | — | 53.16 | 77.15 | 35.05 | 30.95 | — | 40.09 | 54.33 | 61.24 | |
| KTOBackbone=Mistral-7B, Variant=Base2025.06 | 54.51 | — | 81.75 | — | 48.34 | 77.17 | 27.03 | 27.66 | — | 35.51 | 50.79 | 54.37 | |
| DPOPBackbone=Mistral-7B, Variant=Base2025.06 | 54.32 | — | 80.49 | — | 49.07 | 76.23 | 30.75 | 26.24 | — | 36.12 | 50.99 | 54.73 | |
| SimPOBackbone=Mistral-7B, Variant=Instruct2025.06 | 54.25 | — | 82.04 | — | 52.19 | 78.71 | 29.52 | 32.7 | — | 39.1 | 53.52 | 59.69 | |
| IPOBackbone=LLama3-8B, Variant=Instruct2025.06 | 54.23 | — | 81.95 | — | 51.02 | 78.58 | 33.27 | 30.31 | — | 40.97 | 53.78 | 59.95 | |
| DPOBackbone=Mistral-7B, Variant=Instruct2025.06 | 54.22 | — | 83.22 | — | 51.15 | 78.46 | 33.72 | 29.34 | — | 39.53 | 53.71 | 60.04 | |
| IPOBackbone=Mistral-7B, Variant=Base2025.06 | 54.14 | — | 79.15 | — | 47.25 | 74.15 | 25.52 | 25.59 | — | 34.87 | 49.31 | 53.84 | |
| KTOBackbone=LLama3-8B, Variant=Instruct2025.06 | 54.01 | — | 82.7 | — | 53.63 | 77.1 | 34.12 | 31.15 | — | 41.7 | 54.37 | 60.57 | |
| DPOPBackbone=Mistral-7B, Variant=Instruct2025.06 | 53.65 | — | 82.04 | — | 52.95 | 80.13 | 34.53 | 29.24 | — | 39.69 | 54.02 | 59.9 | |
| DPOPBackbone=LLama3-8B, Variant=Instruct2025.06 | 53.57 | — | 82.52 | — | 52.29 | 79.1 | 36.51 | 30.55 | — | 41.89 | 54.46 | 59.26 | |
| DPOBackbone=Mistral-7B, Variant=Base2025.06 | 53.47 | — | 81.4 | — | 48.72 | 76.66 | 29.18 | 26.85 | — | 35.68 | 50.79 | 54.32 | |
| DPOBackbone=LLama3-8B, Variant=Base2025.06 | 53.45 | — | 80.02 | — | 49.15 | 75.46 | 30.97 | 27.34 | — | 36.53 | 50.94 | 54.61 | |
| SimPOBackbone=LLama3-8B, Variant=Base2025.06 | 53.45 | — | 78.13 | — | 49.15 | 73.46 | 27.73 | 26.88 | — | 36.13 | 49.69 | 52.61 | |
| CPOBackbone=LLama3-8B, Variant=Instruct2025.06 | 53.4 | — | 81.08 | — | 52.75 | 76.81 | 35.08 | 30.51 | — | 39.56 | 53.44 | 58.29 | |
| KTOBackbone=Mistral-7B, Variant=Instruct2025.06 | 53.35 | — | 80.34 | — | 52.77 | 78.19 | 34.02 | 30.62 | — | 40.46 | 53.69 | 59.8 | |
| SimPOBackbone=Mistral-7B, Variant=Base2025.06 | 53.21 | — | 81.03 | — | 49.49 | 76.68 | 29.63 | 26.94 | — | 35.13 | 50.72 | 53.63 | |
| CPOBackbone=Mistral-7B, Variant=Base2025.06 | 53.06 | — | 80.45 | — | 49.15 | 75.15 | 26.32 | 27.05 | — | 34.04 | 49.97 | 54.53 | |
| KTOBackbone=LLama3-8B, Variant=Base2025.06 | 52.45 | — | 78.53 | — | 51.67 | 75.12 | 28.88 | 27.09 | — | 35.89 | 50.31 | 52.84 | |
| IPOBackbone=Mistral-7B, Variant=Instruct2025.06 | 52.42 | — | 81.61 | — | 52.02 | 79.55 | 31.85 | 30.21 | — | 38.75 | 53.09 | 58.31 | |
| DPOPBackbone=LLama3-8B, Variant=Base2025.06 | 52.35 | — | 79.62 | — | 50.12 | 74.53 | 29.38 | 26.96 | — | 36.92 | 50.38 | 53.2 | |
| CPOBackbone=Mistral-7B, Variant=Instruct2025.06 | 52.28 | — | 80.01 | — | 50.28 | 79.15 | 27.81 | 32.66 | — | 38.85 | 52.47 | 58.74 | |
| IPOBackbone=LLama3-8B, Variant=Base2025.06 | 51.76 | — | 79.48 | — | 48.03 | 74.87 | 25.76 | 27.01 | — | 35.47 | 49.28 | 51.83 | |
| CPOBackbone=LLama3-8B, Variant=Base2025.06 | 51.52 | — | 79.42 | — | 49.04 | 73.32 | 26.86 | 28.14 | — | 36.18 | 49.82 | 54.11 | |
| SPACEIteration=Iter4, Backbone=Mistral-7B2025.12 | 46.02 | 65.87 | 84.44 | 58.5 | 51.86 | 74.51 | 35.9 | 45.28 | 30.5 | 31.41 | 52.43 | — | |
| SPACEIteration=Iter3, Backbone=Mistral-7B2025.12 | 45.84 | 65.96 | 84.39 | 58.53 | 51.34 | 74.19 | 33.19 | 45.38 | 29.9 | 31.44 | 52.03 | — | |
| SPACEIteration=Iter2, Backbone=Mistral-7B2025.12 | 45.55 | 65.78 | 84.25 | 58.57 | 50.71 | 73.95 | 33.01 | 45.38 | 29.52 | 31.59 | 51.83 | — | |
| SPACEIteration=Iter1, Backbone=Mistral-7B2025.12 | 45.41 | 64.85 | 83.86 | 58.87 | 48.99 | 75.14 | 31.51 | 45.73 | 29.46 | 30.94 | 51.48 | — | |
| S-SimPOIteration=Iter1, Backbone=Mistral-7B2025.12 | 41.51 | 61.69 | 83.77 | 57.48 | 48.98 | 74.9 | 23.01 | 41.86 | 29.9 | 30.22 | 49.33 | — | |
| SPACEIteration=Iter0, Backbone=Mistral-7B2025.12 | 41.32 | 62.71 | 83.79 | 58.67 | 47.15 | 74.9 | 26.96 | 45.87 | 29.66 | 30.73 | 50.18 | — | |
| S-IPOIteration=Iter3, Backbone=Mistral-7B2025.12 | 41.05 | 60.67 | 83.42 | 57.03 | 44.35 | 74.27 | 23 | 41.73 | 29.32 | 29.69 | 48.45 | — | |
| S-IPOIteration=Iter0, Backbone=Mistral-7B2025.12 | 40.96 | 60.49 | 83.58 | 57.17 | 45.45 | 73.88 | 20.85 | 42.33 | 30.71 | 30.01 | 48.54 | — | |
| SPINIteration=Iter2, Backbone=Mistral-7B2025.12 | 40.26 | 62.2 | 83.6 | 58.03 | 46.92 | 74.43 | 25.06 | 43.5 | 28.95 | 30.37 | 49.33 | — | |
| S-IPOIteration=Iter4, Backbone=Mistral-7B2025.12 | 40.11 | 61.43 | 83.56 | 56.61 | 45.56 | 74.11 | 23.11 | 41.05 | 29.31 | 29.36 | 48.42 | — | |
| S-SimPOIteration=Iter0, Backbone=Mistral-7B2025.12 | 39.67 | 61.35 | 83.76 | 57.29 | 47.44 | 74.66 | 24.88 | 41.86 | 30.1 | 30.04 | 49.11 | — | |
| S-IPOIteration=Iter2, Backbone=Mistral-7B2025.12 | 39.35 | 60.07 | 83.19 | 57.29 | 43.35 | 73.8 | 23.29 | 41.69 | 30.5 | 29.86 | 48.24 | — | |
| S-IPOIteration=Iter1, Backbone=Mistral-7B2025.12 | 38.91 | 60.07 | 83.47 | 57.29 | 43.62 | 73.8 | 21.98 | 42.08 | 30.19 | 29.9 | 48.13 | — | |
| SPINIteration=Iter0, Backbone=Mistral-7B2025.12 | 38.29 | 61.18 | 83.49 | 57.88 | 43.73 | 74.11 | 22.09 | 44.6 | 29.62 | 30.15 | 48.51 | — | |
| Mistral-7BBackbone=Mistral-7B-Base2025.12 | 37.68 | 61.01 | 83.24 | 57.86 | 42.62 | 74.03 | 23.63 | 44.26 | 29.86 | 29.99 | 48.42 | — | |
| S-SimPOIteration=Iter2, Backbone=Mistral-7B2025.12 | 37.09 | 61.52 | 83.77 | 57.16 | 49.58 | 74.51 | 22.44 | 43.04 | 29.17 | 29.97 | 48.83 | — | |
| S-SimPOIteration=Iter3, Backbone=Mistral-7B2025.12 | 35.22 | 61.26 | 83.7 | 57.35 | 49.88 | 74.82 | 21.15 | 42.8 | 29.04 | 30.01 | 48.52 | — | |
| SPINIteration=Iter3, Backbone=Mistral-7B2025.12 | 34.87 | 62.29 | 84.01 | 58.09 | 45.72 | 75.14 | 24.64 | 43.03 | 28.68 | 29.62 | 48.61 | — | |
| SPINIteration=Iter4, Backbone=Mistral-7B2025.12 | 34.7 | 61.86 | 83.99 | 57.99 | 46 | 75.14 | 25.73 | 43.34 | 27.1 | 29.82 | 48.57 | — | |
| S-SimPOIteration=Iter4, Backbone=Mistral-7B2025.12 | 34.38 | 61.35 | 83.68 | 57.48 | 50.02 | 74.19 | 21.31 | 42.67 | 29.16 | 39.4 | 49.36 | — | |
| SPINIteration=Iter1, Backbone=Mistral-7B2025.12 | 32.85 | 61.52 | 84.02 | 57.32 | 47.69 | 73.95 | 21.92 | 40.59 | 28.22 | 30.12 | 47.82 | — |