LLM Alignment Evaluation on AlpacaEval 2
51.9LC Win RateSDPO
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SDPOBase Model=Qwen3-8B2026.02 | 51.9 | — | — | — | |
| SDPOBase Model=Olmo3-7B-DPO2026.02 | 51.8 | — | — | — | |
| Olmo3-7B-DPOModel Size=7B, Training=DPO2026.02 | 50.4 | — | — | — | |
| HyPOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 49.5 | 46.2 | — | — | |
| Qwen3-8BModel Size=8B2026.02 | 49.3 | — | — | — | |
| AdaDPOConfiguration Selection=Best across 16 (lr, beta) combinations2026.05 | 48.3 | 46.1 | 1.8 | 1,908 | |
| AAOBackbone=Llama3.1-8B-Instruct2025.11 | 48.11 | 44.89 | — | — | |
| SimPOBackbone=Llama3.1-8B-Instruct2025.11 | 47.5 | 43.64 | — | — | |
| TIS-DPOBackbone=Llama3.1-8B-Instruct2025.11 | 47.04 | 43.99 | — | — | |
| RTOBackbone=Llama3.1-8B-Instruct2025.11 | 46.84 | 41.98 | — | — | |
| TR-DPOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 46.7 | 42.7 | — | — | |
| RainbowPOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 46.7 | 43.5 | — | — | |
| TDPOBackbone=Llama3.1-8B-Instruct2025.11 | 46.56 | 43.21 | — | — | |
| DPOConfiguration Selection=Best across 16 (lr, beta) combinations2026.05 | 46.4 | 44.5 | 1.8 | 1,933 | |
| SDPOBase Model=Qwen3-4B2026.02 | 46.1 | — | — | — | |
| SimPOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 46 | 43.1 | — | — | |
| DPOBackbone=Llama3.1-8B-Instruct2025.11 | 45.87 | 44.34 | — | — | |
| IPOBackbone=Llama3.1-8B-Instruct2025.11 | 45.43 | 44.56 | — | — | |
| FocalPOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 45.1 | 43.6 | — | — | |
| KTOBackbone=Llama3.1-8B-Instruct2025.11 | 43.86 | 42 | — | — | |
| DPOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 40.9 | 41.3 | — | — | |
| KTOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 40.5 | 39 | — | — | |
| HyPOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 38.9 | 47.9 | — | — | |
| SimPOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 38.4 | 40 | — | — | |
| CPOModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 38.1 | 40.4 | — | — | |
| Qwen3-4BModel Size=4B2026.02 | 37.9 | — | — | — | |
| SLiC-HFModel Family=Llama-3, Model State=Instruct, Model Parameters (B)=8B2026.02 | 36.7 | 36.8 | — | — | |
| AAOBackbone=Mistral-7B-Instruct2025.11 | 36.66 | 38.54 | — | — | |
| TR-DPOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 36.5 | 33.7 | — | — | |
| DPO-RandomBackbone=Llama3.1-8B-Instruct2025.11 | 36.36 | 35.1 | — | — | |
| SimPOBackbone=Mistral-7B-Instruct2025.11 | 35.93 | 34.48 | — | — | |
| FocalPOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 35.9 | 35 | — | — | |
| RainbowPOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 35.7 | 33.9 | — | — | |
| TDPOBackbone=Mistral-7B-Instruct2025.11 | 35.34 | 30.55 | — | — | |
| SDPOBase Model=Olmo3-7B-SFT2026.02 | 35.2 | — | — | — | |
| DPOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 35.1 | 31.4 | — | — | |
| KTOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 35 | 31.3 | — | — | |
| CPOModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 34.9 | 39.9 | — | — | |
| HyPOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 34.7 | 33.6 | — | — | |
| KTOBackbone=Mistral-7B-Instruct2025.11 | 34.65 | 30.9 | — | — | |
| TIS-DPOBackbone=Mistral-7B-Instruct2025.11 | 34.54 | 33.51 | — | — | |
| RTOBackbone=Mistral-7B-Instruct2025.11 | 34.49 | 32.52 | — | — | |
| Olmo3-7B-SFTModel Size=7B, Training=SFT2026.02 | 34.3 | — | — | — | |
| HyPOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 32.8 | 29.6 | — | — | |
| DPOBackbone=Mistral-7B-Instruct2025.11 | 32.62 | 28.7 | — | — | |
| SLiC-HFModel Family=Mistral, Model State=Instruct, Model Parameters (B)=7B2026.02 | 32.4 | 31.2 | — | — | |
| TR-DPOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 31.8 | 30.2 | — | — | |
| IPOBackbone=Mistral-7B-Instruct2025.11 | 31.46 | 30.31 | — | — | |
| SimPOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 30.7 | 26.2 | — | — | |
| RainbowPOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 30.3 | 27.1 | — | — | |
| DPO-RandomBackbone=Mistral-7B-Instruct2025.11 | 30.14 | 28.45 | — | — | |
| SFTConfiguration Selection=Best across 16 (lr, beta) combinations2026.05 | 28.5 | 28.8 | 1.6 | 1,976 | |
| RainbowPOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 28.4 | 26.7 | — | — | |
| AAOBackbone=Llama3.1-8B-Base2025.11 | 28.36 | 40.23 | — | — | |
| SimPOBackbone=Llama3.1-8B-Base2025.11 | 27.45 | 33.03 | — | — | |
| SimPOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 27.3 | 25.4 | — | — | |
| FocalPOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 27.2 | 25.4 | — | — | |
| TIS-DPOBackbone=Llama3.1-8B-Base2025.11 | 26.84 | 31.47 | — | — | |
| FocalPOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 25.8 | 19.7 | — | — | |
| KTOBackbone=Llama3.1-8B-Base2025.11 | 25.79 | 24.79 | — | — | |
| AAOBackbone=Mistral-7B-Base2025.11 | 25.01 | 23.45 | — | — | |
| TR-DPOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 24.9 | 21.4 | — | — | |
| RTOBackbone=Llama3.1-8B-Base2025.11 | 24.43 | 25.84 | — | — | |
| SimPOBackbone=Mistral-7B-Base2025.11 | 24.4 | 22.8 | — | — | |
| IPOBackbone=Llama3.1-8B-Base2025.11 | 24.38 | 24.9 | — | — | |
| DPOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 24.3 | 21.9 | — | — | |
| KTOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 23.6 | 20.3 | — | — | |
| TDPOBackbone=Llama3.1-8B-Base2025.11 | 23.34 | 26.45 | — | — | |
| PSFT → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.08 | 23.29 | 20.13 | — | — | |
| DPOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 22.6 | 18.5 | — | — | |
| DPOBackbone=Llama3.1-8B-Base2025.11 | 22.45 | 31.3 | — | — | |
| CPOModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 22.3 | 24.6 | — | — | |
| TIS-DPOBackbone=Mistral-7B-Base2025.11 | 21.57 | 18.84 | — | — | |
| TDPOBackbone=Mistral-7B-Base2025.11 | 21.47 | 18.56 | — | — | |
| DPOBackbone=Mistral-7B-Base2025.11 | 20.45 | 17.55 | — | — | |
| RTOBackbone=Mistral-7B-Base2025.11 | 20.4 | 16.77 | — | — | |
| SLiC-HFModel Family=Llama-3, Model State=Base, Model Parameters (B)=8B2026.02 | 19.8 | 15.9 | — | — | |
| PSFTprolong → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.08 | 19.26 | 15.17 | — | — | |
| DPO-RandomBackbone=Llama3.1-8B-Base2025.11 | 19.07 | 33.97 | — | — | |
| DPO-RandomBackbone=Mistral-7B-Base2025.11 | 17.71 | 11.22 | — | — | |
| IPOBackbone=Mistral-7B-Base2025.11 | 17.13 | 12.13 | — | — | |
| SFT → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.08 | 16.96 | 13.4 | — | — | |
| KTOBackbone=Mistral-7B-Base2025.11 | 16.49 | 9.32 | — | — | |
| CPOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 13.1 | 11.6 | — | — | |
| KTOModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 12.9 | 9.3 | — | — | |
| SFTBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.08 | 12.24 | 8.52 | — | — | |
| PSFTprolongBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.08 | 11.95 | 8.37 | — | — | |
| PSFTBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.08 | 11.79 | 7.49 | — | — | |
| SLiC-HFModel Family=Mistral, Model State=Base, Model Parameters (B)=7B2026.02 | 11.6 | 9.1 | — | — |