LLM Alignment Evaluation on AlpacaEval 2.0 (test)
30.35LC Win RateOTPO
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| OTPOBackbone=Mistral-7B-Instruct-v0.2, Training Dataset=HelpSteer22025.05 | 30.35 | 28.2 | 1,839 | |
| OTPOImplicit Token Weighting=true2025.05 | 29.64 | 29.54 | 1,991 | |
| BCOImplicit Token Weighting=false2025.05 | 29.23 | 28.76 | 1,955 | |
| IPOBase Model=Qwen-2-Instruct-7B2025.05 | 29.03 | 25.68 | 411 | |
| RSOImplicit Token Weighting=false2025.05 | 28.93 | 28.8 | 1,961 | |
| R-DPOImplicit Token Weighting=false2025.05 | 28.66 | 28.34 | 1,940 | |
| CDPOImplicit Token Weighting=false2025.05 | 28.62 | 28.52 | 1,954 | |
| AOTImplicit Token Weighting=false2025.05 | 28.55 | 27.83 | 1,924 | |
| LDDPOImplicit Token Weighting=true2025.05 | 28.55 | 28.54 | 1,956 | |
| SPPOImplicit Token Weighting=false2025.05 | 28.48 | 28.13 | 1,946 | |
| LR-DPOImplicit Token Weighting=false2025.05 | 28.23 | 22.53 | 1,654 | |
| SGDPOBase Model=Llama-3.1-Instruct-8B2025.05 | 28.22 | 28.96 | 444 | |
| LDDPOBackbone=Mistral-7B-Instruct-v0.2, Training Dataset=HelpSteer22025.05 | 28.22 | 25.84 | 1,815 | |
| BCOBase Model=Llama-3.1-Instruct-8B2025.05 | 28.03 | 29.32 | 435 | |
| DPOImplicit Token Weighting=false2025.05 | 27.91 | 27.45 | 1,945 | |
| NCAImplicit Token Weighting=false2025.05 | 27.72 | 27.82 | 1,951 | |
| DPOBase Model=Llama-3.1-Instruct-8B2025.05 | 27.53 | 28.35 | 438 | |
| SamPOBase Model=Llama-3.1-Instruct-8B2025.05 | 27.45 | 27.69 | 443 | |
| RPOImplicit Token Weighting=false2025.05 | 27.43 | 27.41 | 1,963 | |
| SamPOBackbone=Mistral-7B-Instruct-v0.2, Training Dataset=HelpSteer22025.05 | 27.31 | 25.31 | 1,844 | |
| DPOBackbone=Mistral-7B-Instruct-v0.2, Training Dataset=HelpSteer22025.05 | 27.25 | 24.78 | 1,775 | |
| EXOImplicit Token Weighting=false2025.05 | 27.22 | 26.82 | 1,937 | |
| IPOBase Model=Llama-3.1-Instruct-8B2025.05 | 27.07 | 25.82 | 459 | |
| SamPOImplicit Token Weighting=true2025.05 | 26.95 | 27.16 | 1,991 | |
| SFTBase Model=Llama-3.1-Instruct-8B2025.05 | 26.84 | 27.77 | 459 | |
| SimPOImplicit Token Weighting=true2025.05 | 26.77 | 27.25 | 1,984 | |
| NCABase Model=Llama-3.1-Instruct-8B2025.05 | 26.33 | 27.77 | 441 | |
| IPOImplicit Token Weighting=false2025.05 | 25.02 | 26.06 | 2,021 | |
| SamPOBase Model=Qwen-2-Instruct-7B2025.05 | 24.57 | 26.6 | 426 | |
| DPOBase Model=Qwen-2-Instruct-7B2025.05 | 24.26 | 24.5 | 414 | |
| SGDPOBase Model=Qwen-2-Instruct-7B2025.05 | 23.89 | 24.86 | 419 | |
| BCOBase Model=Qwen-2-Instruct-7B2025.05 | 23.76 | 23.95 | 411 | |
| SimPOBackbone=Mistral-7B-Instruct-v0.2, Training Dataset=HelpSteer22025.05 | 23.56 | 22 | 1,906 | |
| InitialImplicit Token Weighting=false2025.05 | 22.92 | 22.57 | 1,899 | |
| NCABase Model=Qwen-2-Instruct-7B2025.05 | 21.94 | 21.75 | 409 | |
| SFTBase Model=Qwen-2-Instruct-7B2025.05 | 20.98 | 22.2 | 418 | |
| OTPOBackbone=Qwen-2.5-3B-Instruct, Training Dataset=HelpSteer22025.05 | 19.71 | 22.06 | 2,074 | |
| SamPOBackbone=Qwen-2.5-3B-Instruct, Training Dataset=HelpSteer22025.05 | 18.26 | 20.75 | 2,105 | |
| InitialBackbone=Mistral-7B-Instruct-v0.2, Training Dataset=HelpSteer22025.05 | 17.74 | 15.05 | 1,630 | |
| LDDPOBackbone=Qwen-2.5-3B-Instruct, Training Dataset=HelpSteer22025.05 | 17.32 | 20.15 | 2,107 | |
| DPOBackbone=Qwen-2.5-3B-Instruct, Training Dataset=HelpSteer22025.05 | 16.93 | 19.62 | 2,100 | |
| InitialBackbone=Qwen-2.5-3B-Instruct, Training Dataset=HelpSteer22025.05 | 16.82 | 20.1 | 2,145 | |
| SimPOBackbone=Qwen-2.5-3B-Instruct, Training Dataset=HelpSteer22025.05 | 16.75 | 19.47 | 2,112 | |
| MixDPOBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback2026.02 | 14.42 | 36.65 | — | |
| vanilla DPOBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback2026.02 | 9.37 | 16.77 | — | |
| CHESBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback, Selected proportion=90%2026.02 | 8.13 | 13.91 | — | |
| CHESBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback, Selected proportion=50%2026.02 | 6.16 | 11.93 | — | |
| Cal-DPOBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback2026.02 | 4.56 | 7.45 | — | |
| DPOPBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback2026.02 | 4.53 | 4.04 | — | |
| DPO + NLLBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback2026.02 | 4.25 | 8.45 | — | |
| Base-SFTBase model=LLaMA-3-8B, Preference dataset=Ultrafeedback2026.02 | 3.73 | 10.19 | — |