LLM Alignment Evaluation on Arena-Hard v0.1
50Win RateBASE (GPT-4-0314)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BASE (GPT-4-0314)ITER=–, Preference Model=GEMMA-2B-IT2026.05 | 50 | — | — | — | |
| BASE (GPT-4-0314)ITER=–, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 50 | — | — | — | |
| HRC+DSPPOITER=3, Preference Model=GEMMA-2B-IT2026.05 | 46.8 | — | — | — | |
| HRC+DSPPOITER=2, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 45.5 | — | — | — | |
| HRC+DSPPOITER=3, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 44.7 | — | — | — | |
| HRC+SPPOITER=3, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 44.6 | — | — | — | |
| BT+SPPOITER=3, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 43.7 | — | — | — | |
| HRC+SPPOITER=3, Preference Model=GEMMA-2B-IT2026.05 | 43.6 | — | — | — | |
| DPOConfiguration Selection=Best across 16 (lr, beta) combinations2026.05 | 42.9 | 44.6 | 41.1 | 525 | |
| GPM+SPPOITER=3, Preference Model=GEMMA-2B-IT2026.05 | 42.1 | — | — | — | |
| HRC+SPPOITER=2, Preference Model=GEMMA-2B-IT2026.05 | 42 | — | — | — | |
| AdaDPOConfiguration Selection=Best across 16 (lr, beta) combinations2026.05 | 41.5 | 43.1 | 39.7 | 530 | |
| HRC+SPPOITER=2, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 41.4 | — | — | — | |
| HRC+DSPPOITER=2, Preference Model=GEMMA-2B-IT2026.05 | 41.2 | — | — | — | |
| GPM+SPPOITER=3, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 41.2 | — | — | — | |
| BT+SPPOITER=2, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 40.9 | — | — | — | |
| BT+SPPOITER=3, Preference Model=GEMMA-2B-IT2026.05 | 40.9 | — | — | — | |
| GPM+SPPOITER=2, Preference Model=GEMMA-2B-IT2026.05 | 39.7 | — | — | — | |
| HRC+SPPOITER=1, Preference Model=GEMMA-2B-IT2026.05 | 39.3 | — | — | — | |
| BT+SPPOITER=2, Preference Model=GEMMA-2B-IT2026.05 | 37.8 | — | — | — | |
| HRC+DSPPOITER=1, Preference Model=GEMMA-2B-IT2026.05 | 36.7 | — | — | — | |
| BT+SPPOITER=1, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 35.2 | — | — | — | |
| HRC+DSPPOITER=1, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 34.4 | — | — | — | |
| GPM+SPPOITER=2, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 34.4 | — | — | — | |
| HRC+SPPOITER=1, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 34.1 | — | — | — | |
| BT+SPPOITER=1, Preference Model=GEMMA-2B-IT2026.05 | 32.9 | — | — | — | |
| GPM+SPPOITER=1, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 31.6 | — | — | — | |
| GPM+SPPOITER=1, Preference Model=GEMMA-2B-IT2026.05 | 31.5 | — | — | — | |
| BASE (LLAMA3-8B)ITER=–, Preference Model=GEMMA-2B-IT2026.05 | 29.9 | — | — | — | |
| BASE (LLAMA3-8B)ITER=–, Preference Model=LLAMA-3.1-8B-INSTRUCT2026.05 | 29.9 | — | — | — | |
| SFTConfiguration Selection=Best across 16 (lr, beta) combinations2026.05 | 26.3 | 28.1 | 24.7 | 589 |