List your company
ResearchFormerly WizWand

LLM Alignment Evaluation on Arena-Hard

42.7Win RateAAO
-0.35610.8222233.178Oct 18, 2024Dec 24, 2024Mar 2, 2025May 9, 2025Jul 15, 2025Sep 21, 2025Nov 28, 2025
Updated 6mo ago

Evaluation results

MethodLinks
AAOBackbone=Llama3.1-8B-Instruct2025.1142.7—
AAOBackbone=Llama3.1-8B-Base2025.1141—
PSFT → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.0836.4—
SimPOBackbone=Llama3.1-8B-Instruct2025.1133.5—
SimPOBackbone=Llama3.1-8B-Base2025.1130.6—
RTOBackbone=Llama3.1-8B-Instruct2025.1130.4—
PSFTprolong → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.0830.2—
TIS-DPOBackbone=Llama3.1-8B-Instruct2025.1130.1—
DPOBackbone=Llama3.1-8B-Instruct2025.1129.6—
IPOBackbone=Llama3.1-8B-Instruct2025.1128.3—
DPO-RandomBackbone=Llama3.1-8B-Instruct2025.1127.9—
TIS-DPOBackbone=Llama3.1-8B-Base2025.1127.8—
TDPOBackbone=Llama3.1-8B-Base2025.1127.2—
KTOBackbone=Llama3.1-8B-Instruct2025.1126.8—
RTOBackbone=Llama3.1-8B-Base2025.1126.7—
SFT → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.0826.5—
TDPOBackbone=Llama3.1-8B-Instruct2025.1126.4—
DPOBackbone=Llama3.1-8B-Base2025.1126—
KTOBackbone=Llama3.1-8B-Base2025.1125.8—
DPO-RandomBackbone=Llama3.1-8B-Base2025.1125.4—
IPOBackbone=Llama3.1-8B-Base2025.1125.1—
AAOBackbone=Mistral-7B-Instruct2025.1124.2—
Off-Policy AlignDistilBase Model=Qwen2.5-1.5B-Instruct2025.0324.121.8
On-Policy AlignDistilBase Model=Qwen2.5-1.5B-Instruct2025.032423
MATRIX-Gen-DPOBase LLM=MATRIX-SFT-Model2024.1022.7—
SimPOBackbone=Mistral-7B-Instruct2025.1122.6—
Llama-3-8B-InstructNote=Reference Model2024.1020.6—
KTOBackbone=Mistral-7B-Instruct2025.1118.6—
TDPOBackbone=Mistral-7B-Instruct2025.1118.3—
RTOBase Model=Qwen2.5-1.5B-Instruct2025.0318.216.6
TIS-DPOBackbone=Mistral-7B-Instruct2025.1118—
RTOBackbone=Mistral-7B-Instruct2025.1117.9—
SFTBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.0817.9—
PSFTprolongBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.0817.4—
DPOBase Model=Qwen2.5-1.5B-Instruct2025.0317.114.8
DPOBackbone=Mistral-7B-Instruct2025.1117—
Initial ModelBase Model=Qwen2.5-1.5B-Instruct2025.0316.812.7
IPOBackbone=Mistral-7B-Instruct2025.1116.8—
DPO-RandomBackbone=Mistral-7B-Instruct2025.1116.8—
TDPO1Base Model=Qwen2.5-1.5B-Instruct2025.0316.514.1
DPO beta=0.01Base Model=Qwen2.5-1.5B-Instruct, beta=0.012025.0316.215.5
PPOBase Model=Qwen2.5-1.5B-Instruct2025.0315.913.7
Magpie-PRO-DPOBase LLM=MATRIX-SFT-Model2024.1015.9—
TDPO2Base Model=Qwen2.5-1.5B-Instruct2025.0315.813
OrcaBase LLM=MATRIX-SFT-Model2024.1015.2—
KTOBase Model=Qwen2.5-1.5B-Instruct2025.031512.3
UltraFeedbackBase LLM=MATRIX-SFT-Model2024.1014—
PUGCModel=PUGC+DPO, Alignment=DPO2025.0613.7—
AAOBackbone=Mistral-7B-Base2025.1113.6—
Tulu-3-DPOBase LLM=MATRIX-SFT-Model2024.1013.1—
Mistral-7B-instructModel=Mistral-7B-instruct2025.0612.6—
PSFTBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.0811.4—
ArgillaMixBase LLM=MATRIX-SFT-Model2024.1011.3—
On-Policy AlignDistilBase Model=Qwen2-1.5B-Instruct2025.03116.7
Off-Policy AlignDistilBase Model=Qwen2-1.5B-Instruct2025.0310.56
SimPOBackbone=Mistral-7B-Base2025.119.7—
DPO-RandomBackbone=Mistral-7B-Base2025.118.6—
KTOBackbone=Mistral-7B-Base2025.118.2—
TIS-DPOBackbone=Mistral-7B-Base2025.118.2—
DPOBackbone=Mistral-7B-Base2025.118.1—
RTOBackbone=Mistral-7B-Base2025.118.1—
TDPOBackbone=Mistral-7B-Base2025.117.9—
IPOBackbone=Mistral-7B-Base2025.117.5—
DPO beta=0.01Base Model=Qwen2-1.5B-Instruct, beta=0.012025.0376.8
SimPOBase Model=Qwen2-1.5B-Instruct2025.036.94.2
RTOBase Model=Qwen2-1.5B-Instruct2025.036.75.9
SimPOBase Model=Qwen2.5-1.5B-Instruct2025.0344
KTOBase Model=Qwen2-1.5B-Instruct2025.033.44.2
TDPO1Base Model=Qwen2-1.5B-Instruct2025.033.23.9
DPOBase Model=Qwen2-1.5B-Instruct2025.0333.6
PPOBase Model=Qwen2-1.5B-Instruct2025.032.73
Initial ModelBase Model=Qwen2-1.5B-Instruct2025.031.82.8
TDPO2Base Model=Qwen2-1.5B-Instruct2025.031.31.9