List your company
ResearchFormerly WizWand

Preference Evaluation on AlpacaEval 2

559WR (%)SelectiveDPO
-21.3304129.3323279.995430.6577Oct 18, 2024Jan 25, 2025May 5, 2025Aug 13, 2025Nov 20, 2025Feb 28, 2026Jun 8, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SelectiveDPOBase Model=Qwen-2.5-7B, Training Dataset=UltraFeedback2026.02559—312—
MixDPOBase Model=Qwen-2.5-7B, Training Dataset=UltraFeedback2026.02559—345—
VANILLA DPOBase Model=Qwen-2.5-7B, Training Dataset=UltraFeedback2026.02360—238—
SimPOBase Model=Qwen-2.5-7B, Training Dataset=UltraFeedback2026.02311—228—
BASE SFTBase Model=Qwen-2.5-7B, Training Dataset=UltraFeedback2026.0299—20—
GRPO + SAVEPolicy Model=Qwen3-4B-Instruct-25072026.0562.6953.28——
GRPO + Improved RMPolicy Model=Qwen3-4B-Instruct-25072026.0562.454.24——
GRPO + R2MPolicy Model=Qwen3-4B-Instruct-25072026.0562.2652——
RePOBackbone=Qwen3-4B-Base2026.0660.1255.08——
GRPOPolicy Model=Qwen3-4B-Instruct-25072026.0559.8151.68——
SFTPolicy Model=Qwen3-4B-Instruct-25072026.0558.5950.01——
REINFORCE++Policy Model=Qwen3-4B-Instruct-25072026.0555.8149.04——
KTOBackbone=Qwen3-4B-Base2026.0655.7852.31——
RePO_detBackbone=Qwen3-4B-Base2026.0655.5351.66——
PRIMEPolicy Model=Qwen3-4B-Instruct-25072026.0552.8343.37——
RePOBackbone=Qwen3-1.7B-Base2026.0643.6636.61——
RePO_detBackbone=Qwen3-1.7B-Base2026.0641.4234.95——
Vanilla RMPolicy=Best-of-N2026.0440.9948.31—1,667
CARPPolicy=Best-of-N2026.0440.9949.94—1,611
KTOBackbone=Qwen3-1.7B-Base2026.0638.9334.73——
IPOBackbone=Qwen3-4B-Base2026.0638.6336.43——
RAG-PrefOffline Alignment=RTO2026.0534.2937.45——
DPOBackbone=Qwen3-4B-Base2026.0633.9232.89——
RAGOffline Alignment=RTO2026.0532.6736.56——
MATRIX-Gen-DPOBase LLM=MATRIX-SFT-Model2024.1031.324.2——
RTO (Offline Baseline)Online Alignment=None2026.0531.336.17——
RPOBackbone=Qwen3-4B-Base2026.0628.2329.51——
GRPO + Improved RMPolicy Model=Qwen2.5-3B-Instruct2026.0527.7220.2——
IPOBackbone=Qwen3-1.7B-Base2026.0626.3724.46——
DPOBackbone=Qwen3-1.7B-Base2026.0625.8423.9——
GRPOPolicy Model=Qwen2.5-3B-Instruct2026.0524.4116.31——
Llama-3-8B-InstructNote=Reference Model2024.1022.5722.92——
GRPO + SAVEPolicy Model=Qwen2.5-3B-Instruct2026.0521.5917.85——
GRPO + R2MPolicy Model=Qwen2.5-3B-Instruct2026.0521.0517.45——
Magpie-PRO-DPOBase LLM=MATRIX-SFT-Model2024.1020.318.99——
OrcaBase LLM=MATRIX-SFT-Model2024.1020.117.26——
RAG-PrefOffline Alignment=DPO2026.0519.524.82——
REINFORCE++Policy Model=Qwen2.5-3B-Instruct2026.0519.115.98——
RAG-PrefOffline Alignment=PPO2026.0519.0728.59——
UltraFeedbackBase LLM=MATRIX-SFT-Model2024.1018.4817.17——
SFTPolicy Model=Qwen2.5-3B-Instruct2026.0518.4115.36——
RAG-PrefOffline Alignment=SimPO2026.0518.1423.18——
RAGOffline Alignment=PPO2026.0517.7627.78——
TDPOBackbone=Qwen3-4B-Base2026.0617.0817.97——
RAGOffline Alignment=DPO2026.0516.8321.81——
RPOBackbone=Qwen3-1.7B-Base2026.0615.4718.63——
RAGOffline Alignment=SimPO2026.0514.6619.66——
PPO (Offline Baseline)Online Alignment=None2026.0514.6618.3——
PRIMEPolicy Model=Qwen2.5-3B-Instruct2026.0514.3312.18——
Tulu-3-DPOBase LLM=MATRIX-SFT-Model2024.1013.7912.91——
DPO (Offline Baseline)Online Alignment=None2026.0512.8614.46——
BaseBackbone=Qwen3-4B-Base2026.0611.6212.8——
ArgillaMixBase LLM=MATRIX-SFT-Model2024.1011.159.75——
RAG-PrefOffline Alignment=SFT2026.0510.8714.48——
TDPOBackbone=Qwen3-1.7B-Base2026.0610.7212.24——
SimPO (Offline Baseline)Online Alignment=None2026.0510.5616.85——
RAGOffline Alignment=SFT2026.0510.3713.26——
OPADOffline Alignment=RTO2026.059.6210.69——
SFT (Offline Baseline)Online Alignment=None2026.059.4414.17——
OPADOffline Alignment=PPO2026.057.9510.16——
OPADOffline Alignment=SimPO2026.057.958.41——
BaseBackbone=Qwen3-1.7B-Base2026.066.058.27——
OPADOffline Alignment=DPO2026.055.597.61——
OPADOffline Alignment=SFT2026.050.992.26——