Summarization on TL;DR (Winrate)
91.8WinrateSignCert-PO
Evaluation Results
| Method | Links | |
|---|---|---|
| SignCert-POBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 91.8 | |
| Dr.GRPOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 90.2 | |
| AdvPOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 90 | |
| CW-DPOAlignment Algorithm=DPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 86.5 | |
| WS-DPOAlignment Algorithm=rDPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 85.9 | |
| CW-IPOAlignment Algorithm=IPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 84.6 | |
| CW-rDPOAlignment Algorithm=rDPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 83.9 | |
| WS-DPOAlignment Algorithm=DPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 81.8 | |
| UWOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 81.8 | |
| HumanAlignment Algorithm=DPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 80.8 | |
| HumanAlignment Algorithm=IPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 80.3 | |
| BSPOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 78.3 | |
| HumanAlignment Algorithm=rDPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 77.6 | |
| WS-DPOAlignment Algorithm=IPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 74.6 | |
| MultiMetricModel=LLaMA, Baseline Comparison=MPO2026.05 | 74.6 | |
| SignCert-POBackbone=Pythia, Model Scale=2.8B, Gold Reward Model=GPT-4.1 Nano2026.04 | 73.8 | |
| AdvPOBackbone=Pythia, Model Scale=2.8B, Gold Reward Model=GPT-4.1 Nano2026.04 | 69.1 | |
| Dr.GRPOBackbone=Pythia, Model Scale=2.8B, Gold Reward Model=GPT-4.1 Nano2026.04 | 68 | |
| Dr.GRPOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 66.9 | |
| SignCert-POBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 66 | |
| TUR-DPOEvaluation Protocol=LLM judge2026.04 | 64.8 | |
| TUR-DPOEvaluation Protocol=Human eval2026.04 | 64.1 | |
| PPOEvaluation Protocol=Human eval2026.04 | 63.7 | |
| CW-IPOAlignment Algorithm=IPO, Model Pair=OPT-125M → OPT-13B2026.03 | 63.2 | |
| BSPOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 63.1 | |
| IPOEvaluation Protocol=LLM judge2026.04 | 61.9 | |
| MultiMetricModel=GPT-J, Baseline Comparison=MPO2026.05 | 61.6 | |
| DPOEvaluation Protocol=LLM judge2026.04 | 61.2 | |
| CW-rDPOAlignment Algorithm=rDPO, Model Pair=OPT-125M → OPT-13B2026.03 | 61 | |
| DPOEvaluation Protocol=Human eval2026.04 | 60.5 | |
| SignCert-POBackbone=Pythia, Model Scale=1B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 60 | |
| UWOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 59.6 | |
| HumanAlignment Algorithm=IPO, Model Pair=OPT-125M → OPT-13B2026.03 | 59.5 | |
| MultiMetricModel=DeepSeek, Baseline Comparison=MPO2026.05 | 58.6 | |
| AdvPOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 58.1 | |
| HumanAlignment Algorithm=rDPO, Model Pair=OPT-125M → OPT-13B2026.03 | 56.6 | |
| HumanAlignment Algorithm=DPO, Model Pair=OPT-125M → OPT-13B2026.03 | 55.7 | |
| MultiMetricModel=BART, Baseline Comparison=MPO2026.05 | 54.8 | |
| WS-DPOAlignment Algorithm=rDPO, Model Pair=OPT-125M → OPT-13B2026.03 | 54.5 | |
| WS-DPOAlignment Algorithm=IPO, Model Pair=OPT-125M → OPT-13B2026.03 | 53.4 | |
| WS-DPOAlignment Algorithm=DPO, Model Pair=OPT-125M → OPT-13B2026.03 | 52.8 | |
| CW-DPOAlignment Algorithm=DPO, Model Pair=OPT-125M → OPT-13B2026.03 | 52.8 | |
| BSPOBackbone=Pythia, Model Scale=2.8B, Gold Reward Model=GPT-4.1 Nano2026.04 | 50 | |
| MultiMetricModel=BART, Baseline Comparison=SFT2026.05 | 47.2 | |
| AdvPOBackbone=Pythia, Model Scale=1B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 47 | |
| MultiMetricModel=GPT-J, Baseline Comparison=SFT2026.05 | 46.8 | |
| MultiMetricModel=LLaMA, Baseline Comparison=SFT2026.05 | 43.4 | |
| MultiMetricModel=GPT-J, Baseline Comparison=RLHF2026.05 | 42.8 | |
| MultiMetricModel=DeepSeek, Baseline Comparison=SFT2026.05 | 40.8 | |
| BSPOBackbone=Pythia, Model Scale=1B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 40.4 | |
| MultiMetricModel=BART, Baseline Comparison=RLHF2026.05 | 40.4 | |
| MultiMetricModel=LLaMA, Baseline Comparison=RLHF2026.05 | 39.2 | |
| UWOBackbone=Pythia, Model Scale=2.8B, Gold Reward Model=GPT-4.1 Nano2026.04 | 38.1 | |
| SFTBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 35.6 | |
| SFTBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 27.3 | |
| SFTBackbone=Pythia, Model Scale=2.8B, Gold Reward Model=GPT-4.1 Nano2026.04 | 25.2 | |
| Dr.GRPOBackbone=Pythia, Model Scale=1B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 21 | |
| SFTBackbone=Pythia, Model Scale=1B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 15.7 | |
| UWOBackbone=Pythia, Model Scale=1B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 6.1 |