Instruction Following on AlpacaFarm
59.2Win RateGR
Evaluation Results
| Method | Links | |
|---|---|---|
| GRModel Size=3B, Judge Model=GPT4.1-Nano2026.02 | 59.2 | |
| KL RegModel Size=3B, Judge Model=GPT4.1-Nano2026.02 | 52.8 | |
| SignCert-POBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 52.3 | |
| AdvPOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 50 | |
| Reference ResetsModel Size=3B, Judge Model=GPT4.1-Nano2026.02 | 49.2 | |
| SignCert-POBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 47.9 | |
| SFTBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 47.9 | |
| Dr.GRPOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 46.9 | |
| UWOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 46.9 | |
| No RegModel Size=3B, Judge Model=GPT4.1-Nano2026.02 | 44.2 | |
| BSPOBackbone=Qwen2.5, Model Scale=3B, Gold Reward Model=GPT-4.1 Nano2026.04 | 43.8 | |
| UWOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 38.3 | |
| SFTBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 37.8 | |
| AdvPOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 37.5 | |
| Dr.GRPOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 32.6 | |
| BSPOBackbone=Qwen2.5, Model Scale=1.5B, Gold Reward Model=Skywork-Reward-Llama-3.1-8B-v0.22026.04 | 31.3 | |
| GRModel Size=1.5B, Judge Model=GPT4.1-Nano2026.02 | 29.2 | |
| KL RegModel Size=1.5B, Judge Model=GPT4.1-Nano2026.02 | 27.6 | |
| Reference ResetsModel Size=1.5B, Judge Model=GPT4.1-Nano2026.02 | 27.1 | |
| SFT ModelModel Size=3B, Judge Model=GPT4.1-Nano2026.02 | 26.3 | |
| No RegModel Size=1.5B, Judge Model=GPT4.1-Nano2026.02 | 21.7 | |
| SFT ModelModel Size=1.5B, Judge Model=GPT4.1-Nano2026.02 | 20.6 | |
| GRModel Size=0.5B, Judge Model=GPT4.1-Nano2026.02 | 18.5 | |
| Reference ResetsModel Size=0.5B, Judge Model=GPT4.1-Nano2026.02 | 17.4 | |
| KL RegModel Size=0.5B, Judge Model=GPT4.1-Nano2026.02 | 16.9 | |
| SFT ModelModel Size=0.5B, Judge Model=GPT4.1-Nano2026.02 | 12.8 | |
| No RegModel Size=0.5B, Judge Model=GPT4.1-Nano2026.02 | 12.8 |