Instruction Following on AlpacaEval Length-controlled
73.9ScoreArmoRM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ArmoRMBase Model=Llama-3-8B-Instruct, Training Method=DPO from DSV3-Distill2026.02 | 73.9 | — | |
| RefEvalBase Model=Llama-3-8B-Instruct, Training Method=DPO from DSV3-Distill2026.02 | 73.1 | — | |
| RefEvalBase Model=Qwen2.5-7B-SFT, Training Method=DPO from DSV3-Distill2026.02 | 70 | — | |
| RefFreeBase Model=Llama-3-8B-Instruct, Training Method=DPO from DSV3-Distill2026.02 | 67.5 | — | |
| ArmoRMBase Model=Qwen2.5-7B-SFT, Training Method=DPO from DSV3-Distill2026.02 | 66.8 | — | |
| RefFreeBase Model=Qwen2.5-7B-SFT, Training Method=DPO from DSV3-Distill2026.02 | 65.1 | — | |
| BERTScoreBase Model=Llama-3-8B-Instruct, Training Method=DPO from DSV3-Distill2026.02 | 58.8 | — | |
| ROUGEBase Model=Llama-3-8B-Instruct, Training Method=DPO from DSV3-Distill2026.02 | 56.4 | — | |
| BERTScoreBase Model=Qwen2.5-7B-SFT, Training Method=DPO from DSV3-Distill2026.02 | 55.3 | — | |
| DSV3-DistillBase Model=Llama-3-8B-Instruct2026.02 | 53.9 | — | |
| ROUGEBase Model=Qwen2.5-7B-SFT, Training Method=DPO from DSV3-Distill2026.02 | 50.9 | — | |
| ArmoRM-BaseBase Model=Llama-3-8B-Instruct2026.02 | 49.2 | — | |
| DSV3-DistillBase Model=Qwen2.5-7B-SFT2026.02 | 48.8 | — | |
| DPO (via Rubric-ARROW)Backbone=Qwen2.5-7B-Instruct2026.05 | 45.7 | 50.7 | |
| IterDPO (via Rubric-ARROW)Backbone=Qwen2.5-7B-Instruct2026.05 | 45.5 | 53 | |
| IterDPO (via RIFL)Backbone=Qwen2.5-7B-Instruct2026.05 | 42.4 | 49.7 | |
| DPO (via Rubric-ARROWw/o RL)Backbone=Qwen2.5-7B-Instruct2026.05 | 42.1 | 47.4 | |
| IterDPO (via Rubric-ARROWw/o RL)Backbone=Qwen2.5-7B-Instruct2026.05 | 42.1 | 49.4 | |
| IterDPO (via Rubric-RM)Backbone=Qwen2.5-7B-Instruct2026.05 | 42 | 51.3 | |
| DPO (via Skywork)Backbone=Qwen2.5-7B-Instruct2026.05 | 41.5 | 47.9 | |
| DPO (via Rubric-RM)Backbone=Qwen2.5-7B-Instruct2026.05 | 41.3 | 48.6 | |
| DPO (via RIFL)Backbone=Qwen2.5-7B-Instruct2026.05 | 41.1 | 47.5 | |
| IterDPO (via RLCF)Backbone=Qwen2.5-7B-Instruct2026.05 | 39.2 | 46 | |
| DPO (via Ultrafbk.)Backbone=Qwen2.5-7B-Instruct2026.05 | 38.7 | 43.3 | |
| DPO (via ArmoRM)Backbone=Qwen2.5-7B-Instruct2026.05 | 38.1 | 43.2 | |
| DPO (via RLCF)Backbone=Qwen2.5-7B-Instruct2026.05 | 37.1 | 44.1 | |
| Qwen2.5-7B-Instruct2026.05 | 36.2 | 41 | |
| GPT-4 (0314)2026.05 | 35.3 | 39.4 | |
| DPO (via AI Judge)Backbone=Qwen2.5-7B-Instruct2026.05 | 33.4 | 39.4 | |
| SFT (Distilled)Backbone=Qwen2.5-7B-Instruct2026.05 | 33.3 | 32.8 | |
| ArmoRM-BaseBase Model=Qwen2.5-7B-SFT2026.02 | 32.6 | — | |
| RL (via OnlineRubrics)Backbone=Qwen2.5-7B-Instruct2026.05 | 30.4 | — | |
| BaseBase Model=Llama-3-8B-Instruct2026.02 | 25 | — | |
| BaseBase Model=Qwen2.5-7B-SFT2026.02 | 14.4 | — |