Preference Alignment on UltraFeedback
83Win RateEFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| EFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 83 | 0.5 | |
| CLIPer (Vanilla Baseline)Evaluator=GPT-4o-mini, Preference Dimensions=3-dim2026.05 | 81 | — | |
| CLIPer (Vanilla Baseline)Evaluator=Reward Model, Preference Dimensions=3-dim2026.05 | 79 | — | |
| Greedy SFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 77 | 0.58 | |
| p-soup & Direct Fine-tuningEvaluator=Reward Model, Preference Dimensions=3-dim2026.05 | 75.33 | — | |
| Greedy SFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | 72 | -0.39 | |
| Greedy DPOModel=Llama 3B, Training Protocol=full finetuning2025.09 | 70 | 1.04 | |
| DPO+FilterBase Architecture=Qwen 7b, Pn Type=12025.10 | 67 | -3.12 | |
| Greedy DPOModel=Llama 1B, Training Protocol=full finetuning2025.09 | 65 | -0.03 | |
| Bo2Model=Llama 3B, Training Protocol=full finetuning2025.09 | 65 | 1.22 | |
| DPO+FilterBase Architecture=Qwen 7b, Pn Type=22025.10 | 64 | -3.75 | |
| DPOBase Architecture=Qwen 7b, Pn Type=12025.10 | 63 | -3.59 | |
| Bo2Model=Llama 1B, Training Protocol=full finetuning2025.09 | 61 | 0.18 | |
| p-soup & Direct Fine-tuningEvaluator=GPT-4o-mini, Preference Dimensions=3-dim2026.05 | 58.33 | — | |
| DPOBase Architecture=Qwen 7b, Pn Type=22025.10 | 58 | -4.42 | |
| DPO+FilterBase Architecture=Llama 8b, Pn Type=12025.10 | 56 | -9.25 | |
| EFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | 56 | 0.3 | |
| Direct PromptingEvaluator=GPT-4o-mini, Preference Dimensions=3-dim2026.05 | 53.67 | — | |
| DPOBase Architecture=Llama 8b, Pn Type=12025.10 | 52 | -9.51 | |
| DPO+FilterBase Architecture=Llama 8b, Pn Type=22025.10 | 52 | -9.7 | |
| DPOBase Architecture=Llama 8b, Pn Type=22025.10 | 51 | -9.87 | |
| BaseBase Architecture=Qwen 7b, Pn Type=-2025.10 | 50 | -5.47 | |
| BaseBase Architecture=Llama 8b, Pn Type=-2025.10 | 50 | -10.02 | |
| Direct PromptingEvaluator=Reward Model, Preference Dimensions=3-dim2026.05 | 46 | — | |
| AADModel=Llama 1B, Training Protocol=full finetuning2025.09 | — | 0.51 | |
| AADModel=Llama 3B, Training Protocol=full finetuning2025.09 | — | 1.59 |