Preference Alignment on Argilla
5.9Reward (R)AAD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AADBase Model=Llama 8B2025.09 | 5.9 | — | |
| AADBase Model=Llama 3B2025.09 | 5.64 | — | |
| AADModel=Llama 3B, Training Protocol=full finetuning2025.09 | 5.25 | — | |
| EFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 5.01 | 56 | |
| EFTBase Model=Llama 8B2025.09 | 4.65 | 72 | |
| EFTBase Model=Llama 3B2025.09 | 4.54 | 70 | |
| Bo2Model=Llama 3B, Training Protocol=full finetuning2025.09 | 4.06 | 76 | |
| AADBase Model=Qwen 4B2025.09 | 3.84 | — | |
| Greedy DPOModel=Llama 3B, Training Protocol=full finetuning2025.09 | 3.64 | 79 | |
| AADModel=Llama 1B, Training Protocol=full finetuning2025.09 | 3.39 | — | |
| EFTBase Model=Qwen 4B2025.09 | 3.28 | 61 | |
| Bo2Base Model=Llama 8B2025.09 | 3.16 | 86 | |
| Bo2Base Model=Llama 3B2025.09 | 3.02 | 84 | |
| EFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | 2.82 | 58 | |
| Greedy DPOBase Model=Llama 8B2025.09 | 2.55 | 87 | |
| Greedy DPOBase Model=Llama 3B2025.09 | 2.48 | 86 | |
| AADBase Model=Qwen 0.6B2025.09 | 2.33 | — | |
| Bo2Model=Llama 1B, Training Protocol=full finetuning2025.09 | 2.17 | 72 | |
| EFTBase Model=Qwen 0.6B2025.09 | 1.99 | 52 | |
| Bo2Base Model=Qwen 4B2025.09 | 1.94 | 78 | |
| Greedy SFTBase Model=Llama 8B2025.09 | 1.72 | 89 | |
| Greedy DPOModel=Llama 1B, Training Protocol=full finetuning2025.09 | 1.65 | 75 | |
| Greedy SFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 1.59 | 91 | |
| Greedy SFTBase Model=Llama 3B2025.09 | 1.59 | 88 | |
| Greedy DPOBase Model=Qwen 4B2025.09 | 1.37 | 82 | |
| Greedy SFTBase Model=Qwen 4B2025.09 | 0.7 | 87 | |
| Bo2Base Model=Qwen 0.6B2025.09 | 0.68 | 77 | |
| Greedy DPOBase Model=Qwen 0.6B2025.09 | 0.12 | 80 | |
| Greedy SFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | 0.02 | 85 | |
| Greedy SFTBase Model=Qwen 0.6B2025.09 | -0.86 | 89 |