Preference Alignment Evaluation on Nectar
3.7Reward (R)AAD
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| AADBase Model=Llama 8B2025.09 | 3.7 | — | — | — | |
| AADBase Model=Llama 3B2025.09 | 3.63 | — | — | — | |
| AADModel=Llama 3B, Training Protocol=full finetuning2025.09 | 3.45 | — | — | — | |
| EFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 3.35 | — | — | 58 | |
| EFTBase Model=Llama 8B2025.09 | 3.3 | — | — | 75 | |
| AADModel=Llama 1B, Training Protocol=full finetuning2025.09 | 3.05 | — | — | — | |
| Bo2Model=Llama 3B, Training Protocol=full finetuning2025.09 | 2.9 | — | — | 79 | |
| AADBase Model=Qwen 4B2025.09 | 2.71 | — | — | — | |
| EFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | 2.68 | — | — | 65 | |
| Bo2Base Model=Llama 8B2025.09 | 2.64 | — | — | 93 | |
| Greedy DPOModel=Llama 3B, Training Protocol=full finetuning2025.09 | 2.46 | — | — | 89 | |
| EFTBase Model=Qwen 4B2025.09 | 2.35 | — | — | 65 | |
| Bo2Model=Llama 1B, Training Protocol=full finetuning2025.09 | 2.28 | — | — | 77 | |
| EFTBase Model=Llama 3B2025.09 | 2.28 | — | — | 89 | |
| Bo2Base Model=Llama 3B2025.09 | 2.15 | — | — | 95 | |
| Greedy DPOBase Model=Llama 8B2025.09 | 2.12 | — | — | 99 | |
| Bo2Base Model=Qwen 4B2025.09 | 1.99 | — | — | 74 | |
| AADBase Model=Qwen 0.6B2025.09 | 1.68 | — | — | — | |
| Greedy DPOBase Model=Llama 3B2025.09 | 1.45 | — | — | 98 | |
| Greedy DPOBase Model=Qwen 4B2025.09 | 1.45 | — | — | 85 | |
| Greedy DPOModel=Llama 1B, Training Protocol=full finetuning2025.09 | 1.32 | — | — | 91 | |
| EFTBase Model=Qwen 0.6B2025.09 | 1.23 | — | — | 58 | |
| Greedy SFTBase Model=Llama 8B2025.09 | 1.17 | — | — | 99 | |
| Bo2Base Model=Qwen 0.6B2025.09 | 1.07 | — | — | 70 | |
| Greedy SFTBase Model=Qwen 4B2025.09 | 0.77 | — | — | 94 | |
| Greedy SFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 0.72 | — | — | 98 | |
| Greedy SFTBase Model=Llama 3B2025.09 | 0.72 | — | — | 99 | |
| Greedy DPOBase Model=Qwen 0.6B2025.09 | 0.09 | — | — | 84 | |
| Greedy SFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | -0.26 | — | — | 98 | |
| Greedy SFTBase Model=Qwen 0.6B2025.09 | -0.77 | — | — | 93 | |
| Pref-CTRLTrained Dataset=HH-RLHF, Zero-shot transfer setting=true2026.04 | — | 32.33 | 35.67 | — | |
| RE-ControlTrained Dataset=HH-RLHF, Zero-shot transfer setting=true2026.04 | — | 30.67 | 33.67 | — |