Preference Alignment on OpenRLHF Mixture
7.6RewardAAD
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AADBase Model=Llama 8B2025.09 | 7.6 | — | |
| AADBase Model=Llama 3B2025.09 | 7.28 | — | |
| EFTBase Model=Llama 8B2025.09 | 6.84 | 0.67 | |
| AADModel=Llama 3B, Training Protocol=full finetuning2025.09 | 6.26 | — | |
| EFTBase Model=Llama 3B2025.09 | 6.18 | 0.72 | |
| Bo2Model=Llama 3B, Training Protocol=full finetuning2025.09 | 5.88 | 57 | |
| Bo2Base Model=Llama 8B2025.09 | 5.6 | 0.85 | |
| AADBase Model=Qwen 4B2025.09 | 5.45 | — | |
| Bo2Base Model=Llama 3B2025.09 | 5.34 | 0.83 | |
| EFTBase Model=Qwen 4B2025.09 | 5.29 | 0.54 | |
| EFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 5.24 | 70 | |
| Greedy DPOBase Model=Llama 8B2025.09 | 4.93 | 0.89 | |
| Greedy DPOModel=Llama 3B, Training Protocol=full finetuning2025.09 | 4.91 | 73 | |
| Greedy DPOBase Model=Llama 3B2025.09 | 4.54 | 0.88 | |
| Bo2Base Model=Qwen 4B2025.09 | 4.48 | 0.69 | |
| Bo2Model=Llama 1B, Training Protocol=full finetuning2025.09 | 4.07 | 51 | |
| AADModel=Llama 1B, Training Protocol=full finetuning2025.09 | 4.04 | — | |
| Greedy SFTBase Model=Llama 8B2025.09 | 3.89 | 0.93 | |
| EFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | 3.64 | 57 | |
| Greedy SFTModel=Llama 3B, Training Protocol=full finetuning2025.09 | 3.59 | 82 | |
| Greedy SFTBase Model=Llama 3B2025.09 | 3.59 | 0.9 | |
| Greedy DPOBase Model=Qwen 4B2025.09 | 3.56 | 0.79 | |
| AADBase Model=Qwen 0.6B2025.09 | 3.42 | — | |
| Greedy DPOModel=Llama 1B, Training Protocol=full finetuning2025.09 | 3.15 | 63 | |
| EFTBase Model=Qwen 0.6B2025.09 | 3.08 | 0.55 | |
| Greedy SFTBase Model=Qwen 4B2025.09 | 2.63 | 0.88 | |
| Bo2Base Model=Qwen 0.6B2025.09 | 2.42 | 0.68 | |
| Greedy SFTModel=Llama 1B, Training Protocol=full finetuning2025.09 | 2.06 | 72 | |
| Greedy DPOBase Model=Qwen 0.6B2025.09 | 1.74 | 0.76 | |
| Greedy SFTBase Model=Qwen 0.6B2025.09 | 0.83 | 0.83 |