Preference Modeling on Helpfulness & Harmlessness
72.2AccuracyBTPO
Evaluation results
| Method | Links | |
|---|---|---|
| BTPOBase Model=Llama3.1-8B-Instruct2025.10 | 72.2 | |
| BTPOBase Model=Llama3.2-3B-Instruct2025.10 | 70.5 | |
| BTPOBase Model=Qwen2.5-7B-Instruct2025.10 | 70.4 | |
| BTBase Model=Llama3.2-3B-Instruct2025.10 | 69.2 | |
| GRAMBase Model=Qwen2.5-7B-Instruct2025.10 | 68.7 | |
| GRAMBase Model=Llama3.2-3B-Instruct2025.10 | 68.7 | |
| BTPOBase Model=Qwen2.5-3B-Instruct2025.10 | 68.5 | |
| BTBase Model=Qwen2.5-7B-Instruct2025.10 | 67.9 | |
| GRAMBase Model=Qwen2.5-3B-Instruct2025.10 | 67.4 | |
| BTBase Model=Llama3.1-8B-Instruct2025.10 | 67.4 | |
| BTBase Model=Qwen2.5-3B-Instruct2025.10 | 67.1 | |
| GRAMBase Model=Llama3.1-8B-Instruct2025.10 | 67.1 | |
| GRPO (pair)Base Model=Llama3.1-8B-Instruct2025.10 | 65.3 | |
| GRPO (pair)Base Model=Qwen2.5-7B-Instruct2025.10 | 64.7 | |
| GRPO (pair)Base Model=Llama3.2-3B-Instruct2025.10 | 63.2 | |
| GRPO (pair)Base Model=Qwen2.5-3B-Instruct2025.10 | 61 | |
| GRPO (point)Base Model=Qwen2.5-7B-Instruct2025.10 | 59.3 | |
| GRPO (point)Base Model=Llama3.1-8B-Instruct2025.10 | 58.7 | |
| GRPO (point)Base Model=Llama3.2-3B-Instruct2025.10 | 55.8 | |
| GRPO (point)Base Model=Qwen2.5-3B-Instruct2025.10 | 54.9 |