Alignment on MT-Bench
9.12MT-Bench ScoreQwen2-72B-Instruct
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2-72B-InstructType=Instruction-tuned2024.07 | 9.12 | — | — | — | |
| Llama-3-70B-InstructType=Instruction-tuned2024.07 | 8.95 | — | — | — | |
| Qwen1.5-110B-ChatType=Instruction-tuned2024.07 | 8.88 | — | — | — | |
| Mixtral-8x22B-InstructType=Instruction-tuned2024.07 | 8.66 | — | — | — | |
| Qwen1.5-72B-ChatType=Instruction-tuned2024.07 | 8.61 | — | — | — | |
| FsfairX + LPRM=+ LP, Backbone=Gemma-2-9B-it2026.04 | 8.16 | — | — | — | |
| FsfairX + CIRMRM=+ CIRM, Backbone=Gemma-2-9B-it2026.04 | 8.15 | — | — | — | |
| INFRM=INF, Backbone=Gemma-2-9B-it2026.04 | 8.11 | — | — | — | |
| FsfairXRM=FsfairX, Backbone=Gemma-2-9B-it2026.04 | 8.07 | — | — | — | |
| GRMRM=GRM, Backbone=Gemma-2-9B-it2026.04 | 8.01 | — | — | — | |
| FsfairX + LWRRM=+ LWR, Backbone=Gemma-2-9B-it2026.04 | 7.96 | — | — | — | |
| GRM + CIRMRM=+ CIRM, Backbone=Gemma-2-9B-it2026.04 | 7.89 | — | — | — | |
| GRM + LWRRM=+ LWR, Backbone=Gemma-2-9B-it2026.04 | 7.87 | — | — | — | |
| GRM + LPRM=+ LP, Backbone=Gemma-2-9B-it2026.04 | 7.84 | — | — | — | |
| Gemma-2-9B-itRM=-, Backbone=Gemma-2-9B-it2026.04 | 7.83 | — | — | — | |
| FsfairXRM=FsfairX, Backbone=Llama-3-8B-Instruct2026.04 | 7.64 | — | — | — | |
| FsfairX + CIRMRM=+ CIRM, Backbone=Llama-3-8B-Instruct2026.04 | 7.62 | — | — | — | |
| FsfairX + LPRM=+ LP, Backbone=Llama-3-8B-Instruct2026.04 | 7.6 | — | — | — | |
| GRM + LWRRM=+ LWR, Backbone=Llama-3-8B-Instruct2026.04 | 7.58 | — | — | — | |
| GRM + CIRMRM=+ CIRM, Backbone=Llama-3-8B-Instruct2026.04 | 7.53 | — | — | — | |
| GRMRM=GRM, Backbone=Llama-3-8B-Instruct2026.04 | 7.45 | — | — | — | |
| FsfairX + LWRRM=+ LWR, Backbone=Llama-3-8B-Instruct2026.04 | 7.44 | — | — | — | |
| INFRM=INF, Backbone=Llama-3-8B-Instruct2026.04 | 7.42 | — | — | — | |
| Llama-3-8B-InstructRM=-, Backbone=Llama-3-8B-Instruct2026.04 | 7.34 | — | — | — | |
| GRM + LPRM=+ LP, Backbone=Llama-3-8B-Instruct2026.04 | 7.29 | — | — | — | |
| FullBackbone=LLaMA-3-8B, % Train=100%, Alignment Method=DPO2025.05 | 5.51 | — | — | 49.4 | |
| FullBackbone=LLaMA-3-8B, % Train=100%, Alignment Method=SimPO2025.05 | 5.42 | — | — | 47.5 | |
| RandomBackbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 5.28 | — | — | 47.5 | |
| HighVar.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 5.21 | — | — | 43.1 | |
| HighAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 5.19 | — | — | 48.1 | |
| LowAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 5.15 | — | — | 48.8 | |
| FullBackbone=Mistral-7B, % Train=100%, Alignment Method=DPO2025.05 | 5.15 | — | — | 49.7 | |
| HighAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 5.13 | — | — | 50.9 | |
| LowAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 5.01 | — | — | 45 | |
| RandomBackbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 4.98 | — | — | 45 | |
| HighAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 4.96 | — | — | 45.6 | |
| HighVar.Backbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 4.8 | — | — | 38.4 | |
| ZeroshotBackbone=LLaMA-3-8B, % Train=0%, Alignment Method=DPO2025.05 | 4.61 | — | — | 27.8 | |
| ZeroshotBackbone=LLaMA-3-8B, % Train=0%, Alignment Method=SimPO2025.05 | 4.61 | — | — | 27.8 | |
| LowAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 4.61 | — | — | 33.1 | |
| RandomBackbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 4.54 | — | — | 33.1 | |
| HighAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 4.51 | — | — | 34.1 | |
| HighVar.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 4.49 | — | — | 30 | |
| FullBackbone=Mistral-7B, % Train=100%, Alignment Method=SimPO2025.05 | 4.46 | — | — | 34.7 | |
| HighVar.Backbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 4.4 | — | — | 23.1 | |
| ZeroshotBackbone=Mistral-7B, % Train=0%, Alignment Method=DPO2025.05 | 4.04 | — | — | 21.6 | |
| ZeroshotBackbone=Mistral-7B, % Train=0%, Alignment Method=SimPO2025.05 | 4.04 | — | — | 21.6 | |
| LowAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 3.88 | — | — | 19.4 | |
| RandomBackbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 3.84 | — | — | 20.3 | |
| PSFTBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.08 | — | 6.83 | 4.86 | — | |
| PSFT → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.08 | — | 8.51 | 6.95 | — | |
| PSFTprolongBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.08 | — | 7.41 | 5.84 | — | |
| PSFTprolong → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.08 | — | 7.63 | 6.74 | — | |
| SFTBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback2025.08 | — | 7.64 | 5.71 | — | |
| SFT → DPOBackbone=Qwen3-4B-Base, Training Dataset=UltraFeedback, Post-training=DPO2025.08 | — | 7.91 | 6 | — |