LLM Alignment on AlpacaEval
25.24Win RateHighAvg.
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HighAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 25.24 | — | — | |
| FullBackbone=LLaMA-3-8B, % Train=100%, Alignment Method=DPO2025.05 | 18.19 | — | — | |
| HighAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 17.73 | — | — | |
| FullBackbone=LLaMA-3-8B, % Train=100%, Alignment Method=SimPO2025.05 | 17.54 | — | — | |
| RandomBackbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 13.47 | — | — | |
| LowAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 11.55 | — | — | |
| HighVar.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=DPO2025.05 | 8.59 | — | — | |
| LowAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 7.2 | — | — | |
| HighVar.Backbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 6.86 | — | — | |
| RandomBackbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 6.82 | — | — | |
| FullBackbone=Mistral-7B, % Train=100%, Alignment Method=DPO2025.05 | 6.81 | — | — | |
| HighAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=DPO2025.05 | 6.65 | — | — | |
| HighAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 5.79 | — | — | |
| FullBackbone=Mistral-7B, % Train=100%, Alignment Method=SimPO2025.05 | 5.32 | — | — | |
| LowAvg.Backbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 5.2 | — | — | |
| LowAvg.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 5.17 | — | — | |
| HighVar.Backbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 5.03 | — | — | |
| RandomBackbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 4.9 | — | — | |
| RandomBackbone=Mistral-7B, % Train=33%, Alignment Method=SimPO2025.05 | 4.77 | — | — | |
| ZeroshotBackbone=Mistral-7B, % Train=0%, Alignment Method=DPO2025.05 | 4.3 | — | — | |
| ZeroshotBackbone=Mistral-7B, % Train=0%, Alignment Method=SimPO2025.05 | 4.3 | — | — | |
| HighVar.Backbone=LLaMA-3-8B, % Train=33%, Alignment Method=SimPO2025.05 | 4.23 | — | — | |
| ZeroshotBackbone=LLaMA-3-8B, % Train=0%, Alignment Method=DPO2025.05 | 3.95 | — | — | |
| ZeroshotBackbone=LLaMA-3-8B, % Train=0%, Alignment Method=SimPO2025.05 | 3.95 | — | — | |
| AdaBoNLM=Mistral, RM=Mistral, K=5, B=120, d=0.75B2025.05 | — | 94 | — | |
| AdaBoNLM=Mistral, RM=FsfairX, K=5, B=120, d=0.75B2025.05 | — | 92 | — | |
| AdaBoNLM=Mistral, RM=Armo, K=5, B=120, d=0.75B2025.05 | — | 96 | — | |
| AdaBoNLM=Qwen, RM=Mistral, K=5, B=120, d=0.75B2025.05 | — | 100 | — | |
| AdaBoNLM=Qwen, RM=FsfairX, K=5, B=120, d=0.75B2025.05 | — | 98 | — | |
| AdaBoNLM=Qwen, RM=Armo, K=5, B=120, d=0.75B2025.05 | — | 78 | — | |
| AdaBoNLM=Gemma, RM=Mistral, K=5, B=120, d=0.75B2025.05 | — | 76 | — | |
| AdaBoNLM=Gemma, RM=FsfairX, K=5, B=120, d=0.75B2025.05 | — | 92 | — | |
| AdaBoNLM=Gemma, RM=Armo, K=5, B=120, d=0.75B2025.05 | — | 86 | — | |
| AdaBoNLM=Llama, RM=Mistral, K=5, B=120, d=0.75B2025.05 | — | 92 | — | |
| AdaBoNLM=Llama, RM=FsfairX, K=5, B=120, d=0.75B2025.05 | — | 96 | — | |
| AdaBoNLM=Llama, RM=Armo, K=5, B=120, d=0.75B2025.05 | — | 92 | — | |
| Mistral-7B-INSTFine-tuning Data=HelpSteer2 prompts2024.08 | — | — | 10.31 | |
| Qwen2-1.5B-INSTFine-tuning Data=HelpSteer2 prompts2024.08 | — | — | 1.06 | |
| RLHFBase Model=Qwen2-1.5B-INST, Fine-tuning Data=HelpSteer2 prompts2024.08 | — | — | 0.66 | |
| RLHFBase Model=Mistral-7B-INST, Fine-tuning Data=HelpSteer2 prompts2024.08 | — | — | 10.15 | |
| UNABase Model=Qwen2-1.5B-INST, Fine-tuning Data=HelpSteer2 prompts2024.08 | — | — | 1.63 | |
| UNABase Model=Mistral-7B-INST, Fine-tuning Data=HelpSteer2 prompts2024.08 | — | — | 10.54 |