Chat on AlpacaEval 2.0 (test)
57.46AlpacaEval (LC win %)GPT-4o
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4oAlignment=RLHF2024.08 | 57.46 | 51.33 | — | — | |
| Pref. Distill.Backbone=LLaMA-3-8B-Instruct, Training Dataset=UltraFeedback2025.06 | 54.49 | 46.86 | — | — | |
| PMLEBackbone=LLaMA-3-8B-Instruct, Training Dataset=UltraFeedback2025.06 | 53.23 | 50.43 | — | — | |
| REBELBackbone=LLaMA-3-8B-Instruct, Training Dataset=UltraFeedback2025.06 | 52.04 | 44.99 | — | — | |
| DPOBackbone=LLaMA-3-8B-Instruct, Training Dataset=UltraFeedback2025.06 | 44.72 | 43.94 | — | — | |
| MTModel=Qwen2.5-7B, Role=Teacher2026.04 | 36.04 | 34.95 | — | — | |
| GRPO + R2MBackbone=LLaMA3-8B-Instruct2026.01 | 35.6 | 39.4 | 2,011 | — | |
| RLOO + R2MBackbone=LLaMA3-8B-Instruct2026.01 | 34.5 | 38.2 | 2,011 | — | |
| GRPO + Iterative RMHeadBackbone=LLaMA3-8B-Instruct2026.01 | 32 | 33.9 | 2,250 | — | |
| GRPO + Pretrained RMBackbone=LLaMA3-8B-Instruct2026.01 | 31.5 | 34.3 | 2,278 | — | |
| RLOO + Iterative RMHeadBackbone=LLaMA3-8B-Instruct2026.01 | 31 | 31.8 | 2,150 | — | |
| RLOO + Pretrained RMBackbone=LLaMA3-8B-Instruct2026.01 | 30.5 | 32.2 | 2,172 | — | |
| Base (LLaMA-3-8B-Instruct)Model Type=Instruct-only Base2025.06 | 29.62 | 28.71 | — | — | |
| Mamba-Llama3Attention %=50%, Size=8B, Alignment=DPO2024.08 | 29.61 | 26.69 | — | — | |
| GRPOBackbone=LLaMA3-8B-Instruct2026.01 | 29.5 | 32.6 | 2,216 | — | |
| REINFORCE++Backbone=LLaMA3-8B-Instruct2026.01 | 29.3 | 31.8 | 2,192 | — | |
| ReMaxBackbone=LLaMA3-8B-Instruct2026.01 | 28.7 | 30.7 | 2,289 | — | |
| RLOOBackbone=LLaMA3-8B-Instruct2026.01 | 28.4 | 30.2 | 2,186 | — | |
| Mamba2-Llama3Attention %=50%, Size=8B, Alignment=DPO2024.08 | 26.78 | 22.69 | — | — | |
| Mamba-Llama3Attention %=25%, Size=8B, Alignment=DPO2024.08 | 25.85 | 22.5 | — | — | |
| GRPO + R2MBackbone=Qwen2.5-3B-Instruct2026.01 | 25.8 | 30.9 | 2,871 | — | |
| GRPO + R2M w/o TrainBackbone=LLaMA3-8B-Instruct2026.01 | 25.6 | 27 | 2,261 | — | |
| RLOO + R2MBackbone=Qwen2.5-3B-Instruct2026.01 | 24.8 | 31.2 | 2,911 | — | |
| RLOO + R2M w/o TrainBackbone=LLaMA3-8B-Instruct2026.01 | 24.4 | 27.4 | 2,366 | — | |
| GRPO + Iterative RMHeadBackbone=Qwen2.5-3B-Instruct2026.01 | 23.5 | 27.8 | 3,050 | — | |
| RLOO + Iterative RMHeadBackbone=Qwen2.5-3B-Instruct2026.01 | 23.2 | 27 | 2,950 | — | |
| Llama-3-InstructSize=8B, Alignment=RLHF2024.08 | 22.9 | 22.6 | — | — | |
| GRPO + Pretrained RMBackbone=Qwen2.5-3B-Instruct2026.01 | 22.9 | 28.2 | 3,101 | — | |
| SFTBackbone=LLaMA3-8B-Instruct2026.01 | 22.9 | 22.6 | 1,899 | — | |
| RLOO + Pretrained RMBackbone=Qwen2.5-3B-Instruct2026.01 | 22.8 | 27.4 | 2,992 | — | |
| Mamba2-Llama3Attention %=25%, Size=8B, Alignment=DPO2024.08 | 22.75 | 19.01 | — | — | |
| GPT-3.5-turboAlignment=RLHF2024.08 | 22.7 | 14.1 | — | — | |
| GRPOBackbone=Qwen2.5-3B-Instruct2026.01 | 22.7 | 25.6 | 3,012 | — | |
| RLOOBackbone=Qwen2.5-3B-Instruct2026.01 | 21.9 | 26 | 3,174 | — | |
| ReMaxBackbone=Qwen2.5-3B-Instruct2026.01 | 21.8 | 25.1 | 2,916 | — | |
| REINFORCE++Backbone=Qwen2.5-3B-Instruct2026.01 | 21.4 | 26.4 | 3,252 | — | |
| Llama-3.1-InstructSize=8B, Alignment=RLHF2024.08 | 20.9 | 21.8 | — | — | |
| Mamba-Llama3Attention %=12.5%, Size=8B, Alignment=DPO2024.08 | 20.76 | 17.93 | — | — | |
| Mamba-ZephyrAttention %=50%, Size=7B, Alignment=DPO2024.08 | 20.66 | 16.69 | — | — | |
| Mamba2-Llama3Attention %=12.5%, Size=8B, Alignment=DPO2024.08 | 20.25 | 16.88 | — | — | |
| Mamba2-Llama3.1Attention %=50%, Size=8B2024.08 | 18.99 | 21.55 | — | — | |
| Mamba-Llama3.1Attention %=50%, Size=8B2024.08 | 18.97 | 21.22 | — | — | |
| GRPO + R2M w/o TrainBackbone=Qwen2.5-3B-Instruct2026.01 | 17.4 | 19.4 | 3,317 | — | |
| Mamba-ZephyrAttention %=25%, Size=7B, Alignment=DPO2024.08 | 17.16 | 13.11 | — | — | |
| RLOO + R2M w/o TrainBackbone=Qwen2.5-3B-Instruct2026.01 | 15.8 | 20.5 | 3,154 | — | |
| SFTBackbone=Qwen2.5-3B-Instruct2026.01 | 15.5 | 15.8 | 2,218 | — | |
| Mamba-ZephyrAttention %=12.5%, Size=7B, Alignment=DPO2024.08 | 15.32 | 12.96 | — | — | |
| Mamba2-Llama3Attention %=0%, Size=8B, Alignment=DPO2024.08 | 14.49 | 10.88 | — | — | |
| HPDModel=Qwen2.5-1.5B, Method=Hybrid Policy Distillation2026.04 | 13.75 | 14.25 | — | — | |
| Mamba-Llama3.2Attention %=50%, Size=3B2024.08 | 13.57 | 15.54 | — | — | |
| JSDModel=Qwen2.5-1.5B, Method=Jensen-Shannon Divergence2026.04 | 13.48 | 13.89 | — | — | |
| ZephyrSize=7B, Alignment=DPO2024.08 | 13.2 | 10.99 | — | — | |
| SFTModel=Qwen2.5-1.5B, Training=Supervised Fine-Tuning2026.04 | 12.74 | 13.72 | — | — | |
| Mamba2-Llama3.2Attention %=50%, Size=3B2024.08 | 12.61 | 14.34 | — | — | |
| RKLDModel=Qwen2.5-1.5B, Method=Reverse KL Distillation2026.04 | 11.26 | 12 | — | — | |
| MSModel=Qwen2.5-1.5B, Role=Student2026.04 | 8.67 | 7.47 | — | — | |
| SeqKDModel=Qwen2.5-1.5B, Method=Sequence-level Knowledge Distillation2026.04 | 7.83 | 9.51 | — | — | |
| Falcon Mamba InstructSize=7B, Alignment=SFT2024.08 | 4.04 | 2.15 | — | — | |
| AdaLoRAModel Scale=7B2025.12 | — | — | — | 24.6 | |
| AdaLoRAModel Scale=13B2025.12 | — | — | — | 32.1 | |
| FFTModel Scale=7B2025.12 | — | — | — | 10.4 | |
| FFTModel Scale=13B2025.12 | — | — | — | 14.5 | |
| GMTModel Scale=7B2025.12 | — | — | — | 12.5 | |
| GMTModel Scale=13B2025.12 | — | — | — | 19.3 | |
| HFTModel Scale=7B2025.12 | — | — | — | 17.6 | |
| HFTModel Scale=13B2025.12 | — | — | — | 25.8 | |
| SourceModel Scale=7B2025.12 | — | — | — | 32.6 | |
| SourceModel Scale=13B2025.12 | — | — | — | 37.2 | |
| SSU-MagModel Scale=7B2025.12 | — | — | — | 14.9 | |
| SSU-MagModel Scale=13B2025.12 | — | — | — | 20.6 | |
| SSU-RandModel Scale=7B2025.12 | — | — | — | 18 | |
| SSU-RandModel Scale=13B2025.12 | — | — | — | 24.7 | |
| SSU-WandaModel Scale=7B2025.12 | — | — | — | 27 | |
| SSU-WandaModel Scale=13B2025.12 | — | — | — | 33.4 |