Instruction Following on MT-Bench (test)
8.33Overall ScoreGPRL
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPRLGroup=Online RL with reward model, RM Size=8B, RM Type=GPM, Iter./Ep.=32026.05 | 8.33 | — | — | — | — | — | — | — | — | — | |
| SPPOGroup=Reward model, RM Size=8B, RM Type=GPM, Iter./Ep.=32026.05 | 8.12 | — | — | — | — | — | — | — | — | — | |
| SimPOGroup=No reward model, RM Size=–, RM Type=–, Iter./Ep.=–2026.05 | 8.03 | — | — | — | — | — | — | — | — | — | |
| GPOGroup=Reward model, RM Size=8B, RM Type=GPM, Iter./Ep.=32026.05 | 8.03 | — | — | — | — | — | — | — | — | — | |
| SPPOGroup=Reward model, RM Size=8B, RM Type=BT, Iter./Ep.=32026.05 | 8.02 | — | — | — | — | — | — | — | — | — | |
| DPOGroup=No reward model, RM Size=–, RM Type=–, Iter./Ep.=–2026.05 | 7.9 | — | — | — | — | — | — | — | — | — | |
| GPOGroup=Reward model, RM Size=8B, RM Type=BT, Iter./Ep.=32026.05 | 7.87 | — | — | — | — | — | — | — | — | — | |
| GRPOGroup=Online RL with reward model, RM Size=8B, RM Type=BT, Iter./Ep.=32026.05 | 7.69 | — | — | — | — | — | — | — | — | — | |
| OWSBackbone=LLaMa2-7B, Judge=GPT-3.5-turbo2024.05 | 6.52 | 8 | 7.65 | 4.95 | 4.15 | 3.25 | 7.45 | 8.25 | 8.45 | — | |
| Full-FTBackbone=LLaMa2-7B, Judge=GPT-3.5-turbo2024.05 | 6.14 | 7.11 | 8.11 | 4.9 | 3.75 | 2.85 | 6.5 | 7.8 | 8.1 | — | |
| LoRA-GABase Model=Llama 2-7b, Rank=1282024.07 | 6.13 | — | — | — | — | — | — | — | — | — | |
| DoRABase Model=Llama 2-7b, Rank=82024.07 | 5.97 | — | — | — | — | — | — | — | — | — | |
| LoRA-GABase Model=Llama 2-7b, Rank=82024.07 | 5.95 | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=LLaMa2-7B, Judge=GPT-3.5-turbo2024.05 | 5.95 | 7.21 | 7.05 | 4.95 | 3.9 | 3.25 | 5.7 | 7.9 | 7.65 | — | |
| LISABackbone=LLaMa2-7B, Judge=GPT-3.5-turbo2024.05 | 5.92 | 6.75 | 7.35 | 4.35 | 3.85 | 3 | 6.85 | 7.74 | 7.47 | — | |
| GaLoreBackbone=LLaMa2-7B, Judge=GPT-3.5-turbo2024.05 | 5.83 | 7.05 | 7.79 | 3.55 | 3.15 | 2.89 | 6.25 | 8.3 | 7.63 | — | |
| LoRA-GABase Model=Llama 2-7b, Rank=322024.07 | 5.79 | — | — | — | — | — | — | — | — | — | |
| LoRA+Base Model=Llama 2-7b, Rank=82024.07 | 5.71 | — | — | — | — | — | — | — | — | — | |
| LoRABase Model=Llama 2-7b, Rank=82024.07 | 5.61 | — | — | — | — | — | — | — | — | — | |
| AdaLoRABase Model=Llama 2-7b, Rank=82024.07 | 5.57 | — | — | — | — | — | — | — | — | — | |
| Full-FTBase Model=Llama 2-7b, Mode=Full Finetune2024.07 | 5.56 | — | — | — | — | — | — | — | — | — | |
| PiSSABase Model=Llama 2-7b, Rank=82024.07 | 5.3 | — | — | — | — | — | — | — | — | — | |
| S2FTBackbone=Mistral-7B2024.12 | 5.27 | 6.95 | 4.4 | 5.5 | 2.7 | 3.55 | 5.95 | 6.35 | 6.75 | — | |
| rsLORABase Model=Llama 2-7b, Rank=82024.07 | 5.25 | — | — | — | — | — | — | — | — | — | |
| S2FTBackbone=LLaMA2-7B2024.12 | 5.2 | 6.75 | 6.6 | 4.15 | 1.65 | 1.85 | 4.75 | 7.45 | 8.38 | — | |
| LISABackbone=LLaMA2-7B2024.12 | 4.94 | 6.55 | 6.9 | 3.45 | 1.6 | 2.16 | 4.5 | 6.75 | 7.65 | — | |
| LISABackbone=Mistral-7B2024.12 | 4.85 | 6.84 | 3.65 | 5.45 | 2.2 | 2.75 | 5.65 | 5.95 | 6.35 | — | |
| Full FTBackbone=LLaMA2-7B2024.12 | 4.75 | 5.55 | 6.45 | 3.6 | 1.75 | 2 | 4.7 | 6.45 | 7.5 | — | |
| Full FTBackbone=Mistral-7B2024.12 | 4.64 | 5.5 | 4.45 | 5.45 | 2.5 | 3.25 | 5.78 | 4.75 | 5.45 | — | |
| GaloreBackbone=LLaMA2-7B2024.12 | 4.63 | 5.6 | 6.4 | 3.2 | 1.25 | 1.95 | 5.05 | 6.57 | 7 | — | |
| LoRABackbone=LLaMA2-7B2024.12 | 4.45 | 6.3 | 5.65 | 4.05 | 1.6 | 1.45 | 4.17 | 6.2 | 6.2 | — | |
| LoRABackbone=Mistral-7B2024.12 | 4.41 | 5.3 | 4.4 | 4.65 | 2.35 | 3.3 | 5.5 | 5.55 | 4.3 | — | |
| GaloreBackbone=Mistral-7B2024.12 | 4.36 | 5.05 | 5.27 | 4.45 | 1.7 | 2.5 | 5.21 | 5.52 | 5.2 | — | |
| VanillaBackbone=Mistral-7B2024.12 | 4.32 | 5.25 | 3.2 | 4.5 | 1.6 | 2.7 | 6.5 | 6.17 | 4.65 | — | |
| VanillaBackbone=LLaMA2-7B2024.12 | 3.29 | 2.75 | 4.4 | 2.8 | 1.55 | 1.8 | 3.2 | 5.25 | 4.6 | — | |
| ALBMBase Model=OpenRLHF-Llama-3-8B-SFT2025.12 | — | — | — | — | — | — | — | — | — | 53.75 | |
| ALBMBase Model=Meta-Llama3.1-8B-Instruct2025.12 | — | — | — | — | — | — | — | — | — | 50.63 | |
| DIRBase Model=OpenRLHF-Llama-3-8B-SFT2025.12 | — | — | — | — | — | — | — | — | — | 56.25 | |
| DIRBase Model=Meta-Llama3.1-8B-Instruct2025.12 | — | — | — | — | — | — | — | — | — | 56.88 | |
| InfoRMBase Model=OpenRLHF-Llama-3-8B-SFT2025.12 | — | — | — | — | — | — | — | — | — | 46.88 | |
| InfoRMBase Model=Meta-Llama3.1-8B-Instruct2025.12 | — | — | — | — | — | — | — | — | — | 51.88 | |
| PoEBase Model=OpenRLHF-Llama-3-8B-SFT2025.12 | — | — | — | — | — | — | — | — | — | 48.75 | |
| PoEBase Model=Meta-Llama3.1-8B-Instruct2025.12 | — | — | — | — | — | — | — | — | — | 51.25 | |
| SkyworkBase Model=OpenRLHF-Llama-3-8B-SFT2025.12 | — | — | — | — | — | — | — | — | — | 49.38 | |
| SkyworkBase Model=Meta-Llama3.1-8B-Instruct2025.12 | — | — | — | — | — | — | — | — | — | 51.25 |