Instruction Following on AlpacaFarm Eval (test)
76.13Win RateBest-of-N
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Best-of-NN=3840, Base Model=Mistral-7B2024.10 | 76.13 | 77.21 | |
| Best-of-NN=1920, Base Model=Mistral-7B2024.10 | 75.2 | 76.27 | |
| SPECULATIVE REJECTIONalpha=0.5, Base Model=Llama-3-8B2024.10 | 73.6 | 77.91 | |
| SPECULATIVE REJECTIONalpha=0.5, Base Model=Mistral-7B2024.10 | 69.42 | 73.31 | |
| Best-of-NN=960, Base Model=Mistral-7B2024.10 | 67.5 | 68.07 | |
| SPECULATIVE REJECTIONalpha=0.5, Base Model=Average2024.10 | 66.17 | 70.01 | |
| Best-of-NN=3840, Base Model=Average2024.10 | 62.89 | 63.04 | |
| Best-of-NN=1920, Base Model=Average2024.10 | 62.54 | 63.48 | |
| Best-of-NN=480, Base Model=Mistral-7B2024.10 | 61.28 | 61.84 | |
| Best-of-NN=240, Base Model=Mistral-7B2024.10 | 60.69 | 60.07 | |
| Best-of-NN=1920, Base Model=Llama-3-8B2024.10 | 60.57 | 61.05 | |
| Best-of-NN=960, Base Model=Llama-3-8B2024.10 | 59.2 | 60.26 | |
| Best-of-NN=3840, Base Model=Llama-3-8B2024.10 | 59.19 | 57.91 | |
| Best-of-NN=960, Base Model=Average2024.10 | 59.03 | 59.99 | |
| Best-of-NN=480, Base Model=Llama-3-8B2024.10 | 58.9 | 59.93 | |
| Best-of-NN=480, Base Model=Average2024.10 | 56.89 | 58.29 | |
| SPECULATIVE REJECTIONalpha=0.5, Base Model=Llama-3-8B-Instruct2024.10 | 55.5 | 58.8 | |
| Best-of-NN=240, Base Model=Average2024.10 | 53.69 | 54.41 | |
| Best-of-NN=3840, Base Model=Llama-3-8B-Instruct2024.10 | 53.36 | 54.01 | |
| Best-of-NN=1920, Base Model=Llama-3-8B-Instruct2024.10 | 51.86 | 53.13 | |
| Best-of-NN=480, Base Model=Llama-3-8B-Instruct2024.10 | 50.49 | 53.11 | |
| Best-of-NN=240, Base Model=Llama-3-8B2024.10 | 50.45 | 50.27 | |
| Best-of-NN=960, Base Model=Llama-3-8B-Instruct2024.10 | 50.39 | 51.64 | |
| Best-of-NN=120, Base Model=Mistral-7B2024.10 | 50 | 50 | |
| Best-of-NN=120, Base Model=Llama-3-8B2024.10 | 50 | 50 | |
| Best-of-NN=120, Base Model=Llama-3-8B-Instruct2024.10 | 50 | 50 | |
| Best-of-NN=120, Base Model=Average2024.10 | 50 | 50 | |
| Best-of-NN=240, Base Model=Llama-3-8B-Instruct2024.10 | 49.92 | 52.89 |