Instruction Tuning Evaluation on Open Instruct Evaluation Suite (test)
61.2MMLUSelective Alpaca
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Selective AlpacaBase Model=LLAMA-3-8B2024.02 | 61.2 | 55 | 37.5 | 41.1 | 65.4 | 47.7 | 51.3 | 1.6 | |
| Alpaca-GPT4Base Model=LLAMA-3-8B2024.02 | 59.6 | 52.3 | 34.5 | 43.1 | 60.2 | 48.2 | 49.7 | — | |
| Selective AlpacaBase Model=Mistral-7B2024.02 | 56.9 | 53.7 | 36 | 49.3 | 55.3 | 44.3 | 49.3 | 1.5 | |
| Alpaca-GPT4Base Model=LLAMA-2-13B2024.02 | 55.7 | 46.6 | 30.5 | 47.1 | 38.8 | 46.5 | 44.2 | — | |
| Selective AlpacaBase Model=LLAMA-2-13B2024.02 | 55.3 | 48.5 | 32.5 | 54.1 | 41.2 | 47.8 | 46.6 | 2.4 | |
| Alpaca-GPT4Base Model=Mistral-7B2024.02 | 52.5 | 51.7 | 33.5 | 51.1 | 54.7 | 43.1 | 47.8 | — | |
| Selective AlpacaBase Model=LLAMA-2-7B2024.02 | 47.4 | 40.6 | 16.8 | 47.4 | 29.4 | 35.7 | 36.2 | 2.1 | |
| Alpaca-GPT4Base Model=LLAMA-2-7B2024.02 | 46.5 | 38.4 | 15 | 43.4 | 26.8 | 34.2 | 34.1 | — | |
| WizardLM + SelectITBase Model=LLaMA-2-7B, Data Size=28.6K, Instruction Dataset=WizardLM2024.02 | 45.1 | 40.1 | 11 | 43.1 | 27.5 | 34.7 | 33.6 | 1.4 | |
| Orca-GPT4 + SelectITBase Model=LLaMA-2-7B, Data Size=0.2M, Instruction Dataset=Orca-GPT42024.02 | 43.9 | 38.7 | 16.5 | 42 | 27.7 | 37.4 | 34.4 | 1.7 | |
| WizardLMBase Model=LLaMA-2-7B, Data Size=143K, Instruction Dataset=WizardLM2024.02 | 43.8 | 37.8 | 10 | 41.2 | 25.2 | 35.3 | 32.2 | — | |
| Orca-GPT4Base Model=LLaMA-2-7B, Data Size=1M, Instruction Dataset=Orca-GPT42024.02 | 40.1 | 35.6 | 13 | 46 | 23.3 | 38.1 | 32.7 | — |