Classification on MMLU (test)
77.4Accuracy (5-shot)OpenAI code-davinci-002
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| OpenAI code-davinci-0022022.12 | 77.4 | — | — | |
| OpenAI text-davinci-0032022.12 | 74.2 | — | — | |
| FLAN-PaLM 540B2022.12 | 73.5 | — | — | |
| PaLM 540B2022.12 | 71.3 | — | — | |
| OpenAI text-davinci-0022022.12 | 64.5 | — | — | |
| FLAN-PaLM 62B2022.12 | 59.6 | — | — | |
| PaLM 62B2022.12 | 55.1 | — | — | |
| FLAN-T5 11B2022.12 | 54.9 | 53.7 | — | |
| OPT-IML-Max 175B2022.12 | 47.1 | 49.1 | — | |
| OPT-IML-Max 30B2022.12 | 43.2 | 46.3 | — | |
| OPT 175B2022.12 | 34.2 | 27.3 | — | |
| T0pp 11B2022.12 | 33.7 | 46.7 | — | |
| OpenAI davinci2022.12 | 32.3 | — | — | |
| OPT-IML-Max 1.3B2022.12 | 29.5 | 34.9 | — | |
| OPT 30B2022.12 | 26.1 | 24.2 | — | |
| OPT 1.3B2022.12 | 25.9 | 23.5 | — | |
| T5 11B2022.12 | 25.9 | — | — | |
| Baichuan2-13B-ChatParameter size group=13~20B Models, Evaluation protocol=5-shot2024.03 | — | — | 56.6 | |
| Baichuan2-7B-ChatParameter size group=< 7B Models, Evaluation protocol=5-shot2024.03 | — | — | 50.1 | |
| ChatGLM3-6BParameter size group=< 7B Models, Evaluation protocol=5-shot2024.03 | — | — | 58 | |
| GPT-3.5Model type=API Models, Evaluation protocol=5-shot2024.03 | — | — | 69.1 | |
| InternLM2-Chat-20BParameter size group=13~20B Models, Evaluation protocol=5-shot2024.03 | — | — | 66.5 | |
| InternLM2-Chat-20B-SFTParameter size group=13~20B Models, Evaluation protocol=5-shot, Alignment=SFT2024.03 | — | — | 66.5 | |
| InternLM2-Chat-7BParameter size group=< 7B Models, Evaluation protocol=5-shot2024.03 | — | — | 63.7 | |
| InternLM2-Chat-7B-SFTParameter size group=< 7B Models, Evaluation protocol=5-shot, Alignment=SFT2024.03 | — | — | 63.8 | |
| Llama2-13B-ChatParameter size group=13~20B Models, Evaluation protocol=5-shot2024.03 | — | — | 54.1 | |
| Llama2-7B-ChatParameter size group=< 7B Models, Evaluation protocol=5-shot2024.03 | — | — | 48.2 | |
| Mistral-7B-Instruct-v0.2Parameter size group=< 7B Models, Evaluation protocol=5-shot, Alignment=Instruct2024.03 | — | — | 59.2 | |
| Mixtral-8x7B-Instruct-v0.1Parameter size group=13~20B Models, Evaluation protocol=5-shot, Alignment=Instruct2024.03 | — | — | 70.3 | |
| Qwen-14B-ChatParameter size group=13~20B Models, Evaluation protocol=5-shot2024.03 | — | — | 66.7 | |
| Qwen-7B-ChatParameter size group=< 7B Models, Evaluation protocol=5-shot2024.03 | — | — | 57.1 |