Language Understanding on MMLU (0-shot and 5-shot)
67.6MMLU Accuracy (0-shot)Yi-Chat
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Yi-ChatSize=34B2024.03 | 67.6 | 73.5 | |
| Yi-Chat-8bits(GPTQ)Size=34B2024.03 | 66.2 | 73.7 | |
| Yi-Chat-4bits(AWQ)Size=34B2024.03 | 65.8 | 72.4 | |
| AquilaChat2Size=34B2024.03 | 65.2 | 66.7 | |
| Qwen-ChatSize=14B2024.03 | 64 | 65 | |
| LLaMA2-ChatSize=70B2024.03 | 59.4 | 59.9 | |
| Yi-Chat-8bits(GPTQ)Size=6B2024.03 | 58.3 | 61 | |
| Yi-ChatSize=6B2024.03 | 58.2 | 61 | |
| Yi-Chat-4bits(AWQ)Size=6B2024.03 | 56.8 | 59.9 | |
| InternLM-ChatSize=20B2024.03 | 55.6 | 57.4 | |
| Baichuan2-ChatSize=13B2024.03 | 55.1 | 50.1 | |
| LLaMA2-ChatSize=13B2024.03 | 50.9 | 47.3 |