Language Understanding on Multilingual MMLU internal translated version (Accuracy)
85.5AccuracyGPT-4o
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4oshot=5-shot2024.07 | 85.5 | |
| Llama 3 405Bshot=5-shot2024.07 | 83.2 | |
| GPT-4shot=5-shot2024.07 | 80.2 | |
| Llama 3 70Bshot=5-shot2024.07 | 78.2 | |
| Qwen 3 14Bshots=5-Shot2026.01 | 75.4 | |
| Ministral 3 14Bshots=5-Shot2026.01 | 74.2 | |
| Ministral 3 8Bshots=5-Shot2026.01 | 70.6 | |
| Qwen 3 8Bshots=5-Shot2026.01 | 70 | |
| Gemma 3 12Bshots=5-Shot2026.01 | 69 | |
| Qwen 3 4Bshots=5-Shot2026.01 | 67.7 | |
| Ministral 3 3Bshots=5-Shot2026.01 | 65.2 | |
| Mixtral 8×22Bshot=5-shot2024.07 | 64.3 | |
| GPT-3.5 Turboshot=5-shot2024.07 | 58.8 | |
| Llama 3 8Bshot=5-shot2024.07 | 58.6 | |
| Gemma 3 4Bshots=5-Shot2026.01 | 51.6 | |
| Mistral 7Bshot=5-shot2024.07 | 46.8 |