Multiple-choice Question Answering on m-MMLU Non-English
63.84AccuracyLlama 3.1
Evaluation results
| Method | Links | |
|---|---|---|
| Llama 3.1Model=Llama 3.1, Parameters=70B2024.10 | 63.84 | |
| Gemma 2Model=Gemma 2, Parameters=9B2024.10 | 55.82 | |
| Gemma 1Model=Gemma 1, Parameters=7B2024.10 | 47.34 | |
| Llama 3.1Model=Llama 3.1, Parameters=8B2024.10 | 47.2 | |
| Llama 3Model=Llama 3, Parameters=8B2024.10 | 46.64 | |
| Mistral 0.3Model=Mistral 0.3, Parameters=7B2024.10 | 42.07 | |
| OLMO 1.7Model=OLMO 1.7, Parameters=7B2024.10 | 33.9 | |
| Llama 2Model=Llama 2, Parameters=7B2024.10 | 32.6 | |
| Llama 1Model=Llama 1, Parameters=7B2024.10 | 28.07 |