Question Answering on Belebele English
94.56AccuracyLlama 3.1
Evaluation results
| Method | Links | |
|---|---|---|
| Llama 3.1Model=Llama 3.1, Parameters=70B2024.10 | 94.56 | |
| Gemma 2Model=Gemma 2, Parameters=9B2024.10 | 91.78 | |
| Qwen2.5-7Bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 91.2 | |
| Ministral-3-8B-Base-2512setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 89.7 | |
| Llama 3.1Model=Llama 3.1, Parameters=8B2024.10 | 87.67 | |
| Llama-3.1-8Bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 87.3 | |
| Llama-3.1-EstLLM-8B-0525setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 87 | |
| Llama 3Model=Llama 3, Parameters=8B2024.10 | 86.89 | |
| Gemma 1Model=Gemma 1, Parameters=7B2024.10 | 84.67 | |
| Apertus-EstLLM-8B-1125setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 84.3 | |
| Mistral 0.3Model=Mistral 0.3, Parameters=7B2024.10 | 83.89 | |
| Apertus-8B-2509setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 82.7 | |
| EuroLLM-9Bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 77.3 | |
| OLMO 1.7Model=OLMO 1.7, Parameters=7B2024.10 | 77.11 | |
| salamandra-7bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 53.1 | |
| Llama 2Model=Llama 2, Parameters=7B2024.10 | 48.22 | |
| Llammas-basesetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 45 | |
| Llama 1Model=Llama 1, Parameters=7B2024.10 | 41.56 |