Multi-task Language Understanding on MMLU-R English
75AccuracyQwen2.5-7B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen2.5-7Bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 75 | |
| Ministral-3-8B-Base-2512setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 72.9 | |
| Llama-3.1-8Bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 64.9 | |
| Llama-3.1-EstLLM-8B-0525setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 62.7 | |
| Apertus-EstLLM-8B-1125setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 62.5 | |
| Apertus-8B-2509setting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 59.8 | |
| EuroLLM-9Bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 55.7 | |
| salamandra-7bsetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 44.9 | |
| Llammas-basesetting=zero-shot, decoding=deterministic, scoring=log-likelihood, harness=LM Evaluation Harness2026.03 | 35 |