Discriminative language understanding on Estonian Discriminative Benchmarks
78.8Belebele AccuracyApertus-EstLLM-8B-1125
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Apertus-EstLLM-8B-1125evaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 78.8 | 63.6 | 83.4 | 52.3 | 38.9 | 75.2 | 73 | 79 | 68 | 1.8 | |
| Llama-3.1-EstLLM-8B-0525evaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 77.2 | 57 | 87.5 | 61.9 | 44.9 | 74 | 75.2 | 78 | 69.5 | 1.8 | |
| Apertus-8B-2509evaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 76.8 | 60.7 | 78.9 | 47.8 | 32.9 | 71.1 | 67.8 | 73 | 63.6 | 3.9 | |
| EuroLLM-9Bevaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 69.9 | 61.8 | 66.3 | 44 | 37.1 | 69.2 | 71.2 | 69 | 61.1 | 4.4 | |
| Llama-3.1-8Bevaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 67 | 44.7 | 65.8 | 58.7 | 30 | 59.6 | 53.2 | 53 | 54 | 6.8 | |
| Qwen2.5-7Bevaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 66.4 | 45.5 | 65.4 | 45.2 | 29 | 53 | 49.4 | 54 | 51 | 7.8 | |
| salamandra-7bevaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 44.8 | 50.5 | 69.9 | 26.8 | 29.6 | 67.3 | 65.8 | 71 | 53.2 | 6.4 | |
| Llammas-baseevaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 38.7 | 46.2 | 53.8 | 26.9 | 33.6 | 69.7 | 68.6 | 76 | 51.7 | 5.9 | |
| Ministral-3-8B-Base-2512evaluation_mode=zero-shot, decoding=deterministic, scoring=log-likelihood, library=LM Evaluation Harness2026.03 | 26.3 | 52.8 | 64.1 | 58.5 | 31.6 | 62.3 | 56 | 60 | 51.4 | 6.5 |