Reading Comprehension on Belebele (34 Languages)
84.7AccuracyTildeOpen LLM
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TildeOpen LLMshot_count=5-shot2026.03 | 84.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EuroLLMshot_count=5-shot2026.03 | 82.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma 2shot_count=5-shot2026.03 | 79.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALIAshot_count=5-shot2026.03 | 76.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NeuronMoE-S1 (BN)Language=English2026.03 | 75.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DenseLanguage=English2026.03 | 74.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NeuronMoE-S1 (NE)Language=English2026.03 | 71.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BYOL-nyaParameters=12B, Pre-training Type=CPT2026.01 | 61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3Parameters=12B, Pre-training Type=PT2026.01 | 59.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NeuronMoE-S1 (BN)Language=Bengali2026.03 | 46.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BYOL-nyaParameters=4B, Pre-training Type=CPT2026.01 | 45.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DenseLanguage=Bengali2026.03 | 44.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NeuronMoE-S1 (NE)Language=Nepali2026.03 | 44.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DenseLanguage=Nepali2026.03 | 42.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ApertusParameters=8B, Pre-training Type=25092026.01 | 38.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3Parameters=4B, Pre-training Type=PT2026.01 | 38.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SmolLM2-1.7b-Instructtuning=Instruction-tuned, prompt_language=English2025.12 | 36.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3-1b-ittuning=Instruction-tuned, prompt_language=English2025.12 | 36.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen-3Parameters=14B, Pre-training Type=Base2026.01 | 36.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen-3Parameters=8B, Pre-training Type=Base2026.01 | 32.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SmolLM2-1.7bprompting=Zero-shot, data_tokens=11T, gpu_hours=not reported2025.12 | 32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen-3Parameters=1.7B, Pre-training Type=Base2026.01 | 29.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-3.1Parameters=8B2026.01 | 29.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XGLMNumber of Parameters=564M, Evaluation Protocol=Zero-shot2024.08 | 29 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BLOOMNumber of Parameters=560M, Evaluation Protocol=Zero-shot2024.08 | 28.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA 3Number of Parameters=1B, Evaluation Protocol=Zero-shot2024.08 | 28.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GoldfishNumber of Parameters=124M, Evaluation Protocol=Zero-shot2024.08 | 28.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XGLMNumber of Parameters=1.7B, Evaluation Protocol=Zero-shot2024.08 | 28.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3Parameters=1B, Pre-training Type=PT2026.01 | 28.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-3.2Parameters=1B2026.01 | 27.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MiniLingua-1b-Instructtuning=Instruction-tuned, prompt_language=English2025.12 | 26.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BYOL-nyaParameters=1B, Pre-training Type=CPT2026.01 | 26 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EuroLLM-1.7bprompting=Zero-shot, data_tokens=4T, gpu_hours=500K2025.12 | 25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Chance2024.08 | 25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MiniLingua-1bprompting=Zero-shot, data_tokens=1.5T, gpu_hours=75K2025.12 | 23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma 2Number of Parameters=270M, Evaluation Protocol=Zero-shot2024.08 | 22.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma 2Number of Parameters=1B, Evaluation Protocol=Zero-shot2024.08 | 22.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3-1b-ptprompting=Zero-shot, data_tokens=2T, gpu_hours=∼300K2025.12 | 22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EuroLLM-1.7b-Instructtuning=Instruction-tuned, prompt_language=English2025.12 | 21.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BYOL-mriParameters=4B, Training Stage=CPT2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 47.78 | |
| BYOL-mriParameters=12B, Training Stage=CPT2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 63.56 | |
| Gemma-3Parameters=12B, Training Stage=PT2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 59.11 | |
| Granite 4 MicroVariant=Base2026.01 | — | 30 | 51.4 | 64.5 | 45.9 | 35.1 | 54.2 | 33.3 | 42.8 | 33.7 | 32.7 | 50.8 | 31.7 | 47.4 | 30.9 | 27.6 | 41.8 | 31.5 | 38.1 | 59.3 | 29.2 | 38.9 | 35.9 | 41.6 | 38.7 | 37 | 30.8 | 44.4 | 45.2 | 45.6 | 61 | 27.6 | 30.3 | 29.8 | 30.4 | — | |
| KakugoVariant=GenReas+Tran (full)2026.01 | — | 36.8 | 58.6 | 65.9 | 59.2 | 45.4 | 68 | 40.4 | 61.4 | 42.3 | 39.9 | 65.1 | 48.7 | 56.9 | 36.7 | 27.6 | 61.5 | 35 | 47 | 67.3 | 37.3 | 56.1 | 49.1 | 55.7 | 52.4 | 52.2 | 38.1 | 59.5 | 54.6 | 59.2 | 60.8 | 31.2 | 41.2 | 34.3 | 41.5 | — | |
| Qwen-3Parameters=8B, Training Stage=Base2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 43.33 |