Reading Comprehension on BELEBELE (Korean, Bengali, Swahili, Telugu)
80Average RC Score (BELEBELE)Trinity Large (MoE)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Trinity Large (MoE)Architecture=MoE, Protocol=Zero-shot2026.02 | 80 | — | — | — | — | |
| Qwen3 8BParameters=8B, Protocol=Zero-shot2026.02 | 77 | — | — | — | — | |
| Qwen3 4BParameters=4B, Protocol=Zero-shot2026.02 | 71 | — | — | — | — | |
| Module-wise SwappingBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.6 | 86.7 | 75.3 | 53.1 | 67.4 | |
| Layer-wise SwappingBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.5 | 85.9 | 75.2 | 53.3 | 68.4 | |
| Task ArithmeticBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.4 | 85.7 | 75.2 | 52.6 | 68 | |
| Ties MergingBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.4 | 86.1 | 75 | 52.4 | 68 | |
| DAREBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.4 | 86 | 74.9 | 52.7 | 67.8 | |
| Safety-OnlyBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.3 | 85.9 | 74.6 | 53 | 67.6 | |
| baseBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.2 | 86 | 74.6 | 52.3 | 67.7 | |
| Lang-OnlyBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.1 | 86.3 | 74.1 | 54.2 | 65.7 | |
| Layer SwappingBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70.1 | 86 | 74.4 | 52.1 | 67.7 | |
| Mixed SFTBackbone=Qwen 2 7B, Evaluation Protocol=0-shot2026.01 | 70 | 85.7 | 74.3 | 52.8 | 67.1 | |
| Lang-OnlyBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 56.3 | 69.3 | 57.8 | 48 | 50.2 | |
| Layer SwappingBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 56.3 | 66.3 | 57.7 | 49.9 | 51.2 | |
| Layer-wise SwappingBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 56.2 | 66.1 | 59.1 | 50.6 | 48.9 | |
| DAREBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 56.1 | 69.2 | 60 | 46 | 49.2 | |
| Module-wise SwappingBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 55.8 | 67 | 58.9 | 47 | 50.2 | |
| Task ArithmeticBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 55.7 | 70 | 57.4 | 45.9 | 49.4 | |
| Ties MergingBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 55.6 | 69.6 | 57.4 | 48.1 | 47.4 | |
| Mixed SFTBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 55.5 | 68.4 | 56.3 | 46.8 | 50.4 | |
| Sarvam1 2BParameters=2B, Protocol=Zero-shot2026.02 | 55 | — | — | — | — | |
| Safety-OnlyBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 54.2 | 66.5 | 56.8 | 46.7 | 46.7 | |
| Llama-3.1 8BParameters=8B, Protocol=Zero-shot2026.02 | 53 | — | — | — | — | |
| Datology 8BParameters=8B, Protocol=Zero-shot2026.02 | 51 | — | — | — | — | |
| Granite-4.0 MicroParameters=Micro, Protocol=Zero-shot2026.02 | 51 | — | — | — | — | |
| baseBackbone=LLaMA 3.1 8B it, Evaluation Protocol=0-shot2026.01 | 50.1 | 68.1 | 55.8 | 49.3 | 47.2 | |
| Datology 3BParameters=3B, Protocol=Zero-shot2026.02 | 43 | — | — | — | — | |
| Llama-3.2 3BParameters=3B, Protocol=Zero-shot2026.02 | 39 | — | — | — | — | |
| SmolLM3 3BParameters=3B, Protocol=Zero-shot2026.02 | 30 | — | — | — | — | |
| Llama-3.2 1BParameters=1B, Protocol=Zero-shot2026.02 | 25 | — | — | — | — |