General Knowledge on Global MMLU Ukrainian (test)
67.03Accuracy (%)Gemma 3 12B PT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemma 3 12B PT2025.10 | 67.03 | 78.18 | |
| MATTTraining Time=7h 00m2025.10 | 64.98 | 77.27 | |
| FOCUS w/ NTPTraining Time=10h 30m2025.10 | 60.57 | 74.01 | |
| FOCUS w/ NTPTraining Time=7h 00m2025.10 | 60.55 | 73.78 | |
| Transtokenizers w/ NTPTraining Time=7h 00m2025.10 | 59.83 | 72.53 | |
| Transtokenizers w/ NTPTraining Time=10h 30m2025.10 | 59.38 | 72.53 | |
| Token DistillationTraining Time=8h 20m2025.10 | 59.23 | 71.06 | |
| Transtokenizers w/ NTPTraining Time=3h 30m2025.10 | 59.02 | 70.73 | |
| Token DistillationTraining Time=5h 30m2025.10 | 58.1 | 70.05 | |
| FOCUS w/ NTPTraining Time=3h 30m2025.10 | 57.38 | 71.41 | |
| Transtokenizers2025.10 | 46.03 | 53.96 | |
| FOCUS2025.10 | 37.14 | 42.96 | |
| TokAlign2025.10 | 32.98 | 32.21 | |
| WECHSEL2025.10 | 24.61 | 23.64 |