Abstractive Summarization on XL-Sum Ukrainian (test)
6.52BLEU ScoreGemma 3 12B PT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemma 3 12B PT2025.10 | 6.52 | 12.68 | |
| MATTTraining Time=7h 00m2025.10 | 5.95 | 11.81 | |
| FOCUS w/ NTPTraining Time=10h 30m2025.10 | 5.16 | 8.46 | |
| FOCUS w/ NTPTraining Time=7h 00m2025.10 | 5.04 | 8.72 | |
| Transtokenizers w/ NTPTraining Time=7h 00m2025.10 | 4.8 | 3.14 | |
| Transtokenizers w/ NTPTraining Time=10h 30m2025.10 | 4.8 | 3.24 | |
| FOCUS w/ NTPTraining Time=3h 30m2025.10 | 4.32 | 7.56 | |
| Transtokenizers w/ NTPTraining Time=3h 30m2025.10 | 4.06 | 2.46 | |
| Token DistillationTraining Time=8h 20m2025.10 | 4.06 | 6.68 | |
| Token DistillationTraining Time=5h 30m2025.10 | 3.71 | 6.42 | |
| FOCUS2025.10 | 0.2 | 2.07 | |
| Transtokenizers2025.10 | 0.02 | 0.06 | |
| TokAlign2025.10 | 0.01 | 0.08 | |
| WECHSEL2025.10 | 0 | 0 |