Multilingual Robustness Evaluation on MULTYPO (test)
59.11Average ScoreGemma
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GemmaModel Size=Large, Typo Rate=10%, Instruction-tuned=true2025.10 | 59.11 | -3.7 | |
| GemmaModel Size=Medium, Typo Rate=10%, Instruction-tuned=true2025.10 | 48.5 | -5.7 | |
| QwenModel Size=Large, Typo Rate=10%, Instruction-tuned=true2025.10 | 47.19 | -5.7 | |
| QwenModel Size=Medium, Typo Rate=10%, Instruction-tuned=true2025.10 | 44.5 | -8.2 | |
| OLMoModel Size=Large, Typo Rate=10%, Instruction-tuned=true2025.10 | 36.82 | -4.3 | |
| OLMoModel Size=Medium, Typo Rate=10%, Instruction-tuned=true2025.10 | 29.16 | -7.9 | |
| QwenModel Size=Small, Typo Rate=10%, Instruction-tuned=true2025.10 | 27.86 | -5.7 | |
| GemmaModel Size=Small, Typo Rate=10%, Instruction-tuned=true2025.10 | 21.46 | -9.9 | |
| OLMoModel Size=Small, Typo Rate=10%, Instruction-tuned=true2025.10 | 16.3 | -9.5 |