List your company

Multilingual Robustness Evaluation on MULTYPO (test)

59.11Average ScoreGemma
14.587626.146337.70549.2637Oct 10, 2025

Evaluation results

MethodLinks
GemmaModel Size=Large, Typo Rate=10%, Instruction-tuned=true2025.1059.11-3.7
GemmaModel Size=Medium, Typo Rate=10%, Instruction-tuned=true2025.1048.5-5.7
QwenModel Size=Large, Typo Rate=10%, Instruction-tuned=true2025.1047.19-5.7
QwenModel Size=Medium, Typo Rate=10%, Instruction-tuned=true2025.1044.5-8.2
OLMoModel Size=Large, Typo Rate=10%, Instruction-tuned=true2025.1036.82-4.3
OLMoModel Size=Medium, Typo Rate=10%, Instruction-tuned=true2025.1029.16-7.9
QwenModel Size=Small, Typo Rate=10%, Instruction-tuned=true2025.1027.86-5.7
GemmaModel Size=Small, Typo Rate=10%, Instruction-tuned=true2025.1021.46-9.9
OLMoModel Size=Small, Typo Rate=10%, Instruction-tuned=true2025.1016.3-9.5