Automatic Speech Recognition on Fleurs
0.1058WERFAMCONN
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FAMCONNBackbone=Gemma, Language Family=Romance2026.01 | 0.1058 | — | — | — | — | — | — | — | — | — | — | — | -2.22 | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Romance2026.01 | 0.1115 | — | — | — | — | — | — | — | — | — | — | — | -26.32 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Romance2026.01 | 0.128 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Germanic2026.01 | 0.1422 | — | — | — | — | — | — | — | — | — | — | — | -3.04 | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Germanic2026.01 | 0.1567 | — | — | — | — | — | — | — | — | — | — | — | -7.7 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Germanic2026.01 | 0.1726 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Slavic2026.01 | 0.2169 | — | — | — | — | — | — | — | — | — | — | — | -3.38 | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Germanic2026.01 | 0.2337 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Turkic2026.01 | 0.2455 | — | — | — | — | — | — | — | — | — | — | — | -1.74 | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Slavic2026.01 | 0.2461 | — | — | — | — | — | — | — | — | — | — | — | -71.35 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Slavic2026.01 | 0.2507 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Turkic2026.01 | 0.2629 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Dravidian2026.01 | 0.2722 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Baltic2026.01 | 0.2756 | — | — | — | — | — | — | — | — | — | — | — | -3.67 | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Dravidian2026.01 | 0.2834 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Dravidian2026.01 | 0.2834 | — | — | — | — | — | — | — | — | — | — | — | 1.12 | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Indo-Iranian2026.01 | 0.296 | — | — | — | — | — | — | — | — | — | — | — | -9.83 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Baltic2026.01 | 0.3123 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Romance2026.01 | 0.3747 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Dravidian2026.01 | 0.386 | — | — | — | — | — | — | — | — | — | — | — | 10.26 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Indo-Iranian2026.01 | 0.3943 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Baltic2026.01 | 0.3964 | — | — | — | — | — | — | — | — | — | — | — | -73.57 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Afro-Asiatic2026.01 | 0.4477 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Turkic2026.01 | 0.4545 | — | — | — | — | — | — | — | — | — | — | — | -14.64 | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Indo-Iranian2026.01 | 0.4609 | — | — | — | — | — | — | — | — | — | — | — | -3.11 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Niger-Congo2026.01 | 0.4625 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Afro-Asiatic2026.01 | 0.4666 | — | — | — | — | — | — | — | — | — | — | — | 1.89 | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Niger-Congo2026.01 | 0.4814 | — | — | — | — | — | — | — | — | — | — | — | 1.89 | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Indo-Iranian2026.01 | 0.492 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Niger-Congo2026.01 | 0.5239 | — | — | — | — | — | — | — | — | — | — | — | -1.2 | — | — | — | |
| FAMCONNBackbone=Gemma, Language Family=Celtic2026.01 | 0.5311 | — | — | — | — | — | — | — | — | — | — | — | -0.47 | — | — | — | |
| LANGCONNBackbone=Gemma, Language Family=Celtic2026.01 | 0.5358 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Niger-Congo2026.01 | 0.5359 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Turkic2026.01 | 0.6009 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Afro-Asiatic2026.01 | 0.7056 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Afro-Asiatic2026.01 | 0.9269 | — | — | — | — | — | — | — | — | — | — | — | 22.13 | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Slavic2026.01 | 0.9596 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Baltic2026.01 | 1.1321 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FAMCONNBackbone=Salamandra, Language Family=Celtic2026.01 | 1.1373 | — | — | — | — | — | — | — | — | — | — | — | -3.05 | — | — | — | |
| LANGCONNBackbone=Salamandra, Language Family=Celtic2026.01 | 1.1678 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BEST-IWSLT25-IFBackbone=BEST-IWSLT25-IF2026.01 | 7.4 | — | — | — | — | — | — | — | — | 3.7 | — | — | — | — | — | — | |
| Seamless ASRBackbone=Seamless2026.01 | 8.9 | — | — | — | — | — | — | — | — | 5.3 | — | — | — | — | — | — | |
| MiLorE-SSL (Ours)MoE=true, Replay=true2026.01 | 10.4 | 9.4 | — | — | — | — | — | — | — | — | 10.2 | 11.6 | — | — | — | — | |
| SpeechMapper (EuroLLM)Backbone=EuroLLM, Stage=stage 2, Objective=CE+MSE2026.01 | 10.7 | — | — | — | — | — | — | — | — | 6.8 | — | — | — | — | — | — | |
| -MoEMoE=false, Replay=true2026.01 | 10.87 | 10 | — | — | — | — | — | — | — | — | 10.7 | 11.9 | — | — | — | — | |
| SpeechMapper (EuroLLM)Backbone=EuroLLM, Stage=stage 2, Objective=ASR CE2026.01 | 11.2 | — | — | — | — | — | — | — | — | 7.1 | — | — | — | — | — | — | |
| SpeechMapper (Llama 3.1)Backbone=Llama 3.1, Stage=stage 2, Objective=ASR CE2026.01 | 11.9 | — | — | — | — | — | — | — | — | 7.7 | — | — | — | — | — | — | |
| -MoE+ReplayMoE=false, Replay=false2026.01 | 15.27 | 24.4 | — | — | — | — | — | — | — | — | 9.9 | 11.5 | — | — | — | — | |
| -ReplayMoE=true, Replay=false2026.01 | 15.5 | 24.8 | — | — | — | — | — | — | — | — | 10.3 | 11.4 | — | — | — | — | |
| mHuBERT-147 (3iter)iterations=32026.01 | 15.53 | 15.9 | — | — | — | — | — | — | — | — | 15.1 | 15.6 | — | — | — | — | |
| HuBERT_Large2026.01 | 16.57 | 10.4 | — | — | — | — | — | — | — | — | 21.2 | 18.1 | — | — | — | — | |
| SpeechMapper (Llama 3.1)Backbone=Llama 3.1, Stage=stage 2, Objective=CE+MSE2026.01 | 16.6 | — | — | — | — | — | — | — | — | 11.1 | — | — | — | — | — | — | |
| mHuBERT-147 (2iter)iterations=22026.01 | 17.27 | 18.2 | — | — | — | — | — | — | — | — | 16.4 | 17.2 | — | — | — | — | |
| mHuBERT-147 (1iter)iterations=12026.01 | 25.83 | 26.6 | — | — | — | — | — | — | — | — | 25.6 | 25.3 | — | — | — | — | |
| SpeechMapper (Llama 3.1)Backbone=Llama 3.1, Stage=stage 12026.01 | 30.2 | — | — | — | — | — | — | — | — | 27.6 | — | — | — | — | — | — | |
| SpeechMapper (EuroLLM)Backbone=EuroLLM, Stage=stage 12026.01 | 45.3 | — | — | — | — | — | — | — | — | 37.5 | — | — | — | — | — | — | |
| Biodatlab Distil-Whisper LargeType=Offline2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.77 | 8.63 | — | |
| Biodatlab Whisper LargeType=Offline2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 16.5 | 15.26 | — | |
| BoostedTSV-MGroup=tau-Sub2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.37 | |
| Full-FT2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.63 | |
| Gemini 3 ProType=Offline2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.35 | 6.91 | — | |
| ISO-CGroup=tau-Sub2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 8.21 | |
| ISO-CTS w/ NSGroup=tau-Sub2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.61 | |
| Karcher MeanGroup=PS2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.43 | |
| Model StockGroup=PS2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.79 | |
| Multi-SLERPGroup=PS2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.49 | |
| Pathumma-Whisper Large-v3Type=Offline2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.29 | 7.88 | — | |
| PCBGroup=tau-Spa2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 8.92 | |
| SCEGroup=tau-Spa2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.86 | |
| Seed-ASR (ML)2024.07 | — | 0.0343 | 0.1305 | 0.025 | 0.0709 | 0.0424 | 0.0346 | 0.0325 | 0.0355 | — | — | — | — | — | — | — | |
| SOUPGroup=PS2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.98 | |
| SOUP w/o pt.Group=PS2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.4 | |
| TAGroup=tau-Spa2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.48 | |
| TIESGroup=tau-Spa2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.03 | |
| TSV-M w/ NSGroup=tau-Sub2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.68 | |
| TSV-M w/ OPGroup=tau-Sub2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.67 | |
| Typhoon ASR RealtimeType=Streaming2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 13.87 | 9.68 | — | |
| Typhoon Isan ASR RealtimeType=Streaming2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.55 | 10.15 | — | |
| Typhoon Whisper Large-v3Type=Offline2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.98 | 5.69 | — | |
| Typhoon Whisper TurboType=Offline2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.52 | 7.08 | — | |
| Universal-12024.07 | — | — | — | 0.05 | 0.068 | — | — | — | — | — | — | — | — | — | — | — | |
| Whisper Large-v22024.07 | — | 0.044 | 0.16 | 0.03 | 0.083 | 0.071 | 0.053 | 0.143 | 0.043 | — | — | — | — | — | — | — | |
| Whisper Large-v32024.07 | — | — | — | 0.028 | 0.056 | — | — | — | — | — | — | — | — | — | — | — | |
| zero-shot2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.43 |