Automatic Speech Recognition on MyST (test)
8.21WERGumbel-BEARD
Evaluation Results
| Method | Links | |
|---|---|---|
| Gumbel-BEARDBackbone=Whisper-medium, Labeled Data Size=Full2026.06 | 8.21 | |
| Single Dataset Fine-tuneFine-tuning Protocol=Encoder+Projector+LoRA2026.06 | 8.34 | |
| Previous SOTA2026.06 | 8.5 | |
| Gumbel-BEARD (Hard selection)Backbone=Whisper-small, Labeled Data Size=Full2026.06 | 8.51 | |
| Single Dataset Fine-tuneFine-tuning Protocol=Projector+LoRA2026.06 | 8.58 | |
| C-DR MoERouting Strategy=Hard Routing, Classifier Type=Ground-Truth Routing2026.06 | 8.58 | |
| C-DR MoERouting Strategy=Hard Routing, Classifier Type=Weighted-layer Classifier (Single-Stage)2026.06 | 8.58 | |
| C-DR MoERouting Strategy=Hard Routing, Classifier Type=Weighted-layer Classifier (Coarse-to-Fine)2026.06 | 8.58 | |
| C-DR MoERouting Strategy=Hard Routing, Classifier Type=Weighted-layer Classifier (Coarse-to-Fine), Entropy-Aware Routing (EAR)=true2026.06 | 8.58 | |
| C-DR MoERouting Strategy=Soft Routing, Classifier Type=Weighted-layer Classifier (Coarse-to-Fine)2026.06 | 8.58 | |
| C-DR MoERouting Strategy=Soft Routing, Classifier Type=Weighted-layer Classifier (Coarse-to-Fine), Entropy-Aware Routing (EAR)=true2026.06 | 8.58 | |
| GC-LoRAWhisper model size=medium2026.06 | 8.6 | |
| GC-LoRAParams=447k, Backbone=Whisper-Medium2026.06 | 8.6 | |
| C-DR MoERouting Strategy=Hard Routing, Classifier Type=Top-layer Classifier (Single-Stage)2026.06 | 8.61 | |
| Vanilla-Routing MoEFine-tuning Protocol=Pretrain + Joint2026.06 | 8.63 | |
| GC-LoRAWhisper model size=large-v32026.06 | 8.7 | |
| Baseline (BEARD)Backbone=Whisper-small, Labeled Data Size=Full2026.06 | 8.73 | |
| Vanilla-Routing MoEFine-tuning Protocol=Joint2026.06 | 8.74 | |
| Gumbel-BEARD (Soft selection)Backbone=Whisper-small, Labeled Data Size=Full2026.06 | 8.76 | |
| Baseline (SFT)Backbone=Whisper-medium, Labeled Data Size=Full2026.06 | 8.86 | |
| Gumbel-BEARDBackbone=Whisper-medium, Labeled Data Size=10 h2026.06 | 8.88 | |
| LoRAWhisper model size=medium2026.06 | 8.9 | |
| Full FTParams=764M, Backbone=Whisper-Medium2026.06 | 8.9 | |
| LoRAParams=829k, Backbone=Whisper-Medium2026.06 | 8.9 | |
| Canary-QwenFine-tuning Protocol=Zero-shot2026.06 | 8.96 | |
| Gumbel-BEARDBackbone=Whisper-medium, Labeled Data Size=1 h2026.06 | 9.15 | |
| Baseline (SFT)Backbone=Whisper-medium, Labeled Data Size=10 h2026.06 | 9.19 | |
| Single-Expert2026.06 | 9.27 | |
| Baseline (SFT)Backbone=Whisper-small, Labeled Data Size=Full2026.06 | 9.34 | |
| Gumbel-BEARD (Hard selection)Backbone=Whisper-small, Labeled Data Size=10 h2026.06 | 9.35 | |
| Baseline (BEARD)Backbone=Whisper-small, Labeled Data Size=10 h2026.06 | 9.44 | |
| LoRAWhisper model size=large-v32026.06 | 9.5 | |
| Baseline (SFT)Backbone=Whisper-medium, Labeled Data Size=1 h2026.06 | 9.56 | |
| GC-LoRAWhisper model size=small2026.06 | 9.6 | |
| Gumbel-BEARD (Soft selection)Backbone=Whisper-small, Labeled Data Size=10 h2026.06 | 9.62 | |
| Baseline (PL)Backbone=Whisper-small, Labeled Data Size=Full2026.06 | 9.63 | |
| LoRAWhisper model size=small2026.06 | 9.8 | |
| Baseline (SFT)Backbone=Whisper-small, Labeled Data Size=10 h2026.06 | 9.94 | |
| Gumbel-BEARD (Hard selection)Backbone=Whisper-small, Labeled Data Size=1 h2026.06 | 10.18 | |
| Baseline (BEARD)Backbone=Whisper-small, Labeled Data Size=1 h2026.06 | 10.31 | |
| Baseline (SFT)Backbone=Whisper-small, Labeled Data Size=1 h2026.06 | 10.64 | |
| Gumbel-BEARD (Soft selection)Backbone=Whisper-small, Labeled Data Size=1 h2026.06 | 10.65 | |
| Baseline (PL)Backbone=Whisper-small, Labeled Data Size=10 h2026.06 | 11.66 | |
| GC-LoRAWhisper model size=base2026.06 | 12.2 | |
| Vanilla-Routing MoEFine-tuning Protocol=Pretrain + Gate-only2026.06 | 12.4 | |
| Baseline (PL)Backbone=Whisper-small, Labeled Data Size=1 h2026.06 | 12.45 | |
| LoRAWhisper model size=base2026.06 | 12.7 | |
| Zero-shotParams=0, Backbone=Whisper-Medium2026.06 | 13.1 | |
| Zero-shotBackbone=Whisper-medium, Labeled Data Size=no fine-tuning2026.06 | 13.1 | |
| Zero-shotBackbone=Whisper-small, Fine-tuning=None2026.06 | 13.4 | |
| GC-LoRAWhisper model size=tiny2026.06 | 15.7 | |
| LoRAWhisper model size=tiny2026.06 | 15.9 |