Automatic Speech Recognition on AMI
8.43WERCrisperWhisper
Evaluation results
| Method | Links | |
|---|---|---|
| CrisperWhisper2026.04 | 8.43 | |
| Mixed Training + Length Aug + Timestamp RegLength Augmentation=true, Timestamp Regularization=true2026.04 | 9.79 | |
| CrisperWhisperSize/h=10,000, Params/B=1.50, Evaluation Protocol=short-form2026.01 | 9.89 | |
| ASR-only Baseline2026.04 | 9.95 | |
| Mixed Training2026.04 | 10.09 | |
| FastWhisper-largeSize/h=1,634, Params/B=0.74, Evaluation Protocol=short-form2026.01 | 10.6 | |
| Mixed Training + Length AugLength Augmentation=true2026.04 | 10.69 | |
| Mixed Training + Length Aug + Reduced TFLength Augmentation=true, Reduced Teacher Forcing=true2026.04 | 10.87 | |
| Phi-4-multimodalArchitecture Type=AR, Params (B)=5.6, RTFx=4.85, Avg. WER=6.72026.02 | 11.7 | |
| MDM-ASRArchitecture Type=NAR, Params (B)=1.0, RTFx=46.81, Avg. WER=6.92026.02 | 12.2 | |
| Prompt Projectorprompt=best, tuning=LoRA2026.01 | 12.72 | |
| Prompt Projectorprompt=best, tuning=frozen2026.01 | 12.74 | |
| Vanilla LLM-ASRprompt=best, tuning=LoRA2026.01 | 12.79 | |
| Canary-1b-flashArchitecture Type=AR, Params (B)=1.0, RTFx=29.25, Avg. WER=7.22026.02 | 13.1 | |
| Vanilla LLM-ASRprompt=base, tuning=LoRA2026.01 | 13.25 | |
| Vanilla LLM-ASRprompt=best, tuning=frozen2026.01 | 13.36 | |
| Prompt Projectorprompt=base, tuning=LoRA2026.01 | 13.38 | |
| Prompt Projectorprompt=base, tuning=frozen2026.01 | 13.42 | |
| Vanilla LLM-ASRprompt=base, tuning=frozen2026.01 | 13.86 | |
| Vanilla LLM-ASRprompt=empty, tuning=frozen2026.01 | 13.88 | |
| CanarySize/h=31,200, Params/B=1.00, Evaluation Protocol=short-form2026.01 | 13.9 | |
| DraxArchitecture Type=NAR, Params (B)=1.2, RTFx=11.32, Avg. WER=9.22026.02 | 13.9 | |
| Qwen2.5-Omni-3B# Total Params=5B2025.11 | 15.05 | |
| distil-Whisper-large-v3Size/h=22,000, Params/B=0.75, Evaluation Protocol=short-form2026.01 | 15.1 | |
| SLM2026.01 | 15.14 | |
| Qwen2-AudioArchitecture Type=AR, Params (B)=8.4, RTFx=4.18, Avg. WER=8.32026.02 | 15.2 | |
| LFM2-Audio-1.5B# Total Params=1.5B2025.11 | 15.36 | |
| Parakeet-CTCArchitecture Type=NAR, Params (B)=1.1, RTFx=120.20, Avg. WER=8.32026.02 | 15.7 | |
| Whisper-large-v3Size/h=680,000, Params/B=1.50, Evaluation Protocol=short-form2026.01 | 16 | |
| Whisper-large-v3Architecture Type=AR, Params (B)=1.5, RTFx=12.83, Avg. WER=8.52026.02 | 16 | |
| Whisper-large-v3-turbo# Total Params=1.5B2025.11 | 16.13 | |
| Voxtral-MiniArchitecture Type=AR, Params (B)=3.0, RTFx=12.19, Avg. WER=8.02026.02 | 16.3 | |
| AR Beam2026.06 | 17 | |
| AR Greedy2026.06 | 17.8 | |
| NAR-MBR|Z|=64, Niter=12026.06 | 18.1 | |
| NAR-MBR|Z|=64, Niter=102026.06 | 18.1 | |
| NAR-MBR|Z|=256, Niter=12026.06 | 18.1 | |
| NAR-MBR|Z|=256, Niter=102026.06 | 18.1 | |
| NARNiter=102026.06 | 18.4 | |
| NAR-MBR|Z|=256, Niter=02026.06 | 18.6 | |
| NAR-MBR|Z|=64, Niter=02026.06 | 18.7 | |
| NARNiter=12026.06 | 18.8 | |
| NARNiter=02026.06 | 18.9 | |
| OWSM-v3.1Architecture Type=AR, Params (B)=1.0, RTFx=15.52, Avg. WER=10.32026.02 | 20.4 | |
| OWSM-CTCArchitecture Type=NAR, Params (B)=1.0, RTFx=132.54, Avg. WER=11.42026.02 | 23.8 | |
| Qwen-Audio2026.04 | 31.82 |