Speech-to-Text on VoiceBench
4.78AlpacaEval ScoreGPT-4o-Audio
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-4o-AudioAccess=Proprietary2026.02 | 4.78 | 4.49 | 4.58 | 75.5 | 80.25 | 89.23 | 84.1 | 76.02 | 98.65 | 86.75 | |
| Qwen2.5-OmniAccess=Open-Source2026.02 | 4.49 | 3.93 | — | 55.71 | 61.32 | 81.1 | — | 52.87 | 99.42 | — | |
| Kimi-AudioAccess=Proprietary2026.02 | 4.46 | 3.97 | 4.2 | 63.12 | 62.17 | 83.52 | 69.7 | 61.1 | 100 | 76.91 | |
| VITA-1.5Access=Open-Source2026.02 | 4.21 | 3.66 | 3.48 | 38.88 | 52.15 | 71.65 | 55.3 | 38.14 | 97.69 | 64.53 | |
| Ultravox-v0.5-Llama-3.2-1b# Total Params=0.7B2025.11 | 4.02 | 3.53 | 3.48 | 46.38 | 29.99 | 33.41 | 51.4 | 43.51 | 97.88 | — | |
| LFM2-Audio-1.5B# Total Params=1.5B2025.11 | 3.78 | 3.48 | 3.12 | 34.81 | 33.99 | 45.49 | 51.2 | 30.13 | 98.85 | — | |
| Qwen2.5-Omni-3B# Total Params=5B2025.11 | 3.72 | 3.51 | 3.42 | 44.94 | 55.29 | 76.26 | 61.3 | 32.9 | 88.46 | — | |
| LLaMA-OmniAccess=Open-Source2026.02 | 3.7 | 3.46 | 2.92 | 39.69 | 25.93 | 27.47 | 49.2 | 14.87 | 11.35 | 41.12 | |
| VITA-1.0Access=Open-Source2026.02 | 3.38 | 2.15 | 1.87 | 27.94 | 25.7 | 29.01 | 47.7 | 22.82 | 26.73 | 36.43 | |
| Ex-OmniAccess=Open-Source2026.02 | 2.57 | 2.87 | 2.19 | 40.14 | 24.24 | 25.49 | 50.4 | 16.22 | 83.08 | 43.57 | |
| Mini-Omni2Access=Open-Source2026.02 | 2.32 | 2.18 | 1.79 | 9.31 | 24.27 | 26.59 | 46.4 | 11.56 | 57.5 | 33.49 | |
| Mini-Omni2# Total Params=0.6B2025.11 | 2.32 | 2.18 | 1.79 | 9.31 | 24.27 | 26.59 | 46.4 | 11.56 | 57.5 | — | |
| MoshiAccess=Open-Source2026.02 | 2.01 | 1.6 | 1.3 | 15.64 | 24.04 | 25.93 | 47.4 | 10.12 | 44.23 | 29.51 | |
| Moshi# Total Params=7B2025.11 | 2.01 | 1.6 | 1.3 | 15.64 | 24.04 | 25.93 | 47.4 | 10.12 | 44.23 | — | |
| Mini-OmniAccess=Open-Source2026.02 | 1.95 | 2.02 | 1.61 | 13.92 | 24.69 | 26.59 | 46.3 | 13.58 | 37.12 | 30.42 |