Speech-to-Text Spoken Question Answering on OpenAudioBench S2T (test)
88.94AlpacaEval ScoreFun-Audio-Chat-8B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Fun-Audio-Chat-8BLLM Size=8B, Frame Rate-In=5, Frame Rate-Out=52025.12 | 88.94 | 83.33 | 69.8 | 68.1 | 72.9 | 76.61 | |
| MiMo-AudioLLM Size=7B, Frame Rate-In=6.25, Frame Rate-Out=6.25+τ2025.12 | 85.43 | 79.67 | 53.96 | 52.8 | 55.4 | 65.45 | |
| Baichuan-Omni-1.5LLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=12.5+τ2025.12 | 77.9 | 78.5 | 50 | 57.2 | 59.1 | 64.54 | |
| Kimi-AudioLLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=12.52025.12 | 75.73 | 79.33 | 58.02 | 62.1 | 70.2 | 69.08 | |
| MiniCPM-o 2.6LLM Size=7B, Frame Rate-In=25, Frame Rate-Out=τ2025.12 | 64.1 | 78 | 38.6 | 63 | 69.2 | 62.58 | |
| Step-Audio2-MiniLLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=25+τ2025.12 | 59.6 | 75 | 46.04 | 57.7 | 65.1 | 60.69 | |
| GLM4-VoiceLLM Size=9B, Frame Rate-In=12.5, Frame Rate-Out=12.5+τ2025.12 | 57.89 | 76 | 47.43 | 51.8 | 55.4 | 57.7 |