Speech-to-Text Spoken Question Answering on VoiceBench S2T (test)
4.8AlpacaEvalFun-Audio-Chat-8B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Fun-Audio-Chat-8BLLM Size=8B, Frame Rate-In=5, Frame Rate-Out=52025.12 | 4.8 | 4.42 | 66.27 | 71.08 | 83.52 | 78.52 | 98.65 | 83.21 | |
| MiMo-AudioLLM Size=7B, Frame Rate-In=6.25, Frame Rate-Out=6.25+τ2025.12 | 4.6 | 3.77 | 54.79 | 59.66 | 73.41 | 66.45 | 96.73 | 74.06 | |
| Baichuan-Omni-1.5LLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=12.5+τ2025.12 | 4.5 | 4.05 | 43.4 | 57.25 | 74.51 | 54.54 | 97.31 | 71.14 | |
| Kimi-AudioLLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=12.52025.12 | 4.46 | 3.97 | 63.12 | 62.17 | 83.52 | 61.1 | 100 | 76.93 | |
| MiniCPM-o 2.6LLM Size=7B, Frame Rate-In=25, Frame Rate-Out=τ2025.12 | 4.42 | 4.15 | 50.72 | 54.78 | 78.02 | 49.25 | 97.69 | 71.69 | |
| Step-Audio2-MiniLLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=25+τ2025.12 | 4.17 | 3 | 56.06 | 52.18 | 64.18 | 38.01 | 93.08 | 63.84 | |
| GLM4-VoiceLLM Size=9B, Frame Rate-In=12.5, Frame Rate-Out=12.5+τ2025.12 | 3.97 | 3.42 | 36.98 | 39.75 | 53.41 | 52.8 | 88.08 | 59.83 |