Spoken Question Answering on UltraEval-Audio S2S
0.7338AlpacaEval ScoreGPT-Audio
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-Audio2025.12 | 0.7338 | 0.89 | 0.7285 | 0.5541 | 0.7266 | |
| Fun-Audio-Chat-30B-A3BLLM Size=30B-A3B, Frame Rate-In=5, Frame Rate-Out=52025.12 | 0.6449 | 0.7867 | 0.542 | 0.5118 | 0.6214 | |
| Fun-Audio-Chat-8BLLM Size=8B, Frame Rate-In=5, Frame Rate-Out=52025.12 | 0.6187 | 0.7833 | 0.4951 | 0.4852 | 0.5956 | |
| MiMo-AudioLLM Size=7B, Frame Rate-In=6.25, Frame Rate-Out=6.25+τ2025.12 | 0.6146 | 0.7733 | 0.4043 | 0.4286 | 0.5552 | |
| Baichuan-Omni-1.5LLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=12.5+τ2025.12 | 0.5869 | 0.6733 | 0.3057 | 0.3809 | 0.4867 | |
| Step-Audio2-MiniLLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=25+τ2025.12 | 0.5172 | 0.6767 | 0.335 | 0.3465 | 0.4689 | |
| GLM4-VoiceLLM Size=9B, Frame Rate-In=12.5, Frame Rate-Out=12.5+τ2025.12 | 0.51 | 0.5 | 0.364 | 0.32 | 0.4235 | |
| MiniCPM-o 2.6LLM Size=7B, Frame Rate-In=25, Frame Rate-Out=τ2025.12 | 0.51 | 0.61 | 0.402 | 0.4 | 0.4805 | |
| Kimi-AudioLLM Size=7B, Frame Rate-In=12.5, Frame Rate-Out=12.52025.12 | 0.442 | 0.5733 | 0.3571 | 0.339 | 0.4279 |