Audio Question Answering on speech-web-questions (S-Web)
74.9AccuracyQwen3-Omni
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3-OmniSize=30B-A3B2026.04 | 74.9 | |
| MiniCPM-o 4.5Size=9B2026.04 | 70.2 | |
| AudioKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 55.5 | |
| PyramidKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 55.1 | |
| AdaKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 54.9 | |
| SnapKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 54.5 | |
| Kimi-AudioSize=9B2026.04 | 46.4 | |
| AudioKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 46.1 | |
| PyramidKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 45.3 | |
| SnapKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 44.9 | |
| AdaKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 44.5 |