Audio Question Answering on speech-chatbot-alpaca-eval (S-Alpaca)
66.7AccuracyAudioKV
Evaluation Results
| Method | Links | |
|---|---|---|
| AudioKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 66.7 | |
| PyramidKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 65.2 | |
| SnapKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 64.7 | |
| AdaKVModel=Qwen2.5-Omni-7B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 64.7 | |
| AudioKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 57.6 | |
| AdaKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 56.1 | |
| PyramidKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 56.1 | |
| SnapKVModel=Qwen2.5-Omni-3B, KV cache budget=40%, max_gen_tokens=64, Judge=Gemini 2.5 Flash2026.04 | 54.5 |