Audio Question Answering on SH-Bench (test)
0.974Accuracy (Main Speaker, General)Step-Audio-2-mini + BPFT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Step-Audio-2-mini + BPFTModel Type=BPFT Model, BPFT=true2025.12 | 0.974 | 0.943 | 0.81 | 0.961 | 0.917 | |
| Gemini 2.5 ProModel Type=Proprietary, BPFT=false2025.12 | 0.973 | 0.97 | 0.655 | 0.592 | 0.758 | |
| Kimi-Audio 7B InstructModel Type=Open-source, BPFT=false2025.12 | 0.969 | 0.963 | 0.674 | 0.314 | 0.594 | |
| PipelineModel Type=Pipeline2025.12 | 0.967 | 0.972 | 0.507 | 0.491 | 0.659 | |
| Qwen-2.5-Omni 3BModel Type=Open-source, BPFT=false2025.12 | 0.962 | 0.956 | 0.531 | 0.547 | 0.69 | |
| Qwen-2.5-Omni 7BModel Type=Open-source, BPFT=false2025.12 | 0.96 | 0.955 | 0.482 | 0.476 | 0.639 | |
| Llama-Omni 2 14BModel Type=Open-source, BPFT=false2025.12 | 0.957 | 0.329 | 0.153 | 0.87 | 0.34 | |
| Step-Audio-2-miniModel Type=Open-source, BPFT=false2025.12 | 0.942 | 0.937 | 0.547 | 0.315 | 0.561 | |
| Qwen-2.5-Omni 7B + BPFTModel Type=BPFT Model, BPFT=true2025.12 | 0.933 | 0.927 | 0.82 | 0.938 | 0.902 | |
| GPT-4o-audio-previewModel Type=Proprietary, BPFT=false2025.12 | 0.723 | 0.844 | 0.432 | 0.445 | 0.561 |