Speech-to-Text Question Answering on OBQA
83.08AccuracyAPin
Evaluation Results
| Method | Links | |
|---|---|---|
| APinSetting=Aggressive (τin=0.80, τdeep=0.70), lin=✓, ldeep=-, FRR=86.28, Pre. GFLOPs=351.18, FLOPs Ratio=86.242026.04 | 83.08 | |
| APinSetting=Conservative (τin=0.90, τdeep=0.80), lin=✓, ldeep=-, FRR=96.44, Pre. GFLOPs=392.69, FLOPs Ratio=96.432026.04 | 82.64 | |
| APdeepSetting=Conservative (τin=0.90, τdeep=0.80), lin=-, ldeep=✓, FRR=71.18, Pre. GFLOPs=390.42, FLOPs Ratio=95.872026.04 | 82.64 | |
| DAPSetting=Aggressive (τin=0.80, τdeep=0.70), lin=✓, ldeep=✓, FRR=40.71, Pre. GFLOPs=324.64, FLOPs Ratio=79.722026.04 | 82.42 | |
| APdeepSetting=Aggressive (τin=0.80, τdeep=0.70), lin=-, ldeep=✓, FRR=48.81, Pre. GFLOPs=377.39, FLOPs Ratio=92.682026.04 | 82.2 | |
| Vanillalin=-, ldeep=-, FRR=100.0, Pre. GFLOPs=407.22, FLOPs Ratio=100.02026.04 | 81.98 | |
| DAPSetting=Conservative (τin=0.90, τdeep=0.80), lin=✓, ldeep=✓, FRR=68.19, Pre. GFLOPs=376.14, FLOPs Ratio=92.362026.04 | 81.98 | |
| Phi-4-MultimodalType=AR, S→S capability=false2026.01 | 65.9 | |
| DIFFUSPEECHType=Diff., S→S capability=true2026.01 | 51.3 | |
| Qwen2-AudioType=AR, S→S capability=false2026.01 | 49.5 | |
| MinMoType=AR, S→S capability=true2026.01 | 44.5 | |
| DiFFAType=Diff., S→S capability=false2026.01 | 35.6 | |
| Llama-Omni2Type=AR, S→S capability=true2026.01 | 28.1 | |
| MoshiType=AR, S→S capability=true2026.01 | 26 | |
| SpiritLMType=AR, S→S capability=true2026.01 | 21.7 | |
| SpeechGPTType=AR, S→S capability=true2026.01 | 12.4 |