Spoken Question Answering on LibriSQA (Accuracy and ALAS Metrics)
1,750N Verified Verifications CountAF3
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| AF3Backbone=AF-Whisper, LLM=Qwen2.5-7B2025.05 | 1,750 | 80 | 72 | 20 | 0.95 | 0.97 | 38.5 | 37.6 | |
| SALMONNBackbone=Whisper-v2 + BEATs, LLM=Vicuna-13B, Adapter=Q-Former2025.05 | 1,382 | 67 | 64 | 38 | 1.14 | 1.14 | 20.2 | 20.4 | |
| Qwen2-AudioBackbone=Whisper-large-v3, LLM=Qwen-7B, Adapter=None2025.05 | 1,174 | 57 | 56 | 32 | 1 | 1.02 | 36.9 | 36.2 | |
| Qwen-OmniBackbone=Streaming-modified Whisper, LLM=Qwen2.5 Thinker, Adapter=Learned projection2025.05 | 1,167 | 51 | 56 | 22 | 1.02 | 1.04 | 35.2 | 34 |