Spoken Short-Response Question Answering on Synthetic Spoken SRQA
89.6ARC-E AccuracyQwen2.5-Omni-7B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-Omni-7BEvaluation Protocol=Complete-audio baseline2026.05 | 89.6 | 81.1 | 70.9 | 71.3 | 26.7 | 71 | 70.8 | 10.27 | |
| DAPO controller (6 rewards)Evaluation Protocol=Streaming controller, Optimization=DAPO, Rewards=62026.05 | 89.6 | 81.7 | 71 | 69.2 | 25.9 | 71 | 70.3 | 8.99 | |
| DAPO controller (5 rewards)Evaluation Protocol=Streaming controller, Optimization=DAPO, Rewards=52026.05 | 89.1 | 81.7 | 69.6 | 66.4 | 24.9 | 71 | 69.2 | 10.94 | |
| DAPO controller (4 rewards)Evaluation Protocol=Streaming controller, Optimization=DAPO, Rewards=42026.05 | 88.9 | 81.7 | 68.4 | 65.1 | 24.6 | 70.3 | 68.5 | 10.87 | |
| Qwen2.5-Omni-7BEvaluation Protocol=Streaming controller2026.05 | 87.8 | 80.8 | 68.6 | 63.5 | 22.8 | 71 | 67.6 | 10.44 | |
| SFT controllerEvaluation Protocol=Streaming controller, Optimization=SFT2026.05 | 86.3 | 78.1 | 68.6 | 60.5 | 21.6 | 71.7 | 66.1 | 9.82 | |
| Moshi + CoTEvaluation Protocol=Streaming baseline, Source=Shih et al. [31]2026.05 | 77.7 | 59.8 | 56.1 | 56.9 | 16.1 | 57.8 | 56.6 | — | |
| Audio Flamingo 3Evaluation Protocol=Complete-audio baseline2026.05 | 75.5 | 59.4 | 42.2 | 32.7 | 10.1 | 66.3 | 47.4 | 4.97 | |
| Moshi QC + length-DPOEvaluation Protocol=Streaming baseline, Source=Shih et al. [31]2026.05 | 65.4 | 46 | 45.3 | 46 | 14.7 | 56.9 | 46.7 | 19.31 | |
| GLM-4-Voice-9BEvaluation Protocol=Complete-audio baseline2026.05 | 64.9 | 48.5 | 41.2 | 22.1 | 6.7 | 63.7 | 40.2 | 20.88 | |
| Moshi QC-SFTEvaluation Protocol=Streaming baseline, Source=Shih et al. [31]2026.05 | 62.8 | 43.2 | 45.1 | 40.7 | 13.8 | 56.2 | 44.4 | 52.42 | |
| Moshi + CoT w/o streaming ASREvaluation Protocol=Streaming baseline, Source=Shih et al. [31]2026.05 | 55.8 | 44 | 50.1 | 46.3 | 12.2 | 59.9 | 44.8 | — | |
| Audio Flamingo 3 + AF-ThinkEvaluation Protocol=Complete-audio baseline2026.05 | 47.3 | 35 | 31.6 | 21.7 | 3 | 48 | 30.5 | 9.18 | |
| Moshi baselineEvaluation Protocol=Streaming baseline, Source=Shih et al. [31]2026.05 | 30.2 | 21.5 | 22.8 | 23.8 | 8.7 | 42.8 | 23.4 | — |