# Spoken Short-Response Question Answering on Synthetic Spoken SRQA Source: [Spoken Short-Response Question Answering on Synthetic Spoken SRQA]() Collection index: [llms.txt]() Data updated: 2026-05-27T10:11:30.053Z Evaluation of model performance on the Synthetic Spoken SRQA dataset for Spoken Short-Response Question Answering, measured using a composite set of metrics including ARC, SIQA, PIQA, and GSM8K scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Spoken Short-Response Question Answering | | Dataset (including any stated split) | Synthetic Spoken SRQA | | Metric | ARC-E Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ARC-E Accuracy](): Higher is better; unit: PERCENT. - [ARC-C Accuracy](): Higher is better; unit: PERCENT. - [SIQA Score](): Higher is better; unit: PERCENT. - [PIQA Score](): Higher is better; unit: PERCENT. - [GSM8K Accuracy](): Higher is better; unit: PERCENT. - [LLaMA-QS Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. - [Final Composite Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | ARC-E Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Qwen2.5-Omni-7B | 89.6 | Not recorded | 6a167ad1dbd7150506f80286 | | DAPO controller (6 rewards) | 89.6 | Not recorded | 6a167ad1dbd7150506f80286 | | DAPO controller (5 rewards) | 89.1 | Not recorded | 6a167ad1dbd7150506f80286 | | DAPO controller (4 rewards) | 88.9 | Not recorded | 6a167ad1dbd7150506f80286 | | Qwen2.5-Omni-7B | 87.8 | Not recorded | 6a167ad1dbd7150506f80286 | | SFT controller | 86.3 | Not recorded | 6a167ad1dbd7150506f80286 | | Moshi + CoT | 77.7 | Not recorded | 6a167ad1dbd7150506f80286 | | Audio Flamingo 3 | 75.5 | Not recorded | 6a167ad1dbd7150506f80286 | | Moshi QC + length-DPO | 65.4 | Not recorded | 6a167ad1dbd7150506f80286 | | GLM-4-Voice-9B | 64.9 | Not recorded | 6a167ad1dbd7150506f80286 | | Moshi QC-SFT | 62.8 | Not recorded | 6a167ad1dbd7150506f80286 | | Moshi + CoT w/o streaming ASR | 55.8 | Not recorded | 6a167ad1dbd7150506f80286 | | Audio Flamingo 3 + AF-Think | 47.3 | Not recorded | 6a167ad1dbd7150506f80286 | | Moshi baseline | 30.2 | Not recorded | 6a167ad1dbd7150506f80286 | ### Qwen2.5-Omni-7B ARC-E Accuracy: 89.6. Result ID: 6a16c33efbc373592c107f28. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Complete-audio baseline"}. This record does not assert independent reproduction. ### DAPO controller (6 rewards) ARC-E Accuracy: 89.6. Result ID: 6a16c33efbc373592c107f35. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming controller","Optimization":"DAPO","Rewards":"6"}. This record does not assert independent reproduction. ### DAPO controller (5 rewards) ARC-E Accuracy: 89.1. Result ID: 6a16c33efbc373592c107f34. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming controller","Optimization":"DAPO","Rewards":"5"}. This record does not assert independent reproduction. ### DAPO controller (4 rewards) ARC-E Accuracy: 88.9. Result ID: 6a16c33efbc373592c107f33. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming controller","Optimization":"DAPO","Rewards":"4"}. This record does not assert independent reproduction. ### Qwen2.5-Omni-7B ARC-E Accuracy: 87.8. Result ID: 6a16c33efbc373592c107f31. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming controller"}. This record does not assert independent reproduction. ### SFT controller ARC-E Accuracy: 86.3. Result ID: 6a16c33efbc373592c107f32. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming controller","Optimization":"SFT"}. This record does not assert independent reproduction. ### Moshi + CoT ARC-E Accuracy: 77.7. Result ID: 6a16c33efbc373592c107f2d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming baseline","Source":"Shih et al. \[31\]"}. This record does not assert independent reproduction. ### Audio Flamingo 3 ARC-E Accuracy: 75.5. Result ID: 6a16c33efbc373592c107f29. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Complete-audio baseline"}. This record does not assert independent reproduction. ### Moshi QC + length-DPO ARC-E Accuracy: 65.4. Result ID: 6a16c33efbc373592c107f30. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming baseline","Source":"Shih et al. \[31\]"}. This record does not assert independent reproduction. ### GLM-4-Voice-9B ARC-E Accuracy: 64.9. Result ID: 6a16c33efbc373592c107f2b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Complete-audio baseline"}. This record does not assert independent reproduction. ### Moshi QC-SFT ARC-E Accuracy: 62.8. Result ID: 6a16c33efbc373592c107f2f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming baseline","Source":"Shih et al. \[31\]"}. This record does not assert independent reproduction. ### Moshi + CoT w/o streaming ASR ARC-E Accuracy: 55.8. Result ID: 6a16c33efbc373592c107f2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming baseline","Source":"Shih et al. \[31\]"}. This record does not assert independent reproduction. ### Audio Flamingo 3 + AF-Think ARC-E Accuracy: 47.3. Result ID: 6a16c33efbc373592c107f2a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Complete-audio baseline"}. This record does not assert independent reproduction. ### Moshi baseline ARC-E Accuracy: 30.2. Result ID: 6a16c33efbc373592c107f2c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Streaming baseline","Source":"Shih et al. \[31\]"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.