Close-ended Spoken Question Answering on Audio-MLQA
4.34Score (EN)SEA-LION-v3-8B-IT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SEA-LION-v3-8B-ITApproach category=Text-only reference2026.03 | 4.34 | 4.1 | 4.19 | 4.08 | 3.98 | 4.14 | |
| Whisper-v3 + SEA-LION-v3-8B-ITApproach category=Cascaded2026.03 | 4.2 | 3.75 | 4.09 | 4.01 | 3.94 | 3.99 | |
| Ours (hard-gating)Approach category=Distilled end-to-end, Gating strategy=hard-gating2026.03 | 4.16 | 3.72 | 4.13 | 4.07 | 3.7 | 3.96 | |
| EN-DiVAApproach category=Distilled end-to-end, Training protocol=re-trained baseline2026.03 | 4.12 | — | — | — | — | 4.12 | |
| Ours (soft-gating)Approach category=Distilled end-to-end, Gating strategy=soft-gating2026.03 | 4.12 | 3.68 | 4.03 | 4.02 | 3.56 | 3.88 | |
| ML-DiVAApproach category=Distilled end-to-end, Training protocol=re-trained baseline2026.03 | 4.09 | 3.64 | 4.02 | 3.97 | 3.55 | 3.85 | |
| SeaLLMs-AudioApproach category=Zero-shot end-to-end2026.03 | 3.74 | 2.66 | 2.6 | 2.54 | 3.41 | 2.99 | |
| Qwen2-AudioApproach category=Zero-shot end-to-end2026.03 | 3.54 | — | 3.37 | 3.09 | 3.38 | 3.02 | |
| Glm-4-voiceApproach category=Zero-shot end-to-end2026.03 | 3.04 | — | — | — | 2.86 | 2.95 | |
| MERaLiON-2-10BApproach category=Zero-shot end-to-end2026.03 | 1.73 | 0.9 | 1.28 | 1.31 | 1.32 | 1.31 |