Spoken Question Answering on Spoken SQuAD (test)
91.1AccuracyTranscripts + EuroLLM 9B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Transcripts + EuroLLM 9BBackbone=EuroLLM 9B, Input Mode=Transcripts, Evaluation Protocol=topline2026.01 | 91.1 | — | — | |
| Transcripts + Llama 3.1 8BBackbone=Llama 3.1 8B, Input Mode=Transcripts, Evaluation Protocol=topline2026.01 | 89.2 | — | — | |
| Seamless ASR + EuroLLM 9BBackbone=EuroLLM 9B, Input Mode=Seamless ASR, Evaluation Protocol=zero-shot2026.01 | 89.2 | — | — | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 2 [ASR/SQA CE], Evaluation Protocol=in-domain2026.01 | 87.9 | — | — | |
| BEST-IWSLT25-IFEvaluation Protocol=in-domain2026.01 | 87.4 | — | — | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 2 [ASR/SQA CE], Evaluation Protocol=in-domain2026.01 | 87.4 | — | — | |
| Seamless ASR + Llama 3.1 8BBackbone=Llama 3.1 8B, Input Mode=Seamless ASR, Evaluation Protocol=zero-shot2026.01 | 85.6 | — | — | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 2 [ASR CE+MSE], Evaluation Protocol=zero-shot2026.01 | 75.1 | — | — | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 2 [ASR CE+MSE], Evaluation Protocol=zero-shot2026.01 | 72.3 | — | — | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 1, Evaluation Protocol=zero-shot2026.01 | 62.3 | — | — | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 1, Evaluation Protocol=zero-shot2026.01 | 61.9 | — | — | |
| BERTArchitecture=Cascade, Training Set=ASR2019.10 | — | 66.56 | 63.87 | |
| BERTArchitecture=Cascade, Training Set=ASR2019.10 | — | 48.74 | 45.84 | |
| SpeechBERTArchitecture=End-to-end, Training Set=Audio2019.10 | — | 62.76 | 59.7 | |
| SpeechBERTArchitecture=End-to-end, Training Set=Audio2019.10 | — | 50.12 | 46.72 |