Spoken Question Answering on LibriSQA (Part I)
87.6AccuracyTranscripts + EuroLLM 9B
Evaluation results
| Method | Links | |
|---|---|---|
| Transcripts + EuroLLM 9BBackbone=EuroLLM 9B, Input Mode=Transcripts, Evaluation Protocol=topline2026.01 | 87.6 | |
| Transcripts + Llama 3.1 8BBackbone=Llama 3.1 8B, Input Mode=Transcripts, Evaluation Protocol=topline2026.01 | 85.1 | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 2 [ASR/SQA CE], Evaluation Protocol=in-domain2026.01 | 83.2 | |
| Seamless ASR + Llama 3.1 8BBackbone=Llama 3.1 8B, Input Mode=Seamless ASR, Evaluation Protocol=zero-shot2026.01 | 82.3 | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 2 [ASR/SQA CE], Evaluation Protocol=in-domain2026.01 | 81.6 | |
| BEST-IWSLT25-IFEvaluation Protocol=in-domain2026.01 | 80.7 | |
| Seamless ASR + EuroLLM 9BBackbone=EuroLLM 9B, Input Mode=Seamless ASR, Evaluation Protocol=zero-shot2026.01 | 79.8 | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 2 [ASR CE+MSE], Evaluation Protocol=zero-shot2026.01 | 79.3 | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 2 [ASR CE+MSE], Evaluation Protocol=zero-shot2026.01 | 75.6 | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 1, Evaluation Protocol=zero-shot2026.01 | 70.7 | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 1, Evaluation Protocol=zero-shot2026.01 | 51.9 |