Spoken Question Answering on LibriSQA Part II
74.9AccuracyTranscripts + Llama 3.1 8B
Evaluation results
| Method | Links | |
|---|---|---|
| Transcripts + Llama 3.1 8BBackbone=Llama 3.1 8B, Input Mode=Transcripts, Evaluation Protocol=topline2026.01 | 74.9 | |
| Seamless ASR + Llama 3.1 8BBackbone=Llama 3.1 8B, Input Mode=Seamless ASR, Evaluation Protocol=zero-shot2026.01 | 74.7 | |
| Seamless ASR + EuroLLM 9BBackbone=EuroLLM 9B, Input Mode=Seamless ASR, Evaluation Protocol=zero-shot2026.01 | 73.5 | |
| Transcripts + EuroLLM 9BBackbone=EuroLLM 9B, Input Mode=Transcripts, Evaluation Protocol=topline2026.01 | 73.4 | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 2 [ASR/SQA CE], Evaluation Protocol=in-domain2026.01 | 72.5 | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 1, Evaluation Protocol=zero-shot2026.01 | 70.5 | |
| SpeechMapperBackbone=Llama 3.1, Training Stage=Stage 2 [ASR CE+MSE], Evaluation Protocol=zero-shot2026.01 | 68.9 | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 2 [ASR/SQA CE], Evaluation Protocol=in-domain2026.01 | 68.1 | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 2 [ASR CE+MSE], Evaluation Protocol=zero-shot2026.01 | 64.3 | |
| BEST-IWSLT25-IFEvaluation Protocol=in-domain2026.01 | 62.5 | |
| SpeechMapperBackbone=EuroLLM, Training Stage=Stage 1, Evaluation Protocol=zero-shot2026.01 | 60.3 |