Slot Filling on SLURP Whisper Base transcriptions (test)
59.1SLU F1 ScoreFlan-T5-base
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Flan-T5-baseEvaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 59.1 | 72.2 | 50.8 | |
| GPT-2Evaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 58.4 | 70.6 | 49.9 | |
| GPT-4Evaluation Protocol=one-shot2023.11 | 42.3 | 42.1 | 42.5 | |
| GPT-3.5Evaluation Protocol=one-shot2023.11 | 36.7 | 34.1 | 39.7 | |
| Vicuna-13B-v1.5Evaluation Protocol=one-shot2023.11 | 17.8 | 16.3 | 19.6 | |
| Vicuna-13BEvaluation Protocol=one-shot2023.11 | 7.7 | 5 | 17 |