Slot Filling on SLURP Whisper Tiny transcriptions (test)
56.4SLU F1 ScoreFlan-T5-base
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Flan-T5-baseEvaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 56.4 | 69.2 | 47.6 | |
| GPT-2Evaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 54.9 | 66 | 47 | |
| GPT-4Evaluation Protocol=one-shot2023.11 | 41.3 | 43.5 | 39.3 | |
| GPT-3.5Evaluation Protocol=one-shot2023.11 | 34.7 | 32 | 38 | |
| Vicuna-13B-v1.5Evaluation Protocol=one-shot2023.11 | 16.6 | 14.9 | 18.8 | |
| Vicuna-13BEvaluation Protocol=one-shot2023.11 | 7.6 | 4.5 | 20.1 |