Slot Filling on SLURP Whisper Medium transcriptions (test)
66.5SLU F1 ScoreFlan-T5-base
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Flan-T5-baseEvaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 66.5 | 76.7 | 58.8 | |
| GPT-2Evaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 65.2 | 74.1 | 58.2 | |
| GPT-4Evaluation Protocol=one-shot2023.11 | 47 | 43.7 | 50.9 | |
| GPT-3.5Evaluation Protocol=one-shot2023.11 | 40.8 | 36.3 | 46.7 | |
| Vicuna-13B-v1.5Evaluation Protocol=one-shot2023.11 | 19.9 | 17.3 | 23.3 | |
| Vicuna-13BEvaluation Protocol=one-shot2023.11 | 8.9 | 5.8 | 19.3 |