Slot Filling on SLURP Reference transcriptions (test)
86.3SLU F1 ScoreFlan-T5-base
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Flan-T5-baseEvaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 86.3 | 87.1 | 85.5 | |
| GPT-2Evaluation Protocol=fine-tuned, Training Data=Full SLURP2023.11 | 84.8 | 85.5 | 84.1 | |
| GPT-4Evaluation Protocol=one-shot2023.11 | 53.6 | 49.4 | 58.6 | |
| GPT-3.5Evaluation Protocol=one-shot2023.11 | 46.5 | 40.5 | 54.5 | |
| Vicuna-13B-v1.5Evaluation Protocol=one-shot2023.11 | 22.4 | 18.4 | 28.5 | |
| Vicuna-13BEvaluation Protocol=one-shot2023.11 | 16.5 | 11.5 | 29.5 |