Intent Prediction on PhotoChat (test)
65.6F1 ScoreDRIBER ChatGPT 0613
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DRIBER ChatGPT 0613Training Type=Zero-shot, LLM Backbone=ChatGPT 06132023.10 | 65.6 | 66.7 | 64.7 | |
| DRIBER Vicuna-13BTraining Type=Zero-shot, LLM Backbone=Vicuna-13B2023.10 | 65.5 | 64.3 | 67.1 | |
| DRIBER ChatGPT 1106Training Type=Zero-shot, LLM Backbone=ChatGPT 11062023.10 | 64 | 63.1 | 65.1 | |
| PaCETotal parameter count=338 million2023.05 | 63.8 | 63.3 | 68 | |
| PaCETraining Type=Fine-tuned2023.10 | 63.8 | 63.3 | 68 | |
| DRIBER Falcon-Instruct-40BTraining Type=Zero-shot, LLM Backbone=Falcon-Instruct-40B2023.10 | 62.8 | 62.4 | 63.3 | |
| T5-3B2021.10 | 58.9 | — | — | |
| T5-3BParameter count=3B2023.05 | 58.9 | 54.1 | 64.6 | |
| T5-3BTraining Type=Fine-tuned2023.10 | 58.9 | 54.1 | 64.6 | |
| T5-baseModel variant=base2023.05 | 58.1 | 58.2 | 57.9 | |
| T5-baseTraining Type=Fine-tuned2023.10 | 58.1 | 58.2 | 57.9 | |
| Divterjoint learning=true, G pre-train=true, F pre-train=true2021.10 | 56.2 | — | — | |
| DivterF pre-train=false2021.10 | 55.9 | — | — | |
| Divterjoint learning=false2021.10 | 55.6 | — | — | |
| BERT-base2021.10 | 53.2 | — | — | |
| BERT-baseModel variant=base2023.05 | 53.2 | 56.1 | 50.6 | |
| BERT-baseTraining Type=Fine-tuned2023.10 | 53.2 | 56.1 | 50.6 | |
| ViLT2023.05 | 52.4 | 55.4 | 58.9 | |
| ViLTTraining Type=Fine-tuned2023.10 | 52.4 | 55.4 | 58.9 | |
| ALBERT-baseModel variant=base2023.05 | 52.2 | 44.8 | 62.7 | |
| ALBERT-baseTraining Type=Fine-tuned2023.10 | 52.2 | 44.8 | 62.7 | |
| S2S-TF2021.10 | 47.6 | — | — | |
| DivterG pre-train=false2021.10 | 47.3 | — | — | |
| DivterG pre-train=false, F pre-train=false2021.10 | 47.1 | — | — | |
| DRIBER GPT-4 1106Training Type=Zero-shot, LLM Backbone=GPT-4 11062023.10 | 45.6 | 56.5 | 68.1 | |
| DRIBER LLaMa2-Chat-70BTraining Type=Zero-shot, LLM Backbone=LLaMa2-Chat-70B2023.10 | 40.9 | 55 | 63.3 |