Task-oriented Dialogue on StarV2 (test)
82.5Bank ScoreGenie + GPT-4o-mini
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Genie + GPT-4o-miniEvaluation Protocol=Zeroshot, Backbone=GPT-4o-mini2024.07 | 82.5 | 80.5 | 90.3 | |
| Genie + GPT-4 TurboEvaluation Protocol=Zeroshot, Backbone=GPT-4 Turbo2024.07 | 82.5 | 83.4 | 92.7 | |
| Genie + Llama 3.1 70BEvaluation Protocol=Zeroshot, Backbone=Llama 3.1 70B2024.07 | 82.1 | 75.9 | 82.2 | |
| AT-PROG +SGD XXLEvaluation Protocol=Finetuned, Model Size=11B, Policy=Programmable, Additional Finetuning=SGD2024.07 | 65 | 62.9 | 86.3 | |
| AT-PROG XXLEvaluation Protocol=Finetuned, Model Size=11B, Policy=Programmable2024.07 | 61 | 60.8 | 73.7 | |
| GPT-4 Turbo (FC)Evaluation Protocol=Zeroshot, Function Calling (FC)=true2024.07 | 55.1 | 42.7 | 82.5 | |
| AT XXLEvaluation Protocol=Finetuned, Model Size=11B2024.07 | 54.3 | 52.4 | 73.8 | |
| AT-SGD XXLEvaluation Protocol=Finetuned, Model Size=11B, Additional Finetuning=Schema Guided Dialog (SGD)2024.07 | 53.1 | 51.5 | 81.1 | |
| GPT-4o-mini (FC)Evaluation Protocol=Zeroshot, Function Calling (FC)=true2024.07 | 50.8 | 43.8 | 69.8 | |
| Llama 3.1 70B (FC)Evaluation Protocol=Zeroshot, Function Calling (FC)=true2024.07 | 48.9 | 41.7 | 81.7 |