SGD
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SGD
55.01JGA (Overall)
24
SGD 15 tasks CL
76.3Avg JGA
23
SGD (test)
28.6BLEU
18
SGD
64.8Accuracy
14
FewShotSGD unseen schemata (test)
28.76BLEU
13
FewShotSGD seen schemata (test)
29.28BLEU
13
SGD (test)
86.5JGA
11
SGD (train)
55.99JGA
9
SGD Messaging
67.79JGA
9
SGD (test)
46.8Purity
9
SGD
80.68Joint GA
9
SGD 5% training size (test)
75.3Precision
8
SGD Media
76.2JGA
7
SGD 1.0 (test)
81.29Inform Rate
6
SGD Real (test)
0.559AMI
6
SGD Real (test)
66.3Class-Balanced Acc (Full)
6
SGD-X v1-v5 variants (test)
86.4Joint Goal Acc (Original)
6
SGD Music
35.46JGA
5
SGD Flights
30.57Joint Goal Accuracy (JGA)
5
SGD (test)
50.4Inform Rate
5
SGD to MultiWoz (test)
51.2Average JGA
5
SGD
92.5F1 Score
4
SGD All Domains (test)
32.1Joint GA
4
SGD Unseen Domains (test)
24.4Joint GA
4
SGD (Overall)
2.46Naturalness
4