Task-oriented Dialogue Response Generation on Multi-WOZ 2.1 (test)
20.01BLEUGALAXY
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GALAXYBackbone=UniLM, Trainable Parameters=340M2023.05 | 20.01 | — | — | — | — | — | 95.3 | 86.2 | 110.76 | |
| MTTODBackbone=T5-base, Trainable Parameters=360.9M2023.05 | 19.68 | — | — | — | — | — | 90.99 | 82.08 | 106.22 | |
| RSTODBackbone=T5-small, Trainable Parameters=105.5M2023.05 | 19.24 | — | — | — | — | — | 93.5 | 84.7 | 108.34 | |
| PPTODBackbone=T5-base, Trainable Parameters=220M2023.05 | 19.17 | — | — | — | — | — | 87.09 | 79.08 | 102.26 | |
| TOATOD_smallBackbone=T5-small, Trainable Parameters=7.9M2023.05 | 18.29 | — | — | — | — | — | 92.1 | 80.4 | 104.54 | |
| TOATOD_baseBackbone=T5-base, Trainable Parameters=36M2023.05 | 17.12 | — | — | — | — | — | 97 | 87.4 | 109.32 | |
| UBARBackbone=GPT2, Trainable Parameters=1.5B2023.05 | 16.5 | — | — | — | — | — | 95.7 | 81.8 | 104.94 | |
| SimpleTODBelief State=generated, DB Search=oracle, Action=generated2020.05 | 16.01 | — | — | — | — | — | 79.3 | 65.4 | 87.36 | |
| DOTSBackbone=BERT-base, Trainable Parameters=110M2023.05 | 15.9 | — | — | — | — | — | 86.65 | 74.18 | 96.32 | |
| SimpleTODBelief State=generated, DB Search=none, Action=generated2020.05 | 15.23 | — | — | — | — | — | 85 | 70.5 | 92.98 | |
| SimpleTODBelief State=generated, DB Search=dynamic, Action=generated2020.05 | 14.99 | — | — | — | — | — | 83.4 | 67.1 | 90.24 | |
| NS-Dialmaximum depth of HRE module (D)=32022.03 | 10.6 | 37.2 | 42.6 | 30.6 | 33.7 | — | — | — | — | |
| NS-Dialmaximum depth of HRE module (D)=12022.03 | 9.7 | 36.5 | 42 | 29.7 | 32.8 | — | — | — | — | |
| DF-Net2022.03 | 9.4 | 35.1 | 40.9 | 28.1 | 30.6 | — | — | — | — | |
| NS-Dialmaximum depth of HRE module (D)=52022.03 | 9.3 | 36.2 | 41.7 | 28.8 | 31.5 | — | — | — | — | |
| COMET2020.10 | 8.3 | 18.6 | 27.5 | 17.9 | 15.2 | 9.8 | — | — | — | |
| GLMP2022.03 | 6.9 | 32.4 | 38.4 | 24.4 | 28.1 | — | — | — | — | |
| GraphDialog2022.03 | 6.7 | 34.1 | 39.2 | 27.8 | 29.6 | — | — | — | — | |
| Mem2Seq2022.03 | 6.6 | 21.6 | 22.4 | 22 | 21 | — | — | — | — | |
| GraphDialogKnowledge Representation=Graph2020.10 | 6.2 | 11.3 | 16 | 14.1 | 10.8 | 4.4 | — | — | — | |
| GLMPKnowledge Representation=Triplet2020.10 | 4.3 | 6.7 | 11.4 | 9.4 | 3.9 | 3.5 | — | — | — | |
| Mem2SeqKnowledge Representation=Triplet2020.10 | 4.1 | 3.2 | 2.9 | 2.1 | 4.5 | 1.5 | — | — | — |