End-to-end Task-oriented Dialogue on MultiWOZ 2.0 (test)
97.5Inform AccuracyLAVA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LAVAbelief_states=oracle2021.11 | 97.5 | 94.8 | 12.1 | 108.25 | |
| HDNObelief_states=oracle2021.11 | 96.4 | 84.7 | 18.85 | 109.4 | |
| GALAXYbelief_states=oracle2021.11 | 94.8 | 85.7 | 19.93 | 110.18 | |
| JOUSTbelief_states=oracle2021.11 | 94.7 | 86.7 | 18.7 | 109.4 | |
| UBARbelief_states=oracle2021.11 | 94 | 83.6 | 17.2 | 106 | |
| GALAXY(w/o pre-train)belief_states=oracle, initialization=original weights of UniLM2021.11 | 93.6 | 82.6 | 18.6 | 106.7 | |
| GALAXYsetting=end-to-end, pre-training=without2023.05 | 93.1 | 81 | 18.44 | 105.49 | |
| MDTODsetting=end-to-end2023.05 | 92.7 | 85 | 19.72 | 108.57 | |
| MarCobelief_states=oracle2021.11 | 92.3 | 78.6 | 20.02 | 105.47 | |
| MTTODsetting=end-to-end, reproducibility=reproduced results2023.05 | 91.8 | 83.8 | 19.56 | 107.36 | |
| PPTODsetting=end-to-end2023.05 | 89.2 | 79.4 | 18.62 | 102.92 | |
| SimpleTODbelief_states=oracle2021.11 | 88.9 | 67.1 | 16.9 | 94.9 | |
| DOTSsetting=end-to-end2023.05 | 86.59 | 74.14 | 15.06 | 95.43 | |
| SOLOISTsetting=end-to-end2023.05 | 85.5 | 72.9 | 16.54 | 95.74 | |
| UBARsetting=end-to-end, result_source=Su et al., 20222023.05 | 85.1 | 71.02 | 16.21 | 94.27 | |
| MinTLsetting=end-to-end2023.05 | 84.88 | 74.91 | 17.89 | 97.79 | |
| SimpleTODBelief State=generated, Action=generated2020.05 | 84.4 | 70.1 | 15.01 | 92.26 | |
| SimpleTODsetting=end-to-end2023.05 | 84.4 | 70.1 | 15.01 | 92.26 | |
| SimpleTODBelief State=generated, DB Search=dynamic, Action=generated2020.05 | 81.4 | 69.7 | 16.11 | 91.66 | |
| SimpleTODBelief State=generated, DB Search=oracle, Action=generated2020.05 | 78.1 | 63.4 | 16.91 | 87.66 | |
| DAMDsetting=end-to-end2023.05 | 76.33 | 60.4 | 16.6 | 84.97 | |
| DAMD+augmentationBelief State=generated, DB Search=oracle, Action=generated2020.05 | 76.3 | 60.4 | 16.6 | 85 |