Task-oriented Dialogue on Stanford Multi-Domain Dialogue (SMD) (test)
17.3BLEUCOMET
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| COMET2020.10 | 17.3 | 63.6 | 77.6 | 58.3 | 56 | — | — | — | |
| NS-Dialmaximum depth of HRE module (D)=32022.03 | 15.6 | 64.5 | — | 59.2 | 60.3 | — | — | 75.6 | |
| NS-Dialmaximum depth of HRE module (D)=12022.03 | 14.9 | 63.8 | — | 58.7 | 60.1 | — | — | 75 | |
| GLMP K3K (Number of hops)=32019.01 | 14.79 | 59.97 | 69.56 | 62.58 | 52.98 | — | — | — | |
| NS-Dialmaximum depth of HRE module (D)=52022.03 | 14.5 | 63.5 | — | 57.9 | 59.4 | — | — | 74.8 | |
| DF-NetKnowledge Representation=Triplet2020.10 | 14.4 | 62.7 | 73.1 | 57.6 | 57.9 | — | — | — | |
| DF-Net2022.03 | 14.4 | 62.7 | — | 57.6 | 57.9 | — | — | 73.1 | |
| GraphDialog2022.03 | 14.2 | 61.1 | — | 56.9 | 56.4 | — | — | 72.1 | |
| KB-TransformerKnowledge Representation=Triplet2020.10 | 13.9 | 37.1 | 51.2 | 48.2 | 23.3 | — | — | — | |
| KB-RetrieverKnowledge Representation=Row-entity2020.10 | 13.9 | 53.7 | 55.6 | 52.2 | 54.5 | — | — | — | |
| GLMPKnowledge Representation=Triplet2020.10 | 13.9 | 60.7 | 72.9 | 56.5 | 54.6 | — | — | — | |
| GLMP2022.03 | 13.9 | 60.7 | — | 56.5 | 54.6 | — | — | 72.5 | |
| GLMP K1K (Number of hops)=12019.01 | 13.83 | 57.25 | 68.74 | 60.87 | 48.62 | — | — | — | |
| GraphDialogKnowledge Representation=Graph2020.10 | 13.7 | 60.7 | 72.8 | 55.2 | 54.2 | — | — | — | |
| Human Performance2017.05 | 13.5 | 60.7 | 64.3 | 61.6 | 55.2 | — | — | — | |
| KV Retrieval Netencoder attention=true2017.05 | 13.2 | 48 | 62.9 | 47 | 41.3 | — | — | — | |
| KVRCanonicalized forms=true2019.01 | 13.2 | 48 | 62.9 | 47 | 41.3 | — | — | — | |
| Mem2Seq2019.01 | 12.6 | 33.4 | 49.3 | 32.8 | 20 | 3.89 | 3.8 | — | |
| Mem2SeqKnowledge Representation=Triplet2020.10 | 12.6 | 33.4 | 49.3 | 32.8 | 20 | — | — | — | |
| Mem2Seq2022.03 | 12.6 | 33.4 | — | 32.8 | 20 | — | — | 49.3 | |
| GLMP K6K (Number of hops)=62019.01 | 12.37 | 53.54 | 69.38 | 55.89 | 43.08 | — | — | — | |
| Copy Net2017.05 | 11 | 37 | 28.1 | 50.1 | 28.4 | — | — | — | |
| KV Retrieval Netencoder attention=false2017.05 | 10.8 | 40.9 | 59.5 | 35.6 | 36.6 | — | — | — | |
| Attn. Seq2Seqencoder attention=true2017.05 | 10.2 | 30 | 30 | 42.4 | 17.9 | — | — | — | |
| S2S + Attn2019.01 | 9.3 | 19.9 | 23.4 | 25.6 | 10.8 | — | — | — | |
| S2S2019.01 | 8.4 | 10.3 | 9.7 | 14.1 | 7 | — | — | — | |
| Ptr-Unk2019.01 | 8.3 | 22.7 | 26.9 | 26.7 | 14.9 | — | — | — | |
| Rule-Based2017.05 | 6.6 | 43.8 | 61.3 | 39.5 | 40.4 | — | — | — | |
| Rule-BasedCanonicalized forms=true2019.01 | 6.6 | 43.8 | 61.3 | 39.5 | 40.4 | — | — | — | |
| GLMP2019.01 | — | — | — | — | — | 4.15 | 4.02 | — | |
| Human2019.01 | — | — | — | — | — | 4.6 | 4.54 | — |