Dialogue State Tracking on DSTC2 (test)
85Joint Goal AccuracySeq2seq-DU
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Seq2seq-DU2022.05 | 85 | — | — | — | — | — | |
| DiCoS-DSTk=22022.05 | 78.4 | — | — | — | — | — | |
| DiCoS-DSTk=12022.05 | 77.7 | — | — | — | — | — | |
| DiCoS-DSTk=32022.05 | 77.3 | — | — | — | — | — | |
| StateNet_PSI2018.10 | 75.5 | — | — | — | — | — | |
| StateNet-PSIHistory representation=full history, Prediction selection protocol=n-best predictions2021.08 | 75.5 | — | — | — | — | — | |
| GLAD2018.05 | 74.5 | 97.5 | — | — | — | — | |
| GLAD2018.10 | 74.5 | — | — | — | — | — | |
| StateNet_PS2018.10 | 74.5 | — | — | — | — | — | |
| GLADHistory representation=full history, Prediction selection protocol=n-best predictions2021.08 | 74.5 | — | — | — | — | — | |
| StateNet2018.10 | 74.1 | — | — | — | — | — | |
| MemN2N2018.10 | 74 | — | — | — | — | — | |
| Pegasus-DST2022.05 | 73.6 | — | — | — | — | — | |
| PegasusHistory representation=full history, Prediction selection protocol=single best prediction2021.08 | 73.6 | — | — | — | — | — | |
| T5 (span)History representation=full history, Prediction selection protocol=single best prediction2021.08 | 73.6 | — | — | — | — | — | |
| T5 (span+ARLM)History representation=full history, Prediction selection protocol=single best prediction2021.08 | 73.5 | — | — | — | — | — | |
| NBT-CNNArchitecture=Neural Belief Tracker, Word Vectors=Paragram-SL9992016.06 | 73.4 | 96.5 | — | — | — | — | |
| Neural Belief Tracker (NBT)architecture=CNN2018.05 | 73.4 | 96.5 | — | — | — | — | |
| Neural Belief Tracker: NBT-CNN2018.10 | 73.4 | — | — | — | — | — | |
| Neural Belief Tracker: NBT-CNNHistory representation=full history, Prediction selection protocol=n-best predictions2021.08 | 73.4 | — | — | — | — | — | |
| T5 (ARLM)History representation=full history, Prediction selection protocol=single best prediction2021.08 | 73.3 | — | — | — | — | — | |
| Delexicalisation-Based Model + Semantic Dictionary2016.06 | 72.9 | 95.7 | — | — | — | — | |
| Delex. Model + Semantic Dictionarysemantic_dictionary=true2018.05 | 72.9 | 95.7 | — | — | — | — | |
| DB Model + Semantic Dictionary2018.10 | 72.9 | — | — | — | — | — | |
| NBT-DNNArchitecture=Neural Belief Tracker, Word Vectors=Paragram-SL9992016.06 | 72.6 | 96.4 | — | — | — | — | |
| Neural Belief Tracker (NBT)architecture=DNN2018.05 | 72.6 | 96.4 | — | — | — | — | |
| Neural Belief Tracker: NBT-DNN2018.10 | 72.6 | — | — | — | — | — | |
| Neural Belief Tracker: NBT-DNNHistory representation=full history, Prediction selection protocol=n-best predictions2021.08 | 72.6 | — | — | — | — | — | |
| PtrNet2018.10 | 72.1 | — | — | — | — | — | |
| NBT2018.04 | 72 | — | 90 | 84 | 94 | — | |
| Our SL modeltraining_mode=supervised learning (pre-training)2018.04 | 72 | — | 90 | 84 | 92 | — | |
| RNN+sem. dict2018.04 | 71 | — | 92 | 86 | 92 | — | |
| Scalable Multi-domain DST2018.10 | 70.3 | — | — | — | — | — | |
| BERT-DST2022.05 | 69.3 | — | — | — | — | — | |
| Delexicalisation-Based Model2016.06 | 69.1 | 95.7 | — | — | — | — | |
| Delexicalisation-Based Model2018.05 | 69.1 | 95.7 | — | — | — | — | |
| Delexicalisation-Based (DB) Model2018.10 | 69.1 | — | — | — | — | — | |
| RNN2018.04 | 69 | — | 92 | 86 | 86 | — | |
| No pre-trainHistory representation=full history, Prediction selection protocol=single best prediction2021.08 | 50.1 | — | — | — | — | — | |
| Baseline2019.07 | — | — | — | — | — | 95 | |
| STMDescription=Simple Time Mask2019.07 | — | — | — | — | — | 96.1 | |
| TDA2019.07 | — | — | — | — | — | 94.6 |