Dialogue State Tracking on MultiWOZ 2.2 (test)
66.25Joint Goal AccuracyLUAS_R+G
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LUAS_R+GBase Model=LLaMA-2-7B, Training Data=Real + Generated Data2024.05 | 66.25 | — | — | — | |
| LUAS_RBase Model=LLaMA-2-7B, Training Data=Real Data Only2024.05 | 65.42 | — | — | — | |
| GEMModel(s)=(BERTbase + GNN) / T5small + Claude 3.7 S.2026.05 | 65.19 | — | — | 97.65 | |
| Single ExpertModel(s)=T5small + Claude 3.7 S.2026.05 | 64.66 | — | — | 97.5 | |
| Single ExpertModel(s)=BERTbase + GNN + Claude 3.7 S.2026.05 | 64.34 | — | — | 97.74 | |
| TOATODbaseBackbone Model=T5base, Trainable Parameters=36M2023.05 | 63.79 | — | — | — | |
| TOATODModel Size=Base2024.05 | 63.79 | — | — | — | |
| TOATODModel(s)=T5base2026.05 | 63.79 | — | — | — | |
| GEMModel(s)=(BERTbase + GNN) / T5small + Llama 3.1 8B2026.05 | 63.57 | — | — | 97.56 | |
| Single ExpertModel(s)=T5small + Llama 3.1 8B2026.05 | 63.39 | — | — | 97.44 | |
| Single ExpertModel(s)=BERTbase + GNN + Llama 3.1 8B2026.05 | 62.66 | — | — | 97.66 | |
| SSNet2024.05 | 62.1 | — | — | — | |
| TOATODsmallBackbone Model=T5small, Trainable Parameters=7.9M2023.05 | 61.92 | — | — | — | |
| TOATODModel Size=Small2024.05 | 61.92 | — | — | — | |
| TOATODModel(s)=T5small2026.05 | 61.92 | — | — | — | |
| DiCoS-DSTk=22022.05 | 61.13 | 98.06 | — | — | |
| DiCoS-DSTk=12022.05 | 61.04 | 98.05 | — | — | |
| DiCoS-DSTk=32022.05 | 60.88 | 98.05 | — | — | |
| D3STXXLBackbone Model=T5XXL, Trainable Parameters=11B2023.05 | 58.7 | — | — | — | |
| D3ST (XXL)Pretrained Model=T5-XXL (11B), Single-Pass=true2023.06 | 58.7 | — | — | — | |
| D3STModel Size=XXL2024.05 | 58.7 | — | — | — | |
| D3STModel(s)=T5xxl2026.05 | 58.7 | — | — | — | |
| DSS-DST2022.05 | 58.04 | 97.66 | — | — | |
| MSP-L2024.05 | 57.7 | — | — | — | |
| Pegasus-DST2022.05 | 57.6 | — | — | — | |
| DST-as-Prompting2022.05 | 57.6 | — | — | — | |
| SDP-DSTBackbone Model=T5base, Trainable Parameters=220M2023.05 | 57.6 | — | — | — | |
| SDP-DST2024.05 | 57.6 | — | — | — | |
| T5 (span)History=Full2021.08 | 57.6 | — | — | — | |
| IndependentPretrained-Model=T5-base, Number of Parameters=220M, Decoding Strategy=Independent, Trained with description=true2021.09 | 57.6 | — | — | — | |
| SPACE-3Backbone Model=UniLM, Trainable Parameters=340M2023.05 | 57.5 | — | — | — | |
| DaP (ind)Pretrained Model=T5-base (220M), Single-Pass=false2023.06 | 57.5 | — | — | — | |
| SPACE-32024.05 | 57.5 | — | — | — | |
| SPACE-3Model(s)=UniLM2026.05 | 57.5 | — | — | — | |
| SPLAT (Large)Pretrained Model=Longformer-large (340M), Single-Pass=true2023.06 | 57.4 | — | 91.5 | — | |
| SPLATModel(s)=Longformerlarge2026.05 | 57.4 | — | — | — | |
| AG-DSTPretrained Model=PLATO-2 (310M), Single-Pass=X‡2023.06 | 57.3 | — | — | — | |
| AG-DST2021.10 | 57.26 | — | — | — | |
| AG-DSTBackbone Model=PLATO-2, Trainable Parameters=310M2023.05 | 57.26 | — | — | — | |
| AG-DST2024.05 | 57.26 | — | — | — | |
| T5 (span+ARLM)History=Full2021.08 | 57.1 | — | — | — | |
| IndependentPretrained-Model=T5-base, Number of Parameters=220M, Decoding Strategy=Independent, Trained with description=false2021.09 | 56.7 | — | — | — | |
| SPLAT (Base)Pretrained Model=Longformer-base (110M), Single-Pass=true2023.06 | 56.6 | — | 91.4 | — | |
| SPLAT2024.05 | 56.6 | — | — | — | |
| PegasusHistory=Full2021.08 | 56.6 | — | — | — | |
| SPLATModel(s)=Longformerbase2026.05 | 56.6 | — | — | — | |
| Diable2024.05 | 56.48 | — | — | — | |
| DiableModel(s)=T5v1.1base2026.05 | 56.48 | — | — | — | |
| IndependentPretrained-Model=T5-small, Number of Parameters=60M, Decoding Strategy=Independent, Trained with description=true2021.09 | 56.3 | — | — | — | |
| LUNAModel(s)=BERTbase2026.05 | 56.13 | — | — | — | |
| D3STbaseBackbone Model=T5base, Trainable Parameters=220M2023.05 | 56.1 | — | — | — | |
| LUNAPretrained Model=BERT-base (110M), Single-Pass=false2023.06 | 56.1 | — | — | — | |
| AG-DSTPretrained Model=GPT-2 (117M), Single-Pass=X†2023.06 | 56.1 | — | — | — | |
| D3ST (Base)Pretrained Model=T5-base (220M), Single-Pass=true2023.06 | 56.1 | — | — | — | |
| D3STModel Size=Base2024.05 | 56.1 | — | — | — | |
| T5 (ARLM)History=Full2021.08 | 56.1 | — | — | — | |
| D3STModel(s)=T5base2026.05 | 56.1 | — | — | — | |
| DSGFNetbackbone=BERT-base-uncased2022.04 | 55.8 | — | — | — | |
| IndependentPretrained-Model=T5-small, Number of Parameters=60M, Decoding Strategy=Independent, Trained with description=false2021.09 | 55.2 | — | — | — | |
| Seq2Seq-DU2021.10 | 54.4 | — | — | — | |
| Seq2Seq-DU2022.04 | 54.4 | — | — | — | |
| Seq2seq-DU2022.05 | 54.4 | — | — | — | |
| Seq2Seq-DUPretrained Model=BERT-base (110M), Single-Pass=true2023.06 | 54.4 | — | 90.9 | — | |
| Seq2Seq-DUHistory=Full2021.08 | 54.4 | — | — | — | |
| Seq2Seq-DUPretrained-Model=BERT-base, Number of Parameters=110M2021.09 | 54.4 | — | — | — | |
| Seq2Seq-DUModel(s)=BERTbase2026.05 | 54.4 | — | — | — | |
| UniLMBackbone Model=UniLM, Trainable Parameters=340M2023.05 | 54.25 | — | — | — | |
| UniLM2024.05 | 54.25 | — | — | — | |
| D3STlargeBackbone Model=T5large, Trainable Parameters=770M2023.05 | 54.2 | — | — | — | |
| D3ST (Large)Pretrained Model=T5-large (770M), Single-Pass=true2023.06 | 54.2 | — | — | — | |
| D3STModel Size=Large2024.05 | 54.2 | — | — | — | |
| D3STModel(s)=T5large2026.05 | 54.2 | — | — | — | |
| SimpleTODimplementation=reproduction results by official code2021.10 | 54.02 | — | — | — | |
| SOM-DSTimplementation=reproduction results by official code2021.10 | 53.81 | — | — | — | |
| TripPy2022.04 | 53.5 | — | — | — | |
| TripPy2024.05 | 53.5 | — | — | — | |
| DS-DSTsource=borrowed from Zang et al. (2020)2021.10 | 51.7 | — | — | — | |
| DS-DST2022.04 | 51.7 | — | — | — | |
| DS-DSTBackbone Model=BERTbase, Trainable Parameters=110M2023.05 | 51.7 | — | — | — | |
| DS-DSTPretrained Model=BERT-base (110M), Single-Pass=false2023.06 | 51.7 | — | — | — | |
| DS-DST2024.05 | 51.7 | — | — | — | |
| DS-DSTHistory=Full2021.08 | 51.7 | — | — | — | |
| DS-DSTPretrained-Model=BERT-base, Number of Parameters=110M2021.09 | 51.7 | — | — | — | |
| DS-DSTModel(s)=BERTbase2026.05 | 51.7 | — | — | — | |
| DaP (seq)Pretrained Model=T5-base (220M), Single-Pass=true2023.06 | 51.2 | — | — | — | |
| SequentialPretrained-Model=T5-base, Number of Parameters=220M, Decoding Strategy=Sequential2021.09 | 51.2 | — | — | — | |
| SUMBTHistory=Full2021.08 | 49.7 | — | — | — | |
| SequentialPretrained-Model=T5-small, Number of Parameters=60M, Decoding Strategy=Sequential2021.09 | 48.9 | — | — | — | |
| TRADEPretrained-Model=N2021.09 | 48.6 | — | — | — | |
| TRADEsource=borrowed from Zang et al. (2020)2021.10 | 45.4 | — | — | — | |
| TRADEdataset preprocessing=original MultiWOZ2022.04 | 45.4 | — | — | — | |
| TRADE2022.05 | 45.4 | — | — | — | |
| TRADE2023.05 | 45.4 | — | — | — | |
| TRADE2024.05 | 45.4 | — | — | — | |
| TRADEHistory=Full2021.08 | 45.4 | — | — | — | |
| SGD baselinesource=borrowed from Zang et al. (2020)2021.10 | 42 | — | — | — | |
| SGD-baseline2022.04 | 42 | — | — | — | |
| No pre-trainHistory=Full, Pre-training=false2021.08 | 26.1 | — | — | — |