Dialogue State Tracking on MultiWOZ 2.4 (test)
78.2Joint Goal AccLUAS_R+G
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LUAS_R+GBase Model=LLaMA-2-7B, Training Data=Real + Generated Data2024.05 | 78.2 | — | — | — | — | |
| LUAS_RBase Model=LLaMA-2-7B, Training Data=Real Data Only2024.05 | 77.2 | — | — | — | — | |
| MoNETPre-trained Model=BERT-base2022.11 | 76.02 | 98.99 | — | — | — | |
| MONET2024.05 | 76.02 | — | — | — | — | |
| D3STModel Size=XXL2024.05 | 75.9 | — | — | — | — | |
| DDSA2024.05 | 75.58 | — | — | — | — | |
| BaselinePre-trained Model=BERT-base2022.11 | 73.81 | 98.82 | — | — | — | |
| STARPre-trained Model=BERT-base2022.11 | 73.62 | 98.85 | — | — | — | |
| D3STModel Size=Base2024.05 | 72.1 | — | — | — | — | |
| D3STModel Size=Large2024.05 | 70.8 | — | — | — | — | |
| Diable2024.05 | 70.46 | — | — | — | — | |
| T5 (span)History=Full2021.08 | 67.1 | — | — | — | — | |
| SOM-DSTHistory=Full2021.08 | 66.8 | — | — | — | — | |
| SOM-DSTPre-trained Model=BERT-base2022.11 | 66.78 | 98.38 | — | — | — | |
| PegasusHistory=Full2021.08 | 66.6 | — | — | — | — | |
| T5 (span+ARLM)History=Full2021.08 | 65.6 | — | — | — | — | |
| RefPyDST2024.05 | 65.2 | — | — | — | — | |
| TripPy2024.05 | 64.75 | — | — | — | — | |
| T5 (ARLM)History=Full2021.08 | 63 | — | — | — | — | |
| SUMBTHistory=Full2021.08 | 61.9 | — | — | — | — | |
| SUMBTPre-trained Model=BERT-base2022.11 | 61.86 | 97.9 | — | — | — | |
| SAVNPre-trained Model=BERT-base2022.11 | 60.55 | 98.05 | — | — | — | |
| SAVNHistory=Full2021.08 | 60.1 | — | — | — | — | |
| TripPyHistory=Full2021.08 | 59.6 | — | — | — | — | |
| PIN2022.11 | 58.92 | 98.02 | — | — | — | |
| LDSTParam. size=<100B, Data ratio=5%, Training dataset=MultiWOZ 2.22024.06 | 56.48 | — | — | — | — | |
| IC-DSTParam. size=>100B, Data ratio=5%, Backbone=CodeX2024.06 | 55.43 | — | — | — | — | |
| TRADEHistory=Full2021.08 | 55.1 | — | — | — | — | |
| TRADE2022.11 | 55.05 | 97.62 | — | — | — | |
| TRADE2024.05 | 55.05 | — | — | — | — | |
| SVAG + EDZ-DAParam. size=<1B, Data ratio=5%2024.06 | 54.09 | — | — | — | — | |
| DS2 + EDZ-DAParam. size=<1B, Data ratio=5%2024.06 | 51.54 | — | — | — | — | |
| SM2-11BParam. size=<100B, Data ratio=5%2024.06 | 51.14 | — | — | — | — | |
| SVAGParam. size=<1B, Data ratio=5%2024.06 | 50.12 | — | — | — | — | |
| DS2Param. size=<1B, Data ratio=5%2024.06 | 49.89 | — | — | — | — | |
| SM2-3BParam. size=<100B, Data ratio=5%2024.06 | 49.22 | — | — | — | — | |
| IC-DSTParam. size=>100B, Data ratio=1%, Backbone=CodeX2024.06 | 48.35 | — | — | — | — | |
| LDSTParam. size=<100B, Data ratio=1%, Training dataset=MultiWOZ 2.22024.06 | 46.77 | — | — | — | — | |
| SVAG + EDZ-DAParam. size=<1B, Data ratio=1%2024.06 | 43.82 | — | — | — | — | |
| SM2-11BParam. size=<100B, Data ratio=1%2024.06 | 40.03 | — | — | — | — | |
| DS2 + EDZ-DAParam. size=<1B, Data ratio=1%2024.06 | 38.99 | — | — | — | — | |
| SM2-3BParam. size=<100B, Data ratio=1%2024.06 | 37.59 | — | — | — | — | |
| DS2Param. size=<1B, Data ratio=1%2024.06 | 36.76 | — | — | — | — | |
| SVAGParam. size=<1B, Data ratio=1%2024.06 | 35.53 | — | — | — | — | |
| No pre-trainHistory=Full, Pre-training=false2021.08 | 26.7 | — | — | — | — | |
| CombiSearchParadigm=ICL, Language Model=gpt-3.5-turbo2025.05 | — | — | 56.7 | 59.8 | 64.2 | |
| DS2Paradigm=Fine-tuned, Language Model=T5-large2025.05 | — | — | 36.8 | 49.9 | 57.9 | |
| IC-DSTParadigm=ICL, Language Model=code-davinci-0022025.05 | — | — | 50.7 | 48.4 | 55.4 | |
| RefPyDSTParadigm=ICL, Language Model=gpt-3.5-turbo2025.05 | — | — | 44.9 | 52.3 | 58 | |
| SM2-11BParadigm=Fine-tuned, Language Model=T5-XXL2025.05 | — | — | 40 | 51.1 | — | |
| SynthDSTParadigm=ICL, Language Model=gpt-3.5-turbo2025.05 | — | — | 51 | 50.4 | 55.2 |