Dialogue State Tracking on SGD 15 tasks CL
76.3Avg JGARoS Multi-task
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RoS Multi-taskBackbone=T5-small, Protocol=Multi-task (Upper Bound)2024.08 | 76.3 | — | — | |
| DST-EGQA Multi-taskBackbone=T5-small, Protocol=Multi-task (Upper Bound)2024.08 | 74.2 | — | — | |
| RoSMemory Free=false, Backbone=T5-small2024.08 | 72.1 | 26.7 | -2.6 | |
| DST-EGQAMemory Free=false, Backbone=T5-small2024.08 | 68.9 | 22.5 | -5.9 | |
| Multi-task Prompt TuningMemory=-, +Params=0.1%, Tune Params=0.1%2022.03 | 64 | — | — | |
| CPT Multi-taskBackbone=T5-small, Protocol=Multi-task (Upper Bound)2024.08 | 64 | — | — | |
| TaSLMemory Free=true, Backbone=T5-small2024.08 | 62.1 | 26.6 | -9.1 | |
| Continual Prompt Tuning w/ memory & backwardMemory=|M|*T2022.03 | 61.2 | 13.7 | 0.5 | |
| CPTMemory Free=false, Backbone=T5-small2024.08 | 61.2 | 13.7 | 0.5 | |
| Continual Prompt Tuning w/ memoryMemory=|M|*T2022.03 | 60.7 | 13.7 | 0 | |
| Continual Prompt TuningMemory=-, +Params=0.1%*T, Tune Params=0.1%2022.03 | 59.5 | 9.9 | 0 | |
| RoSMemory Free=true, Backbone=T5-small2024.08 | 59 | 25.5 | -17.9 | |
| ReplayMemory=|M|*T, +Params=0, Tune Params=12022.03 | 58.6 | 10.9 | -3.2 | |
| ReplayMemory Free=false, Backbone=T5-small2024.08 | 58.6 | 10.9 | -3.2 | |
| DST-EGQAMemory Free=true, Backbone=T5-small2024.08 | 55.5 | 23.6 | -19.1 | |
| AdapterCL (20x)Memory=-, +Params=2%*T, Tune Params=2%2022.03 | 49.8 | — | — | |
| AdapterCLMemory Free=true, Backbone=T5-small2024.08 | 49.8 | — | — | |
| Prompt TuningMemory=-2022.03 | 48.1 | — | — | |
| EWCMemory Free=true, Backbone=T5-small2024.08 | 47.9 | 8.4 | -38.1 | |
| Fine-tuningMemory Free=true, Backbone=T5-small2024.08 | 44.1 | 8.3 | -36.6 | |
| AdapterCL (1x)Memory=-, +Params=0.1%*T, Tune Params=0.1%2022.03 | 30.6 | — | — | |
| Fine-tuningMemory=-, +Params=0, Tune Params=12022.03 | 14.3 | 8.3 | -49.9 | |
| EWCMemory=|M|*T, +Params=2, Tune Params=12022.03 | 13.9 | 8.4 | -50.8 |