Intent Recognition on OOS (test)
87.2Overall AccuracyFutureTOD
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FutureTODShot Setting=Full (100-shot)2023.06 | 87.2 | 96 | 90 | 47.6 | |
| TOD-BERTShot Setting=Full (100-shot)2023.06 | 86.6 | 96.2 | 89.9 | 43.6 | |
| BERT-mlmShot Setting=Full (100-shot)2023.06 | 85.9 | 96.1 | 89.5 | 46.3 | |
| BERTShot Setting=Full (100-shot)2023.06 | 84.9 | 95.8 | 88.1 | 35.6 | |
| DSEShot Setting=Full (100-shot)2023.06 | 84.3 | 95.8 | 87.7 | 32.5 | |
| DialoGPTShot Setting=Full (100-shot)2023.06 | 83.9 | 95.5 | 87.6 | 32.1 | |
| SimCSEShot Setting=Full (100-shot)2023.06 | 82.3 | 94.7 | 86.6 | 26.6 | |
| FutureTODShot Setting=10-Shot2023.06 | 78.1 | 90.8 | 85.5 | 20.5 | |
| DSEShot Setting=10-Shot2023.06 | 77.8 | 90.8 | 85.2 | 19.1 | |
| TOD-BERTShot Setting=10-Shot2023.06 | 77.3 | 91 | 84.5 | 15.3 | |
| BERT-mlmShot Setting=10-Shot2023.06 | 76.6 | 90.5 | 84.3 | 14 | |
| BERTShot Setting=10-Shot2023.06 | 75.5 | 88.6 | 84.7 | 16.5 | |
| SimCSEShot Setting=10-Shot2023.06 | 74.5 | 88.9 | 83.5 | 9.6 | |
| FutureTODShot Setting=1-Shot2023.06 | 43.1 | 52.2 | 81.8 | 2.1 | |
| TOD-BERTShot Setting=1-Shot2023.06 | 42.5 | 52 | 81.7 | 0.1 | |
| DSEShot Setting=1-Shot2023.06 | 42.3 | 51.7 | 81.8 | 0.4 | |
| BERT-mlmShot Setting=1-Shot2023.06 | 38.9 | 47.4 | 81.6 | 0.5 | |
| SimCSEShot Setting=1-Shot2023.06 | 29.9 | 36.4 | 81.7 | 0.6 | |
| BERTShot Setting=1-Shot2023.06 | 29.3 | 35.7 | 81.3 | 0.4 |