Unknown Intent Detection on ATIS (test)
86.79Macro F1BERT + VAE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BERT + VAEBackbone=BERT-base2023.03 | 86.79 | 87.15 | |
| BERT + ADBBackbone=BERT-base2023.03 | 83.73 | 83.74 | |
| LOF (LMCL)% of known intents=25%2019.06 | 69.6 | — | |
| LOF (Softmax)% of known intents=25%2019.06 | 67.3 | — | |
| glove LMCLEmbeddings=GloVe2023.03 | 65.2 | 65.77 | |
| DOC (Softmax)% of known intents=25%2019.06 | 63.6 | — | |
| fasstext + LMCLEmbeddings=FastText2023.03 | 63.48 | 64.23 | |
| LOF (LMCL)% of known intents=50%2019.06 | 63.4 | — | |
| DOC (Softmax)% of known intents=50%2019.06 | 63.3 | — | |
| DOC% of known intents=50%2019.06 | 62.8 | — | |
| LOF (Softmax)% of known intents=50%2019.06 | 61.8 | — | |
| DOC% of known intents=25%2019.06 | 61.6 | — | |
| LOF (LMCL)% of known intents=75%2019.06 | 39.6 | — | |
| LOF (Softmax)% of known intents=75%2019.06 | 38.9 | — | |
| DOC (Softmax)% of known intents=75%2019.06 | 38.7 | — | |
| DOC% of known intents=75%2019.06 | 37.7 | — | |
| fasstext + KLOOSEmbeddings=FastText2023.03 | 36.6 | 57.72 | |
| MSP% of known intents=75%2019.06 | 17.2 | — | |
| MSP% of known intents=50%2019.06 | 15.3 | — | |
| MSP% of known intents=25%2019.06 | 8.1 | — |