Natural Language Understanding on Snips (test)
99.29Intent AccBiLSTMs + ELMo
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BiLSTMs + ELMoembedding=ELMo2019.09 | 99.29 | 93.9 | — | — | — | |
| CM-Net2019.09 | 99.29 | 97.15 | — | — | — | |
| Stack-PropagationBackbone=BERT2019.09 | 99 | 97 | — | — | 92.9 | |
| BiLSTMs + ELMoLembedding=ELMoL2019.09 | 98.83 | 93.29 | — | — | — | |
| Joint BERTBackbone=BERT-Base2019.02 | 98.6 | 97 | 92.8 | — | — | |
| BERT SLUBackbone=BERT2019.09 | 98.6 | 97 | — | — | 92.8 | |
| Joint BERT2019.09 | 98.6 | 97 | — | — | — | |
| Joint BERT + CRFBackbone=BERT-Base, Decoder=CRF2019.02 | 98.4 | 96.7 | 92.6 | — | — | |
| Sentence-State LSTMimplementation=ours2019.09 | 98.3 | 95.8 | — | — | — | |
| Dilated CNN, Label-Recurrent2019.09 | 98.29 | 93.11 | — | — | — | |
| Stack-PropagationBackbone=Baseline2019.09 | 98 | 94.2 | — | — | 86.9 | |
| Intent detection (BERT)Backbone=BERT, Task=Single-task Intent detection2019.09 | 97.8 | — | — | — | — | |
| CAPSULE-NLU2019.09 | 97.7 | 91.8 | — | — | — | |
| SF-ID NetworkMode=SF-First, CRF=with2019.06 | 97.43 | 91.43 | — | 80.57 | — | |
| SF-ID NetworkMode=SF-First, CRF=without2019.06 | 97.34 | 90.34 | — | 78.43 | — | |
| SF-ID NetworkMode=ID-First, CRF=with2019.06 | 97.29 | 92.23 | — | 80.43 | — | |
| Slot-Gated2019.02 | 97 | 88.8 | 75.5 | — | — | |
| SF-ID NetworkMode=ID-First, CRF=without2019.06 | 97 | 90.46 | — | 78.37 | — | |
| Slot-Gated (Full Atten.)2019.09 | 97 | 88.8 | — | — | — | |
| RNN-LSTM2019.02 | 96.9 | 87.3 | 73.2 | — | — | |
| Joint Seq2019.06 | 96.9 | 87.3 | — | 73.2 | — | |
| Joint Seq2Seqsource=retrieved from Slot-Gated (Goo et al., 2018)2019.09 | 96.9 | 87.3 | — | — | — | |
| Sloted-Gated2019.06 | 96.86 | 89.27 | — | 76.43 | — | |
| Slot-Gated (Intent Atten.)2019.09 | 96.8 | 88.3 | — | — | — | |
| Atten.-BiRNN2019.02 | 96.7 | 87.8 | 74.1 | — | — | |
| Atten.-Based2019.06 | 96.7 | 87.8 | — | 74.1 | — | |
| Attention Bi-RNNsource=retrieved from Slot-Gated (Goo et al., 2018)2019.09 | 96.7 | 87.8 | — | — | — | |
| Slot filling (BERT)Backbone=BERT, Task=Single-task Slot filling2019.09 | — | 95.8 | — | — | — |