Intent Classification on SNIPS (test)
99.7AccuracyMasked Graph + CRF
Evaluation results
| Method | Links | |
|---|---|---|
| Masked Graph + CRFPre-trained model=BERT2023.03 | 99.7 | |
| CTRANPre-trained model=BERT2023.03 | 99.42 | |
| Federated LearningPre-trained model=BERT2023.03 | 99.33 | |
| CM-netPre-trained model=BERT2023.03 | 99.32 | |
| Elmo+BiLSTM+CRFPre-trained model=Elmo2023.03 | 99.29 | |
| Stack-Propagation + BERTModel Size (MB)=>1200.002021.10 | 99 | |
| BERT Seq-CLSTraining Strategy=individual2023.05 | 99 | |
| SASBGCPre-trained model=BERT2023.03 | 98.86 | |
| Co-interactive TransformerPre-trained model=BERT2023.03 | 98.8 | |
| AISEPre-trained model=BERT2023.03 | 98.7 | |
| Joint BertPre-trained model=BERT2023.03 | 98.6 | |
| BERTBASEParams (M)=110, Latency (ms)=1580, Speedup=1.0x2021.10 | 98.26 | |
| LIDSNetModel Size (MB)=0.632021.10 | 98 | |
| Stack-PropagationModel Size (MB)=3.322021.10 | 98 | |
| LIDSNetParams (M)=0.59, Latency (ms)=18, Speedup=87.0x2021.10 | 98 | |
| DistilBERTParams (M)=66, Latency (ms)=781, Speedup=2.0x2021.10 | 97.94 | |
| MobileBERTParams (M)=24.6, Latency (ms)=545, Speedup=2.9x2021.10 | 97.71 | |
| Capsule-NLUModel Size (MB)=643.272021.10 | 97.7 | |
| SF-ID (BLSTM) networkModel Size (MB)=11.612021.10 | 97.43 | |
| TinyBERT4Params (M)=14.5, Latency (ms)=162, Speedup=9.8x2021.10 | 97.43 | |
| BERT Seq-CLSTraining Strategy=full2023.05 | 97.3 | |
| Slot-Gated BiLSTM with AttentionModel Size (MB)=11.572021.10 | 97 | |
| LLM2LLM% Data=1, # Seed Examples=140, # Augmented=912024.03 | 93.86 | |
| LLM2LLM% Data=0.8, # Seed Examples=105, # Augmented=1092024.03 | 93.71 | |
| LLM2LLM% Data=0.5, # Seed Examples=70, # Augmented=382024.03 | 92.14 | |
| Baseline% Data=1, # Seed Examples=140, # Augmented=912024.03 | 85.43 | |
| Binary BERTTraining Strategy=explicit2023.05 | 83.8 | |
| Binary BERTTraining Strategy=vanilla2023.05 | 82.9 | |
| Binary BERTTraining Strategy=implicit2023.05 | 82.5 | |
| Bi-EncoderTraining Strategy=implicit2023.05 | 72.5 | |
| Bi-EncoderTraining Strategy=explicit2023.05 | 71.9 | |
| Bi-EncoderTraining Strategy=vanilla2023.05 | 70.4 | |
| Baseline% Data=0.8, # Seed Examples=105, # Augmented=1092024.03 | 69.71 | |
| BARTTraining Strategy=Zero-shot2023.05 | 61.4 | |
| Baseline% Data=0.5, # Seed Examples=70, # Augmented=382024.03 | 60.14 | |
| GPT-2Training Strategy=explicit2023.05 | 54.1 | |
| GPT-2Training Strategy=vanilla2023.05 | 52.2 | |
| GPT-2Training Strategy=implicit2023.05 | 33.7 | |
| GPT-3Training Strategy=Zero-shot2023.05 | 13.9 | |
| Baseline% Data=0, # Seed Examples=0, # Augmented=02024.03 | 11.86 |