Natural Language Inference on MultiNLI Mismatched
79.1AccuracyDensely Interactive Inference Network
Evaluation Results
| Method | Links | |
|---|---|---|
| Densely Interactive Inference Network2018.11 | 79.1 | |
| CAFE Ensembleensemble_size=5 models2017.12 | 79 | |
| CAFE2017.12 | 77.9 | |
| Compare-Propagate Alignment-Factorized Encoders2018.11 | 77.9 | |
| ESIM + Readsource=Weissenborn, 20172017.12 | 77 | |
| ESIMsource=Weissenborn, 20172017.12 | 75.8 | |
| CDME*Number of parameters=9.0M, Encoder=BiLSTM-Max, Embedding sets=multiple2018.04 | 74.9 | |
| Chen et al.2018.11 | 74.9 | |
| Unweighted DMENumber of parameters=8.6M, Encoder=BiLSTM-Max2018.04 | 74.4 | |
| DMENumber of parameters=8.6M, Encoder=BiLSTM-Max2018.04 | 74.4 | |
| DME*Number of parameters=9.0M, Encoder=BiLSTM-Max, Embedding sets=multiple2018.04 | 74.3 | |
| CDMENumber of parameters=8.6M, Encoder=BiLSTM-Max2018.04 | 74.1 | |
| BiLSTM generalized poolingpenalization=on parameter matrices2018.06 | 74 | |
| aESIMimplementation=implemented on Keras2018.12 | 73.9 | |
| SSE2018.04 | 73.6 | |
| Nie and Bansal2018.11 | 73.6 | |
| BiLSTM gated-pooling2018.06 | 73.6 | |
| Shortcut stacked BiLSTM2018.06 | 73.6 | |
| ESIMimplementation=implemented on Keras2018.12 | 73.5 | |
| Naive baselineNumber of parameters=61.3M, Encoder=2048-dimensional BiLSTM2018.04 | 73 | |
| BiLSTM max pooling2018.06 | 73 | |
| Distance-Based Self-Attention Network2018.11 | 72.9 | |
| Stacked Bi-LSTMs + shortcut connections + max-poolinghidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 72.2 | |
| ESIMsource=Williams et al., 2017 / Khot et al., 20182017.12 | 72.1 | |
| Enhanced Sequential Inference Model2018.11 | 71.9 | |
| BiLSTM last pooling2018.06 | 71.9 | |
| BiLSTM mean pooling2018.06 | 71.6 | |
| Directional Self-Attention Encoders2018.11 | 71.4 | |
| Naive baselineNumber of parameters=9.8M, Encoder=512-dimensional BiLSTM2018.04 | 71.1 | |
| Bi-LSTM sentence encoder + max-poolinghidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 71.1 | |
| Bi-LSTM sentence encoder + max-pooling + attentionhidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 70.8 | |
| Stacked Bi-LSTMs + shortcut connections + max-pooling + attentionhidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 70.5 | |
| Bi-LSTM sentence encoder + attentionhidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 70.4 | |
| FastText BiLSTM-MaxNumber of parameters=8.6M, Embedding type=FastText, Encoder=BiLSTM-Max2018.04 | 70.3 | |
| GloVe BiLSTM-MaxNumber of parameters=8.6M, Embedding type=GloVe, Encoder=BiLSTM-Max2018.04 | 70 | |
| Bi-GRU sentence encoder + attentionhidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 69.5 | |
| BiLSTM2017.12 | 69.4 | |
| Stacked Bi-GRUs + shortcut connections + max-poolinghidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 68.9 | |
| Stacked Bi-GRUs + shortcut connections + max-pooling + attentionhidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 68.4 | |
| Bi-GRU sentence encoder + max-pooling + attentionhidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 68.2 | |
| SWEM-max2018.05 | 67.7 | |
| SWEM-concat2018.05 | 67.6 | |
| Bi-GRU sentence encoder + max-poolinghidden_state_dim=300, bidirectional_dim=600, optimizer=Adam2018.11 | 67.3 | |
| BiLSTM2018.11 | 67.1 | |
| LSTMType=Bidirectional2018.05 | 66.9 | |
| Bi-LSTMsource=baseline from [17]2018.12 | 66.9 | |
| BiLSTM2018.06 | 66.9 | |
| SWEM-aver2018.05 | 66.2 | |
| CNN2018.05 | 65.3 | |
| CBOW2017.12 | 64.8 | |
| Continuous BOW (Averaging Word Embeddings)2018.11 | 64.6 | |
| CBOWsource=baseline from [17]2018.12 | 64.5 | |
| CBOW2018.06 | 64.5 | |
| LaMini-GPT-J# of params=6B2023.04 | 64 | |
| LaMini-LLaMA# of params=7B2023.04 | 63.8 | |
| Alpaca# of params=7B2023.04 | 39.6 | |
| GPT-J# of params=6B2023.04 | 37.7 | |
| Majority2017.12 | 35.6 | |
| Most Frequent Class2018.11 | 35.6 | |
| LLaMA# of params=7B2023.04 | 35.6 |