Question Classification on TREC
98.07AccuracyUSE_T+CNN (w2v w.e.)
Evaluation Results
| Method | Links | |
|---|---|---|
| USE_T+CNN (w2v w.e.)Universal Encoder=Transformer (USE_T), Transfer Level=Sentence & Word, Embedding Initialization=word2vec skip-gram, Transfer Task Model=CNN2018.03 | 98.07 | |
| USE_D+CNN (lrn w.e.)Universal Encoder=Deep Averaging Network (USE_D), Transfer Level=Sentence, Embedding Initialization=learned on transfer task, Transfer Task Model=CNN2018.03 | 97.71 | |
| USE_D+CNN (w2v w.e.)Universal Encoder=Deep Averaging Network (USE_D), Transfer Level=Sentence & Word, Embedding Initialization=word2vec skip-gram, Transfer Task Model=CNN2018.03 | 97.67 | |
| BAE: BERT2021.04 | 97.6 | |
| USE_T+CNN (lrn w.e.)Universal Encoder=Transformer (USE_T), Transfer Level=Sentence, Embedding Initialization=learned on transfer task, Transfer Task Model=CNN2018.03 | 97.44 | |
| CNN (w2v w.e.)Transfer Level=Word, Embedding Initialization=word2vec skip-gram, Transfer Task Model=CNN2018.03 | 97.32 | |
| CNN+MCFAAdditional Context=Translation, N=ensemble2018.06 | 96.8 | |
| MUPPET2022.12 | 96.8 | |
| ROBERTa-CLTraining supervision type=clean labels, Method categorization=Fully-supervised2020.10 | 96.68 | |
| USE_T+DAN (lrn w.e.)Universal Encoder=Transformer (USE_T), Transfer Level=Sentence, Embedding Initialization=learned on transfer task, Transfer Task Model=DAN2018.03 | 96.6 | |
| CNN+B1Additional Context=Translation, N=ensemble2018.06 | 96.4 | |
| CNN+B2Additional Context=Translation, N=ensemble2018.06 | 96.4 | |
| BLSTM-2DCNNNN category=ours2016.11 | 96.1 | |
| CNN+MCFAAdditional Context=Translation, N=102018.06 | 96 | |
| TBCNNNN category=CNN2016.11 | 96 | |
| USE_D+DAN (lrn w.e.)Universal Encoder=Deep Averaging Network (USE_D), Transfer Level=Sentence, Embedding Initialization=learned on transfer task, Transfer Task Model=DAN2018.03 | 95.86 | |
| CNN (lrn w.e.)Transfer Level=None, Embedding Initialization=learned on transfer task, Transfer Task Model=CNN2018.03 | 95.82 | |
| DSCNN2018.06 | 95.6 | |
| USE_T+DAN (w2v w.e.)Universal Encoder=Transformer (USE_T), Transfer Level=Sentence & Word, Embedding Initialization=word2vec skip-gram, Transfer Task Model=DAN2018.03 | 95.51 | |
| Dep-CNN2018.06 | 95.4 | |
| CNN+B1Additional Context=Translation, N=12018.06 | 95.4 | |
| CNN+B2Additional Context=Translation, N=12018.06 | 95.4 | |
| CNN+MCFAAdditional Context=Translation, N=12018.06 | 95.4 | |
| DSCNNNN category=Other2016.11 | 95.4 | |
| CNN+B2Additional Context=Translation, N=102018.06 | 95.2 | |
| FADS-ICLBackbone=Llama-1 30B, Shots=128-shots2024.05 | 95.2 | |
| FADS-ICLBackbone=Llama-2 70B, Shots=128-shots2024.05 | 95.2 | |
| SVMs2014.08 | 95 | |
| CNN+B1Additional Context=Translation, N=102018.06 | 95 | |
| FADS-ICLBackbone=Llama-2 13B, Shots=128-shots2024.05 | 95 | |
| BLSTM-2DPoolingNN category=ours2016.11 | 94.8 | |
| USE_D+DAN (w2v w.e.)Universal Encoder=Deep Averaging Network (USE_D), Transfer Level=Sentence & Word, Embedding Initialization=word2vec skip-gram, Transfer Task Model=DAN2018.03 | 94.72 | |
| C-LSTMNN category=Other2016.11 | 94.6 | |
| TopCNNAdditional Context=Topic, Granularity=ensemble2018.06 | 94 | |
| DAN (lrn w.e.)Transfer Level=None, Embedding Initialization=learned on transfer task, Transfer Task Model=DAN2018.03 | 93.88 | |
| BLSTM-AttNN category=ours2016.11 | 93.8 | |
| FADS-ICLBackbone=Llama-1 13B, Shots=128-shots2024.05 | 93.8 | |
| CNN-non-staticword vectors=fine-tuned pre-trained (word2vec)2014.08 | 93.6 | |
| CNN2018.06 | 93.6 | |
| CNN2015.04 | 93.6 | |
| CNN-non-staticNN category=CNN2016.11 | 93.6 | |
| CNN-400Emb.Size=4002019.05 | 93.6 | |
| FADS-ICLBackbone=Llama-1 7B, Shots=128-shots2024.05 | 93.6 | |
| STM+TSED+PT+2LEmb.Size=50, Teacher Model Architecture=CNN2019.05 | 93.48 | |
| DCNN2014.08 | 93 | |
| DCNN2015.04 | 93 | |
| DCNNNN category=CNN2016.11 | 93 | |
| BLSTMNN category=ours2016.11 | 93 | |
| FADS-ICLBackbone=Llama-2 7B, Shots=128-shots2024.05 | 93 | |
| STM+TSED+PT+2LEmb.Size=50, Teacher Model Architecture=LSTM2019.05 | 92.96 | |
| CNN-staticword vectors=static pre-trained (word2vec)2014.08 | 92.8 | |
| NLM+TSED+PT+2LEmb.Size=50, Teacher Model Architecture=CNN2019.05 | 92.8 | |
| Avg. BERT embeddingsBackbone=BERT-base, Pooling=Avg2022.03 | 92.8 | |
| STM+TSEDEmb.Size=50, Teacher Model Architecture=CNN2019.05 | 92.76 | |
| LM+TSED+PT+2LEmb.Size=50, Teacher Model Architecture=CNN2019.05 | 92.58 | |
| LM+TSEDEmb.Size=50, Teacher Model Architecture=LSTM2019.05 | 92.56 | |
| USE TUniversal Encoder=Transformer (USE_T), Transfer Level=Sentence, Embedding Initialization=None, Transfer Task Model=None2018.03 | 92.51 | |
| TopCNNAdditional Context=Topic, Granularity=word2018.06 | 92.5 | |
| LM+TSEDEmb.Size=50, Teacher Model Architecture=CNN2019.05 | 92.48 | |
| AdaSent2018.06 | 92.4 | |
| AdaSent2015.04 | 92.4 | |
| AdaSentNN category=Other2016.11 | 92.4 | |
| NLM+TSEDEmb.Size=50, Teacher Model Architecture=CNN2019.05 | 92.36 | |
| STM+TSEDEmb.Size=50, Teacher Model Architecture=LSTM2019.05 | 92.36 | |
| CNN-multichannelword vectors=pre-trained (word2vec), channels=22014.08 | 92.2 | |
| SkipThoughtData Source Type=Ordered Sentences2016.02 | 92.2 | |
| combine-skipNN category=Other2016.11 | 92.2 | |
| Skip-thoughtModel=Skip-thought2022.03 | 92.2 | |
| NLM+TSED+PT+2LEmb.Size=50, Teacher Model Architecture=LSTM2019.05 | 92.08 | |
| TopCNNAdditional Context=Topic, Granularity=sentence2018.06 | 92 | |
| CNN-MCNN category=CNN2016.11 | 92 | |
| LSTM-400Emb.Size=4002019.05 | 92 | |
| LM+TSED+PT+2LEmb.Size=50, Teacher Model Architecture=LSTM2019.05 | 91.91 | |
| P.V.2015.04 | 91.8 | |
| NLM+TSEDEmb.Size=50, Teacher Model Architecture=LSTM2019.05 | 91.8 | |
| CNN-AnaNN category=CNN2016.11 | 91.37 | |
| CNN-randword vectors=randomly initialized2014.08 | 91.2 | |
| FADS-ICLNumber of training shots (m)=64, LLM scale=1.5B2024.05 | 91.2 | |
| USE DUniversal Encoder=Deep Averaging Network (USE_D), Transfer Level=Sentence, Embedding Initialization=None, Transfer Task Model=None2018.03 | 91.19 | |
| ColD-Fusion2022.12 | 91.04 | |
| BRNN2015.04 | 91 | |
| CNN-50Emb.Size=502019.05 | 91 | |
| BERT [CLS]-embeddingBackbone=BERT-base, Pooling=CLS2022.03 | 91 | |
| FADS-ICLcandidate pool size (m)=2562024.05 | 91 | |
| ENCEmb.Size=502019.05 | 90.6 | |
| kNN-promptingNumber of training shots (m)=128, LLM scale=1.5B2024.05 | 90.5 | |
| FADS-ICLNumber of training shots (m)=128, LLM scale=1.5B2024.05 | 90.5 | |
| FADS-ICLBackbone=GPT-2 1.5B, Shots=128-shots2024.05 | 90.5 | |
| FADS-ICLcandidate pool size (m)=1282024.05 | 90.5 | |
| ConMeZOBackbone=RoBERTa-Large, Evaluation Protocol=prompt-conditioned, Time Constraint=equal wall-clock time2025.11 | 90.4 | |
| FADS-ICLBackbone=GPT-2 0.8B, Shots=128-shots2024.05 | 90.3 | |
| RNN2015.04 | 90.2 | |
| kNN-promptingBackbone=Llama-2 70B, Shots=128-shots2024.05 | 90.2 | |
| kNN-promptBackbone=Llama-2 70B, Shots=128-shots2024.05 | 90.1 | |
| TM with ϕextension_type=words within a given cosine angle θ2021.04 | 90.04 | |
| TM with kextension_type=k nearest words in embedding space2021.04 | 89.82 | |
| SCD-BERT_baseBackbone=BERT-base2022.03 | 89.8 | |
| LOZO-MBackbone=RoBERTa-Large, Evaluation Protocol=prompt-conditioned, Time Constraint=equal wall-clock time2025.11 | 89.8 | |
| FP+Transformerfeature_extension=FP2021.04 | 89.51 | |
| LOZOBackbone=RoBERTa-Large, Evaluation Protocol=prompt-conditioned, Time Constraint=equal wall-clock time2025.11 | 89.4 |