Text Classification on AgNews
95AccuracyULMFIT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ULMFIT2019.09 | 95 | — | — | — | — | |
| DRNN2019.09 | 94.5 | — | — | — | — | |
| FE-MLM + SpanMasking Unit=Span, Masking Strategy=Fully-Explored MLM, Domain Pre-training=News2020.10 | 94.13 | — | — | — | — | |
| MLM + SubwordMasking Unit=Subword, Masking Strategy=Standard MLM, Domain Pre-training=News2020.10 | 94.05 | — | — | — | — | |
| FE-MLM + SubwordMasking Unit=Subword, Masking Strategy=Fully-Explored MLM, Domain Pre-training=News2020.10 | 94.02 | — | — | — | — | |
| MLM + SpanMasking Unit=Span, Masking Strategy=Standard MLM, Domain Pre-training=News2020.10 | 93.94 | — | — | — | — | |
| DNC-CUWWriting Strategy=Cached Uniform Writing2019.01 | 93.9 | — | — | — | — | |
| RoBERTaDomain Pre-training=None, Masking Strategy=Standard MLM2020.10 | 93.9 | — | — | — | — | |
| DAPTDomain Pre-training=News, Masking Strategy=Standard MLM2020.10 | 93.9 | — | — | — | — | |
| DNC-UWWriting Strategy=Uniform Writing2019.01 | 93.7 | — | — | — | — | |
| ULMFiT (Small data)Training data percentage=20%2019.09 | 93.7 | — | — | — | — | |
| Skim-LSTMMode=Skim2019.01 | 93.6 | — | — | — | — | |
| DC+MFA2019.09 | 93.6 | — | — | — | — | |
| Standard LSTMMode=Standard2019.01 | 93.5 | — | — | — | — | |
| RLLRMIXEDModel=LLaMA2, Model Size=7B2024.05 | 93.5 | — | — | — | — | |
| RLLRModel=LLaMA2, Model Size=7B2024.05 | 93.4 | — | — | — | — | |
| DPCNN2019.09 | 93.1 | — | — | — | — | |
| EXAM2019.09 | 93 | — | — | — | — | |
| RLHFModel=LLaMA2, Model Size=7B2024.05 | 93 | — | — | — | — | |
| RLLRModel=LLaMA2, Model Size=13B2024.05 | 93 | — | — | — | — | |
| RLLRMIXEDModel=LLaMA2, Model Size=13B2024.05 | 92.9 | — | — | — | — | |
| Region Embedding2019.01 | 92.8 | — | — | — | — | |
| WC-Reg2019.09 | 92.8 | — | — | — | — | |
| RLHFModel=LLaMA2, Model Size=13B2024.05 | 92.7 | — | — | — | — | |
| RLHFModel=Bloom, Model Size=7B2024.05 | 92.7 | — | — | — | — | |
| RLLRModel=Bloom, Model Size=7B2024.05 | 92.7 | — | — | — | — | |
| Capsule-B2018.03 | 92.6 | — | — | — | — | |
| SFT w. rat.Model=LLaMA2, Model Size=7B2024.05 | 92.5 | — | — | — | — | |
| RLLRMIXEDModel=Bloom, Model Size=7B2024.05 | 92.5 | — | — | — | — | |
| SFT w. rat.Model=LLaMA2, Model Size=13B2024.05 | 92.4 | — | — | — | — | |
| CNNmode=non-static2018.03 | 92.3 | — | — | — | — | |
| CL-CNN2018.03 | 92.3 | — | — | — | — | |
| RLLRModel=Bloom, Model Size=3B2024.05 | 92.3 | — | — | — | — | |
| CNNmode=rand2018.03 | 92.2 | — | — | — | — | |
| SFTModel=LLaMA2, Model Size=7B2024.05 | 92.2 | — | — | — | — | |
| SFTModel=LLaMA2, Model Size=13B2024.05 | 92.2 | — | — | — | — | |
| Capsule-A2018.03 | 92.1 | — | — | — | — | |
| SFT w. rat.Model=Bloom, Model Size=3B2024.05 | 92 | — | — | — | — | |
| RLHFModel=Bloom, Model Size=3B2024.05 | 92 | — | — | — | — | |
| RLLRMIXEDModel=Bloom, Model Size=3B2024.05 | 92 | — | — | — | — | |
| SFT w. rat.Model=Bloom, Model Size=7B2024.05 | 91.8 | — | — | — | — | |
| ULMFIT (Tiny data)Training data percentage=8%2019.09 | 91.7 | — | — | — | — | |
| ROBERTa-CLTraining supervision type=clean labels, Method categorization=Fully-supervised2020.10 | 91.41 | — | — | — | — | |
| CNNmode=static2018.03 | 91.4 | — | — | — | — | |
| VD-CNN2018.03 | 91.3 | — | — | — | — | |
| VDCNN2019.01 | 91.3 | — | — | — | — | |
| VDCNN2019.09 | 91.3 | — | — | — | — | |
| TransEvolve-fullFF-1Feed-forward type=full, Variant=12021.09 | 91.1 | — | — | — | — | |
| TransEvolve-randomFF-2Feed-forward type=random, Variant=22021.09 | 90.8 | — | — | — | — | |
| TransEvolve-randomFF-1Feed-forward type=random, Variant=12021.09 | 90.6 | — | — | — | — | |
| TransEvolve-fullFF-2Feed-forward type=full, Variant=22021.09 | 90.5 | — | — | — | — | |
| SFTModel=Bloom, Model Size=3B2024.05 | 90.2 | — | — | — | — | |
| Tree-LSTM2018.03 | 90.1 | — | — | — | — | |
| SFTModel=Bloom, Model Size=7B2024.05 | 89.8 | — | — | — | — | |
| Synthesizer2021.09 | 89.1 | — | — | — | — | |
| Transformer2021.09 | 88.8 | — | — | — | — | |
| BILSTM2018.03 | 88.2 | — | — | — | — | |
| COSINEMethod categorization=Framework2020.10 | 87.52 | — | — | — | — | |
| SimPTCHuman/KB=false, Unlabeled=true2022.12 | 86.9 | — | — | — | 0.3 | |
| KPTHuman/KB=true, Unlabeled=true2022.12 | 86.7 | — | — | — | — | |
| Linformer2021.09 | 86.5 | — | — | — | — | |
| LOTClassHuman/KB=true, Unlabeled=true2022.12 | 86.4 | — | — | — | — | |
| USTMethod categorization=Baseline2020.10 | 86.28 | — | — | — | — | |
| SMARTMethod categorization=Baseline2020.10 | 86.12 | — | — | — | — | |
| LSTM2018.03 | 86.1 | — | — | — | — | |
| Self-ensembleMethod categorization=Baseline2020.10 | 85.72 | — | — | — | — | |
| DenoiseMethod categorization=Baseline2020.10 | 85.71 | — | — | — | — | |
| MixupMethod categorization=Baseline2020.10 | 85.4 | — | — | — | — | |
| NPPromptHuman/KB=false, Unlabeled=false2022.12 | 85.2 | — | — | — | 0.5 | |
| FreeLBMethod categorization=Baseline2020.10 | 85.12 | — | — | — | — | |
| FlowGMMn_l=200, n_u=200k, classes=4, Backbone=BERT embeddings, training=semi-supervised2019.12 | 84.8 | — | — | — | — | |
| InitMethod categorization=Framework2020.10 | 84.63 | — | — | — | — | |
| ChatGPT w. descriptionsHuman/KB=true, Unlabeled=false2022.12 | 83.8 | — | — | — | — | |
| GPT-3 w. descriptionsHuman/KB=true, Unlabeled=false2022.12 | 83.4 | — | — | — | — | |
| WeSTClassMethod categorization=Baseline2020.10 | 82.78 | — | — | — | — | |
| vONTSSNumber of topics=202023.07 | 82.3 | 82.1 | — | — | — | |
| ROBERTa-WL+Training supervision type=weak labels, Method categorization=Baseline2020.10 | 82.25 | — | — | — | — | |
| LOTClass w/o. self trainHuman/KB=true, Unlabeled=true2022.12 | 82.2 | — | — | — | — | |
| CatENumber of topics=202023.07 | 82 | 82.2 | — | — | — | |
| Π-modeln_l=200, n_u=200k, classes=4, Backbone=BERT embeddings, training=semi-supervised2019.12 | 80.6 | — | — | — | — | |
| Best UnsupervisedNumber of topics=202023.07 | 79.9 | 79.7 | — | — | — | |
| Manual VerbHuman/KB=true, Unlabeled=false2022.12 | 79.6 | — | — | — | 0.6 | |
| gONTSSNumber of topics=202023.07 | 79.3 | 78.8 | — | — | — | |
| Logistic Regressionn_l=200, n_u=200k, classes=4, Backbone=BERT embeddings, training=labeled data only2019.12 | 78.9 | — | — | — | — | |
| 3-Layer NN + Dropoutn_l=200, n_u=200k, classes=4, Backbone=BERT embeddings, training=labeled data only2019.12 | 78.1 | — | — | — | — | |
| CarExNumber of topics=202023.07 | 77.8 | 77.8 | — | — | — | |
| NSP-BERTHuman/KB=true, Unlabeled=false2022.12 | 77.4 | — | — | — | 0.6 | |
| vONTSSNumber of topics=20, Loss function=CE2023.07 | 75.4 | 71.7 | — | — | — | |
| vONTSSNumber of topics=20, Loss function=all loss2023.07 | 74.1 | 70.2 | — | — | — | |
| SimPLEModel Backbone=DeBERTa-large, Number of Parameters=350M, Number of runs=32023.05 | 73.57 | — | — | — | — | |
| PretrainModel Backbone=DeBERTa-large, Number of Parameters=350M, Number of runs=32023.05 | 73.4 | — | — | — | — | |
| SETREDModel Backbone=DeBERTa-large, Number of Parameters=350M, Number of runs=32023.05 | 73.33 | — | — | — | — | |
| GuidedLDANumber of topics=202023.07 | 73.3 | 73.5 | — | — | — | |
| DropoutModel Backbone=DeBERTa-large, Number of Parameters=350M, Number of runs=32023.05 | 73.16 | — | — | — | — | |
| Semantic RetrievalHuman/KB=true, Unlabeled=false2022.12 | 73.1 | — | — | — | 1.2 | |
| BaseSTModel Backbone=DeBERTa-large, Number of Parameters=350M, Number of runs=32023.05 | 73.1 | — | — | — | — | |
| no-stitchstitching_type=none, decoder=SVM (linear kernel)2023.11 | 73 | — | — | — | — | |
| ImplyLossMethod categorization=Baseline2020.10 | 68.5 | — | — | — | — | |
| Multi-Null PromptHuman/KB=false, Unlabeled=false2022.12 | 68.2 | — | — | — | 1.8 | |
| Null PromptHuman/KB=false, Unlabeled=false2022.12 | 67.9 | — | — | — | 2 |