Topic Classification on DBPedia (test)
96.46AccuracyManualVerb
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ManualVerbshots (K)=8, task-specific knowledge=true2022.03 | 96.46 | — | — | — | |
| ProtoVerbshots (K)=16, task-specific knowledge=false2022.03 | 96.3 | — | — | — | |
| ManualVerbshots (K)=16, task-specific knowledge=true2022.03 | 96.05 | — | — | — | |
| ManualVerbshots (K)=4, task-specific knowledge=true2022.03 | 95.83 | — | — | — | |
| ProtoVerbshots (K)=8, task-specific knowledge=false2022.03 | 95.75 | — | — | — | |
| ConCModel=Llama-2 13B, Shots=42024.11 | 95.3 | — | — | 3.8 | |
| LinCModel=Llama-2 13B, Shots=42024.11 | 95.3 | — | — | 3.8 | |
| LinC+HBTVAModel=Llama-2 13B, Shots=42024.11 | 95.3 | — | — | 2.2 | |
| LinCModel=Llama-2 13B, Shots=82024.11 | 95.3 | — | — | 3.9 | |
| LinC+HBTVAModel=Llama-2 13B, Shots=82024.11 | 95.3 | — | — | 2.1 | |
| UncalibratedModel=Llama-2 13B, Shots=82024.11 | 94.7 | — | — | 5.9 | |
| LinCModel=GPT-J 6B, Shots=42024.11 | 94.3 | — | — | 5.7 | |
| LinC+HBTVAModel=GPT-J 6B, Shots=42024.11 | 94.3 | — | — | 4.8 | |
| ConCModel=Llama-2 13B, Shots=82024.11 | 94.3 | — | — | 3.9 | |
| ConCModel=GPT-J 6B, Shots=42024.11 | 94 | — | — | 6.9 | |
| UncalibratedModel=Llama-2 13B, Shots=42024.11 | 94 | — | — | 9.9 | |
| LinCModel=GPT-J 6B, Shots=82024.11 | 93.9 | — | — | 5.8 | |
| LinC+HBTVAModel=GPT-J 6B, Shots=82024.11 | 93.9 | — | — | 2.6 | |
| SoftVerbshots (K)=8, task-specific knowledge=false2022.03 | 93.69 | — | — | — | |
| ManualVerbshots (K)=2, task-specific knowledge=true2022.03 | 93.61 | — | — | — | |
| LinCModel=Llama-2 13B, Shots=12024.11 | 93.1 | — | — | 4.4 | |
| LinC+HBTVAModel=Llama-2 13B, Shots=12024.11 | 93.1 | — | — | 3 | |
| UncalibratedModel=GPT-J 6B, Shots=82024.11 | 92.3 | — | — | 9.2 | |
| ConCModel=Llama-2 13B, Shots=12024.11 | 92.3 | — | — | 5.2 | |
| NLI-STLearning Paradigm=Reference, Use auxiliary labeled data=true, Use task-specific corpus=true2023.05 | 92.2 | — | — | — | |
| SearchVerbshots (K)=16, task-specific knowledge=false2022.03 | 92 | — | — | — | |
| ConCModel=GPT-J 6B, Shots=82024.11 | 92 | — | — | 6.8 | |
| X-ClassLearning Paradigm=Reference, Use task-specific corpus=true2023.05 | 91.3 | — | — | — | |
| LOTClassLearning Paradigm=Reference, Use task-specific corpus=true2023.05 | 91.1 | — | — | — | |
| ProtoVerbshots (K)=4, task-specific knowledge=false2022.03 | 90.91 | — | — | — | |
| UncalibratedModel=Llama-2 13B, Shots=12024.11 | 90.7 | — | — | 11.4 | |
| SoftVerbshots (K)=4, task-specific knowledge=false2022.03 | 89.12 | — | — | — | |
| UncalibratedModel=GPT-J 6B, Shots=42024.11 | 89 | — | — | 15.7 | |
| SearchVerbshots (K)=8, task-specific knowledge=false2022.03 | 88.41 | — | — | — | |
| REGENLearning Paradigm=Zero-shot Learning via Generating Task-specific Datasets, Standard Deviation=0.92023.05 | 87.6 | — | — | — | |
| SoftVerbshots (K)=16, task-specific knowledge=false2022.03 | 86.9 | — | — | — | |
| SearchVerbshots (K)=4, task-specific knowledge=false2022.03 | 86.4 | — | — | — | |
| ProtoVerbshots (K)=2, task-specific knowledge=false2022.03 | 85.49 | — | — | — | |
| ManualVerbshots (K)=1, task-specific knowledge=true2022.03 | 85.47 | — | — | — | |
| LinCModel=GPT-J 6B, Shots=12024.11 | 84.4 | — | — | 6.6 | |
| LinC+HBTVAModel=GPT-J 6B, Shots=12024.11 | 84.4 | — | — | 5.1 | |
| LinCModel=Llama-2 13B, Shots=02024.11 | 84.4 | — | — | 16.2 | |
| LinC+HBTVAModel=Llama-2 13B, Shots=02024.11 | 84.4 | — | — | 4.1 | |
| ConCModel=GPT-J 6B, Shots=12024.11 | 82.7 | — | — | 6.9 | |
| KPTLearning Paradigm=Reference, Use task-specific corpus=true, Use additional knowledge base=true2023.05 | 82.2 | — | — | — | |
| Mining (Re-implementation)Learning Paradigm=Zero-shot Learning via Generating Task-specific Datasets, Concurrent Work=true, Fair Comparison Setup=true, Standard Deviation=0.62023.05 | 82.1 | — | — | — | |
| SoftVerbshots (K)=2, task-specific knowledge=false2022.03 | 80.69 | — | — | — | |
| SearchVerbshots (K)=2, task-specific knowledge=false2022.03 | 78.21 | — | — | — | |
| ConCModel=Llama-2 13B, Shots=02024.11 | 75.3 | — | — | 17.2 | |
| TE-NLI (Best)Learning Paradigm=Labeled data usage, Use auxiliary labeled data=true2023.05 | 73 | — | — | — | |
| ProtoVerbshots (K)=1, task-specific knowledge=false2022.03 | 72.85 | — | — | — | |
| PromptLearning Paradigm=Zero-shot Learning via Direct Inferencing2023.05 | 71.3 | — | — | — | |
| NSP-BERTLearning Paradigm=Zero-shot Learning via Direct Inferencing2023.05 | 69.4 | — | — | — | |
| ManualVerbshots (K)=0, task-specific knowledge=true2022.03 | 67.06 | — | — | — | |
| SuperGenLearning Paradigm=Zero-shot Learning via Generating Task-specific Datasets, Standard Deviation=2.02023.05 | 66.5 | — | — | — | |
| SoftVerbshots (K)=1, task-specific knowledge=false2022.03 | 65.35 | — | — | — | |
| LinCModel=GPT-J 6B, Shots=02024.11 | 62.2 | — | — | 12.8 | |
| LinC+HBTVAModel=GPT-J 6B, Shots=02024.11 | 62.2 | — | — | 5.7 | |
| SearchVerbshots (K)=1, task-specific knowledge=false2022.03 | 60.06 | — | — | — | |
| GPT-3Learning Paradigm=Zero-shot Learning via Direct Inferencing, Billion-scale PLM=true2023.05 | 59.7 | — | — | — | |
| UncalibratedModel=GPT-J 6B, Shots=12024.11 | 58.7 | — | — | 14.2 | |
| UncalibratedModel=Llama-2 13B, Shots=02024.11 | 54.3 | — | — | 19.7 | |
| ConCModel=GPT-J 6B, Shots=02024.11 | 47.7 | — | — | 24.6 | |
| UncalibratedModel=GPT-J 6B, Shots=02024.11 | 19.7 | — | — | 27.4 | |
| Adversarial2016.05 | — | 79 | — | — | |
| Bag-of-means2015.09 | — | 9.55 | — | — | |
| Bag-of-words2016.05 | — | 357 | — | — | |
| Baselineembedding normalization=false2016.05 | — | 87 | — | — | |
| Baselineembedding normalization=true2016.05 | — | 90 | — | — | |
| BoW2015.09 | — | 3.39 | — | — | |
| BoW TFIDFTF-IDF Weighting=true2015.09 | — | 2.63 | — | — | |
| char-CNNTraining=Supervised2020.09 | — | 1.55 | — | — | |
| CNNTraining=Supervised2020.09 | — | 0.98 | — | — | |
| Conv. (large, Th)Model Size=large, Thesaurus Augmentation=true2015.09 | — | 1.6 | — | — | |
| Conv. (large)Model Size=large2015.09 | — | 1.73 | — | — | |
| Conv. (small, Th)Model Size=small, Thesaurus Augmentation=true2015.09 | — | 1.85 | — | — | |
| Conv. (small)Model Size=small2015.09 | — | 1.98 | — | — | |
| DocVecTraining=Supervised2020.09 | — | 2.34 | — | — | |
| FastTextTraining=Supervised2020.09 | — | 1.4 | — | — | |
| Full Conv. (large, Th)Model Size=large, Case Sensitive=true, Thesaurus Augmentation=true2015.09 | — | 1.55 | — | — | |
| Full Conv. (large)Model Size=large, Case Sensitive=true2015.09 | — | 1.66 | — | — | |
| Full Conv. (small, Th)Model Size=small, Case Sensitive=true, Thesaurus Augmentation=true2015.09 | — | 1.69 | — | — | |
| Full Conv. (small)Model Size=small, Case Sensitive=true2015.09 | — | 1.89 | — | — | |
| L_MIXEDtraining=semi-supervised, objective=mixed objective function2020.09 | — | 0.7 | — | — | |
| L_MLTraining=Supervised2020.09 | — | 0.91 | — | — | |
| Large-CNNgranularity=character-level2016.05 | — | 173 | — | — | |
| Linear Model (TFIDF + SVM)Training=Supervised2020.09 | — | 1.31 | — | — | |
| Lk. Conv. (large, Th)Model Size=large, Input Representation=character-level (lookup table), Thesaurus Augmentation=true2015.09 | — | 1.58 | — | — | |
| Lk. Conv. (large)Model Size=large, Input Representation=character-level (lookup table)2015.09 | — | 1.72 | — | — | |
| Lk. Conv. (small, Th)Model Size=small, Input Representation=character-level (lookup table), Thesaurus Augmentation=true2015.09 | — | 1.77 | — | — | |
| Lk. Conv. (small)Model Size=small, Input Representation=character-level (lookup table)2015.09 | — | 1.85 | — | — | |
| LSTM2015.09 | — | 1.45 | — | — | |
| LSTMtraining=semi-supervised2020.09 | — | 0.76 | — | — | |
| N-grams TFIDF2016.05 | — | 131 | — | — | |
| n-TFIDFAuthor=Zhang2016.06 | — | 1.31 | — | — | |
| ngrams2015.09 | — | 1.37 | — | — | |
| ngrams TFIDFTF-IDF Weighting=true2015.09 | — | 1.31 | — | — | |
| oh-CNNTraining=Supervised2020.09 | — | 0.88 | — | — | |
| oh-LSTMtraining=semi-supervised2020.09 | — | 0.84 | — | — | |
| Random perturbation2016.05 | — | 85 | — | — |