Topic Classification on Yahoo (test)
77.1AccuracyFine-tuning*
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Fine-tuning*Backbone=RoBERTa-large, Setting=Supervised2023.05 | 77.1 | — | |
| DecTn=256, Update model parameters=false2022.12 | 71.3 | — | |
| Fine-tuningn=256, Update model parameters=true2022.12 | 70.2 | — | |
| DecTn=64, Update model parameters=false2022.12 | 69.3 | — | |
| Fine-tuningn=64, Update model parameters=true2022.12 | 69 | — | |
| SSTuning-ALBERTBackbone=ALBERT-xxlarge, Labeled=false, Setting=Zero-shot2023.05 | 63.5 | — | |
| SSTuning-largeBackbone=RoBERTa-large, Labeled=false, Setting=Zero-shot2023.05 | 62.4 | — | |
| KPTLearning Paradigm=Reference, Use task-specific corpus=true, Use additional knowledge base=true2023.05 | 61.6 | — | |
| NLI-STLearning Paradigm=Reference, Use auxiliary labeled data=true, Use task-specific corpus=true2023.05 | 59.8 | — | |
| Soft Prompt TuningFew-shot=5-shot2024.07 | 59.7 | — | |
| REGENLearning Paradigm=Zero-shot Learning via Generating Task-specific Datasets, Standard Deviation=0.82023.05 | 59.4 | — | |
| SSTuning-baseBackbone=RoBERTa-base, Labeled=false, Setting=Zero-shot2023.05 | 59.1 | — | |
| UniMC (Rerun)Backbone=ALBERT-xxlarge, Labeled=true, Setting=Zero-shot2023.05 | 59 | — | |
| Mining (Re-implementation)Learning Paradigm=Zero-shot Learning via Generating Task-specific Datasets, Concurrent Work=true, Fair Comparison Setup=true, Standard Deviation=0.62023.05 | 57 | — | |
| TE-WikiBackbone=BERT-base, Setting=Zero-shot2023.05 | 56.5 | — | |
| Mining-based*Backbone=RoBERTa-large, Labeled=false, Setting=Zero-shot2023.05 | 56.1 | — | |
| LOTClassLearning Paradigm=Reference, Use task-specific corpus=true2023.05 | 55.7 | — | |
| GPT-3Learning Paradigm=Zero-shot Learning via Direct Inferencing, Billion-scale PLM=true2023.05 | 54.7 | — | |
| KNN-PromptLearning Paradigm=Zero-shot Learning via Direct Inferencing2023.05 | 51 | — | |
| X-ClassLearning Paradigm=Reference, Use task-specific corpus=true2023.05 | 50.5 | — | |
| PINFew-shot=5-shot, Policy LM=OPT-125M2024.07 | 49.5 | — | |
| RLPromptFew-shot=5-shot, Policy LM=OPT-125M2024.07 | 48.6 | — | |
| TE-MNLIBackbone=BART-large, Labeled=true, Setting=Zero-shot2023.05 | 48.2 | — | |
| NSP-BERTLearning Paradigm=Zero-shot Learning via Direct Inferencing2023.05 | 47 | — | |
| PromptLearning Paradigm=Zero-shot Learning via Direct Inferencing2023.05 | 44.1 | — | |
| TE-NLI (Best)Learning Paradigm=Labeled data usage, Use auxiliary labeled data=true2023.05 | 43.8 | — | |
| SuperGenLearning Paradigm=Zero-shot Learning via Generating Task-specific Datasets, Standard Deviation=1.52023.05 | 40.8 | — | |
| MiningLearning Paradigm=Zero-shot Learning via Generating Task-specific Datasets, Concurrent Work=true2023.05 | 40.1 | — | |
| In-Context DemonstrationFew-shot=5-shot2024.07 | 36.7 | — | |
| AutoPromptFew-shot=5-shot2024.07 | 35.5 | — | |
| Prompting*Backbone=RoBERTa-large, Setting=Zero-shot2023.05 | 34.1 | — | |
| TE-MNLIBackbone=RoBERTa-large, Labeled=true, Setting=Zero-shot2023.05 | 28.6 | — | |
| PEZFew-shot=5-shot2024.07 | 27 | — | |
| GrIPSFew-shot=5-shot2024.07 | 22.5 | — | |
| InstructionsFew-shot=5-shot2024.07 | 21.4 | — | |
| Manual PromptFew-shot=5-shot2024.07 | 18.1 | — |