Few-shot Text Classification on 26 few-shot tasks Random -> Random transfer setting (test)
48.95AccuracySelf-LLMDA
Evaluation results
| Method | Links | |
|---|---|---|
| Self-LLMDAModel Scale=1.3b2024.04 | 48.95 | |
| Self-LLMDAModel Scale=350m2024.04 | 48.83 | |
| Self-LLMDAModel Scale=125m2024.04 | 46.64 | |
| Word ReplaceModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 45.89 | |
| Word SwapModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 45.79 | |
| POS AugmentModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 45.75 | |
| Sentence ReorderModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 45.64 | |
| Grammar TransformModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 45.53 | |
| Paragraph ShuffleModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 45.41 | |
| Contextual ReplaceModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 45.26 | |
| Contextual ReplaceModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 44.76 | |
| Mask PredictModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 44.64 | |
| Word ReplaceModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 44.57 | |
| Word Insert/DeleteModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 44.46 | |
| GPT2 InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 44.44 | |
| Paragraph ShuffleModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 44.43 | |
| Back TranslationModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 44.41 | |
| Word SwapModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 44.3 | |
| Word DeleteModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 44.28 | |
| Word2vec InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 44.18 | |
| OriginalModel Scale=1.3b2024.04 | 44.1 | |
| ParaphraseModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 44.1 | |
| Word2vec Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 44.06 | |
| Char. PerturbModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 44.06 | |
| ParaphraseModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.97 | |
| Back TranslationModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.95 | |
| Char. SwapModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.89 | |
| Char. OCRModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.83 | |
| BERT Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.79 | |
| Grammar TransformModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.75 | |
| Spell ErrorModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.71 | |
| Sentence ReorderModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.7 | |
| Char. PerturbModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.68 | |
| Word Insert/DeleteModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.68 | |
| Word SwapModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.63 | |
| Back TranslationModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 43.6 | |
| Word SwapModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 43.54 | |
| Char. DeleteModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.52 | |
| Word ReplaceModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.52 | |
| POS AugmentModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.49 | |
| Char. Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.41 | |
| Word DeleteModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.41 | |
| Back TranslationModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.36 | |
| Sentence ReorderModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.35 | |
| Char. InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.29 | |
| Char. PerturbModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.27 | |
| Data MixModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.26 | |
| Paragraph ShuffleModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.25 | |
| Word SwapModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.24 | |
| Back TranslationModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.24 | |
| Grammar TransformModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 43.24 | |
| POS AugmentModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.21 | |
| Back TranslationModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.2 | |
| BERT Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.17 | |
| Mask PredictModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 43.16 | |
| Char. Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.08 | |
| Spell ErrorModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 43.08 | |
| GPT2 InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 43.07 | |
| Word2vec Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 43.07 | |
| Char. OCRModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 43.02 | |
| Data MixModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.01 | |
| Char. OCRModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 42.95 | |
| Word2vec InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.95 | |
| Word Insert/DeleteModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 42.95 | |
| OriginalModel Scale=350m2024.04 | 42.92 | |
| BERT Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.92 | |
| Char. SwapModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.9 | |
| Char. InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.86 | |
| Word2vec InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.86 | |
| ParaphraseModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 42.8 | |
| Char. DeleteModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.77 | |
| Word2vec Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.76 | |
| Data MixModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 42.76 | |
| OriginalModel Scale=125m2024.04 | 42.73 | |
| Spell ErrorModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 42.71 | |
| Char. Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.65 | |
| GPT2 InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 42.65 | |
| Word SwapModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 42.63 | |
| Mask PredictModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 42.59 | |
| Contextual ReplaceModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 42.58 | |
| Char. DeleteModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 42.53 | |
| Char. SwapModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.45 | |
| Char. InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.37 | |
| Word DeleteModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 42.02 |