Few-shot Text Classification on 26 few-shot tasks Class -> Non-Class transfer setting (test)
45.54AccuracyChar. Perturb
Evaluation results
| Method | Links | |
|---|---|---|
| Char. PerturbModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 45.54 | |
| Word Insert/DeleteModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 44.49 | |
| Mask PredictModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 44.05 | |
| Back TranslationModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.8 | |
| Self-LLMDAModel Scale=1.3b2024.04 | 43.8 | |
| Char. OCRModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.73 | |
| GPT2 InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.69 | |
| Data MixModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.67 | |
| POS AugmentModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.57 | |
| Word SwapModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.51 | |
| Contextual ReplaceModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.49 | |
| Word SwapModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.28 | |
| Paragraph ShuffleModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.28 | |
| Sentence ReorderModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.2 | |
| Word DeleteModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.12 | |
| Word ReplaceModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43.1 | |
| Char. DeleteModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 43.08 | |
| Grammar TransformModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 43 | |
| Back TranslationModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.96 | |
| BERT Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.86 | |
| Spell ErrorModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.85 | |
| Word2vec Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.8 | |
| Self-LLMDAModel Scale=350m2024.04 | 42.8 | |
| Char. InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.77 | |
| Char. SwapModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.76 | |
| Word2vec InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.66 | |
| Char. Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 42.46 | |
| OriginalModel Scale=1.3b2024.04 | 42.42 | |
| OriginalModel Scale=350m2024.04 | 42.04 | |
| Grammar TransformModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 41.34 | |
| Data MixModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 41.29 | |
| POS AugmentModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 41.14 | |
| Back TranslationModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 41.12 | |
| ParaphraseModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 41.09 | |
| Word ReplaceModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 41.05 | |
| Spell ErrorModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 41.03 | |
| Char. OCRModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 41.02 | |
| Contextual ReplaceModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.99 | |
| Word Insert/DeleteModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.97 | |
| ParaphraseModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.94 | |
| GPT2 InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.92 | |
| Back TranslationModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.89 | |
| Word SwapModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.84 | |
| Word2vec InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.8 | |
| Sentence ReorderModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.76 | |
| Paragraph ShuffleModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.7 | |
| Word SwapModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.66 | |
| Char. InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.61 | |
| Char. PerturbModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.61 | |
| Mask PredictModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 40.59 | |
| Char. SwapModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.56 | |
| Char. DeleteModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.5 | |
| Word DeleteModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.34 | |
| BERT Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.23 | |
| Char. Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.22 | |
| Word2vec Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 40.21 | |
| Self-LLMDAModel Scale=125m2024.04 | 40.02 | |
| GPT2 InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.79 | |
| Back TranslationModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.78 | |
| Word2vec InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.67 | |
| Mask PredictModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.58 | |
| BERT Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.57 | |
| Word DeleteModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.54 | |
| Char. SwapModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.46 | |
| ParaphraseModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.44 | |
| Char. Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.39 | |
| Char. PerturbModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.39 | |
| Sentence ReorderModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.37 | |
| Char. OCRModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.31 | |
| Grammar TransformModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.29 | |
| Paragraph ShuffleModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.29 | |
| Char. InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.22 | |
| Back TranslationModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.18 | |
| Word SwapModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 39.15 | |
| Word ReplaceModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.14 | |
| POS AugmentModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.12 | |
| Word SwapModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 39.01 | |
| Char. DeleteModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 38.99 | |
| Word2vec Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 38.9 | |
| Word Insert/DeleteModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 38.81 | |
| Contextual ReplaceModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 38.73 | |
| Spell ErrorModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 38.68 | |
| Data MixModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 38.5 | |
| OriginalModel Scale=125m2024.04 | 38.49 |