Few-shot Text Classification on 26 few-shot tasks Non-Class -> Class transfer setting (test)
0.5275AccuracySelf-LLMDA
Evaluation results
| Method | Links | |
|---|---|---|
| Self-LLMDAModel Scale=350m2024.04 | 0.5275 | |
| Self-LLMDAModel Scale=125m2024.04 | 0.5 | |
| Self-LLMDAModel Scale=1.3b2024.04 | 0.4948 | |
| Sentence ReorderModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4846 | |
| Word2vec InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4836 | |
| Paragraph ShuffleModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4802 | |
| ParaphraseModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4802 | |
| Word ReplaceModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4798 | |
| Paragraph ShuffleModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.477 | |
| Word ReplaceModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4762 | |
| Contextual ReplaceModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4762 | |
| Back TranslationModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4739 | |
| Word ReplaceModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4736 | |
| ParaphraseModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4731 | |
| POS AugmentModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4725 | |
| Char. SwapModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4708 | |
| POS AugmentModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4706 | |
| Word SwapModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4704 | |
| Char. PerturbModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4702 | |
| Back TranslationModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4702 | |
| BERT Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4687 | |
| Grammar TransformModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4681 | |
| Word SwapModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.468 | |
| Grammar TransformModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4676 | |
| Char. InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4669 | |
| Word Insert/DeleteModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4659 | |
| Word Insert/DeleteModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4658 | |
| OriginalModel Scale=125m2024.04 | 0.4649 | |
| Grammar TransformModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4641 | |
| BERT Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4635 | |
| Sentence ReorderModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4635 | |
| Spell ErrorModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4634 | |
| Sentence ReorderModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.463 | |
| Word SwapModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4625 | |
| Char. PerturbModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4625 | |
| Word SwapModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4621 | |
| Word2vec Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4617 | |
| Char. Subs.Model Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4611 | |
| GPT2 InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4611 | |
| POS AugmentModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4607 | |
| Char. OCRModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4602 | |
| Data MixModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4602 | |
| Mask PredictModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4602 | |
| Char. DeleteModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4601 | |
| Back TranslationModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4597 | |
| GPT2 InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4593 | |
| Word Insert/DeleteModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4593 | |
| Char. OCRModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4591 | |
| Word DeleteModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.457 | |
| Mask PredictModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.457 | |
| Data MixModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4568 | |
| Word2vec Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4558 | |
| Back TranslationModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4558 | |
| Word DeleteModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4556 | |
| Contextual ReplaceModel Scale=125m, Augmentation Group=Manual-LLMDA2024.04 | 0.4538 | |
| BERT Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4532 | |
| Char. PerturbModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4532 | |
| Back TranslationModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4531 | |
| Char. Subs.Model Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4522 | |
| Word SwapModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.452 | |
| Word2vec InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4513 | |
| Char. OCRModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4511 | |
| Back TranslationModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.451 | |
| GPT2 InsertModel Scale=125m, Augmentation Group=Non-LLMDA2024.04 | 0.4507 | |
| Char. DeleteModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4504 | |
| Paragraph ShuffleModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.4502 | |
| Spell ErrorModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4497 | |
| Word SwapModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4493 | |
| Spell ErrorModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4485 | |
| Char. SwapModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4469 | |
| Char. InsertModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4461 | |
| Mask PredictModel Scale=350m, Augmentation Group=Manual-LLMDA2024.04 | 0.446 | |
| OriginalModel Scale=1.3b2024.04 | 0.4452 | |
| Word DeleteModel Scale=350m, Augmentation Group=Non-LLMDA2024.04 | 0.4449 | |
| Word2vec Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4434 | |
| OriginalModel Scale=350m2024.04 | 0.4413 | |
| Char. SwapModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4406 | |
| Char. DeleteModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4402 | |
| ParaphraseModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4399 | |
| Data MixModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4393 | |
| Contextual ReplaceModel Scale=1.3b, Augmentation Group=Manual-LLMDA2024.04 | 0.4392 | |
| Word2vec InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4378 | |
| Char. Subs.Model Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4325 | |
| Char. InsertModel Scale=1.3b, Augmentation Group=Non-LLMDA2024.04 | 0.4244 |