Few-shot Language Understanding on SST-2, SST-5, SNLI, MNLI, RTE, TREC (k=16)
91.8Accuracy (SST-2)FO
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FOBackbone=RoBERTa-large, k (shots)=16, Memory constraints=≥ 4× memory2026.05 | 91.8 | 47.5 | 77.5 | 70 | 66.4 | 85 | 73 | |
| AdaMeZOBackbone=RoBERTa-large, k (shots)=162026.05 | 90.9 | 45.2 | 66.8 | 58.6 | 63.1 | 71.5 | 66 | |
| MeZOBackbone=RoBERTa-large, k (shots)=162026.05 | 90.6 | 44.1 | 67.3 | 58.1 | 61.6 | 67.3 | 64.8 | |
| MeZO-switchBackbone=RoBERTa-large, k (shots)=162026.05 | 90.6 | 44.3 | 67.3 | 58 | 61.6 | 67 | 64.8 | |
| Zero-shotBackbone=RoBERTa-large, k (shots)=162026.05 | 79 | 35.5 | 50.2 | 48.8 | 51.4 | 32 | 49.4 |