Few-shot and Zero-shot Evaluation on NLU Suite (SST-2, CB, RTE, COPA, ReCoRD, SQuAD test)
93.7SST-2 AccuracyLoRA-nBAR
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LoRA-nBARBackbone=OPT-1.3B, Setting=Few-shot learning2024.10 | 93.7 | 79.8 | 70.5 | 78 | 71 | 82.3 | 79.2 | |
| LoRA-oBARBackbone=OPT-1.3B, Setting=Few-shot learning2024.10 | 93.6 | 75.6 | 70.4 | 78 | 70.9 | 82.5 | 78.5 | |
| LoRA-SAMBackbone=OPT-1.3B, Setting=Few-shot learning2024.10 | 93.5 | 74.3 | 70.6 | 78 | 70.9 | 83 | 78.4 | |
| LoRABackbone=OPT-1.3B, Setting=Few-shot learning2024.10 | 93.1 | 72.6 | 69.1 | 78 | 70.8 | 81.9 | 77.6 | |
| PrefixBackbone=OPT-1.3B, Setting=Few-shot learning2024.10 | 92.9 | 71.6 | 65.2 | 73 | 69.7 | 82.1 | 75.8 | |
| Zero-ShotBackbone=OPT-1.3B, Setting=Zero-shot2024.10 | 53.6 | 39.3 | 53.1 | 75 | 70.2 | 27.2 | 53.1 |