NLU and Question Answering on SuperGLUE
94.7SST-2 AccuracyLoRA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| LoRAModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=LoRA2026.03 | 94.7 | 81.6 | 82.7 | 74.1 | 59.6 | 62.1 | 84.5 | 35.1 | 71.8 | |
| DiZOModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=Zeroth-Order (ZO)2026.03 | 94.4 | 65.6 | 66 | 66.8 | 56.7 | 57.1 | 62.1 | 22.5 | 61.4 | |
| CurvZOModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=Zeroth-Order (ZO)2026.03 | 94.1 | 67.7 | 69.8 | 68.9 | 62.9 | 61.9 | 81.5 | 27.6 | 66.8 | |
| FTModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=Full Fine-Tuning2026.03 | 93.8 | 80.1 | 82.1 | 72.6 | 63.4 | 69.1 | 82.9 | 33.2 | 72.2 | |
| CurvZO + LoRAModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=ZO + PEFT2026.03 | 93.6 | 61.7 | 69.6 | 66.1 | 63.4 | 57.6 | 81.1 | 27.6 | 65.1 | |
| MeZO + LoRAModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=ZO + PEFT2026.03 | 93 | 58.1 | 64.2 | 63.2 | 56.7 | 53.7 | 80.8 | 26.9 | 62.1 | |
| MeZOModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=Zeroth-Order (ZO)2026.03 | 92.1 | 63.3 | 67.8 | 66.7 | 58.6 | 59.8 | 79 | 26.5 | 64.2 | |
| DiZO + LoRAModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=ZO + PEFT2026.03 | 92 | 55.5 | 64.2 | 64 | 58.6 | 54.2 | 67.5 | 22.3 | 59.8 | |
| Zero-shotModel=OPT-2.7B, Number of training samples=1,000, Evaluation protocol=Zero-shot2026.03 | 56.3 | 54.1 | 50 | 47.6 | 36.5 | 52.6 | 29.8 | 9.5 | 42.1 |