Natural Language Understanding on SuperGLUE 1,000 examples
84BoolQ AccuracyLoRA
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LoRAModel=LLaMA2-7b2024.02 | 84 | 83.2 | 68.8 | 96 | 83 | |
| S-MeZOModel=LLaMA2-7b2024.02 | 82.2 | 77.6 | 65.3 | 94.8 | 80 | |
| MeZO-LoRAModel=LLaMA2-7b2024.02 | 80.3 | 76.5 | 63 | 94 | 78.5 | |
| AdaZetaModel=LLaMA2-7b2024.02 | 79.8 | 75.8 | 62 | 94 | 77.9 | |
| ZO-SGD-AdamModel=LLaMA2-7b2024.02 | 78.9 | 73.6 | 62.1 | 93.8 | 77.1 | |
| MeZOModel=LLaMA2-7b2024.02 | 78.8 | 70.2 | 62.2 | 94 | 76.3 | |
| ZO-AdaMUModel=LLaMA2-7b2024.02 | 78.2 | 76.5 | 63 | 93.6 | 77.8 | |
| ZO-SGD-ConsModel=LLaMA2-7b2024.02 | 77.1 | 68.6 | 63 | 94 | 75.7 | |
| R-MeZOModel=LLaMA2-7b2024.02 | 76.7 | 74 | 61.4 | 94.2 | 76.6 | |
| S-MeZOModel=OPT-13b, Description=Sparse MeZO2024.02 | 73.8 | 77.6 | 63.7 | — | — | |
| R-MeZOModel=OPT-13b, Description=MeZO with Random Mask2024.02 | 72.3 | 75.2 | 61.7 | — | — | |
| MeZOModel=OPT-13b2024.02 | 72.1 | 75.5 | 62.2 | — | — | |
| ZO-SGD-SignModel=LLaMA2-7b2024.02 | 68.2 | 61 | 55.6 | 82.8 | 66.9 | |
| ICLModel=OPT-13b, Full Name=In-Context Learning2024.02 | 66.9 | 62.1 | 50.5 | — | — | |
| Zero ShotModel=OPT-13b2024.02 | 59 | 59.6 | 55 | — | — |