Natural Language Understanding on GLUE Various (val)
90.8MNLIPfeiffer Adapter
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Pfeiffer Adapter#Tuned=0.5%, #Stored=0.5%2023.05 | 90.8 | 71.9 | 92.1 | 88.4 | 86.7 | 90 | 86.6 | |
| Adapter#Tuned=0.5%, #Stored=0.5%2023.05 | 90.6 | 71.9 | 92.1 | 87.4 | 84.8 | 88 | 85.8 | |
| LORA#Tuned=0.5%, #Stored=0.5%2023.05 | 90.6 | 72.3 | 91.7 | 88.4 | 88.4 | 92 | 87.2 | |
| MAM Adapter#Tuned=0.5%, #Stored=0.5%2023.05 | 90.6 | 72.4 | 92.1 | 89.5 | 88.3 | 92 | 87.5 | |
| Full FT#Tuned=100%, #Stored=100%2023.05 | 90.2 | 75.6 | 92.4 | 86.6 | — | 94 | — | |
| PaFi#Tuned=0.5%, #Stored=0.5%2023.05 | 90.2 | 72.6 | 91.9 | 88.8 | 85.2 | 92 | 86.8 | |
| HiWirank (r)=4, #Tuned=0.5%, #Stored=0.03%2023.05 | 90.2 | 73.4 | 91.6 | 88.1 | 87.5 | 93 | 87.3 | |
| HiWirank (r)=16, #Tuned=2.0%, #Stored=0.03%2023.05 | 90.2 | 73.7 | 91.9 | 88.8 | 89.3 | 95 | 88.2 | |
| Our Full FT#Tuned=100%, #Stored=100%2023.05 | 90.1 | 74 | 91.9 | 88.1 | 87.5 | 96 | 88 | |
| Prefix Tuning#Tuned=0.5%, #Stored=0.5%2023.05 | 89.9 | 69.7 | 91.4 | 76.5 | 91.1 | 74 | 82.1 | |
| BitFit#Tuned=0.08%, #Stored=0.08%2023.05 | 89.5 | 71.9 | 91.4 | 88.1 | 85.7 | 89 | 85.9 | |
| Linear FT norm#Tuned=0.03%, #Stored=0.03%2023.05 | 88.4 | 67.9 | 75.3 | 53.4 | 75.9 | 74 | 73 | |
| Linear FT#Tuned=0%, #Stored=0%2023.05 | 52.4 | 67.6 | 31.2 | 54.5 | 68.3 | 72 | 58.7 |