Natural Language Understanding on GLUE DeBERTaV3-base (test)
72.03CoLA ScoreLOFT (P_skew)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LOFT (P_skew)r=46, support=P_skew (SkewGrad), #Params=0.07M, Mem.(GB)=4.12026.05 | 72.03 | 91.75 | 86.83 | 92.36 | 95.42 | 93.97 | 88.73 | |
| LOFT (P_grad)r=46, support=P_grad (GradSVD), #Params=0.07M, Mem.(GB)=4.12026.05 | 70.63 | 91.32 | 87.05 | 91.67 | 95.28 | 94.39 | 88.39 | |
| LOFT (P_prin)r=46, support=P_prin, #Params=0.07M, Mem.(GB)=4.12026.05 | 70.52 | 91.37 | 86.19 | 90.9 | 95.73 | 93.63 | 88.06 | |
| PSOFTr=46, #Params=0.08M, Mem.(GB)=4.12026.05 | 70.42 | 91.56 | 86.74 | 90.49 | 95.55 | 93.47 | 88.04 | |
| BOFTb=8, m=2, #Params=1.41M, Mem.(GB)=6.32026.05 | 68.85 | 91.09 | 83.6 | 88.4 | 95.28 | 93.78 | 86.83 | |
| qGOFTv2#Params=0.33M, Mem.(GB)=18.52026.05 | 68.03 | — | — | — | — | — | — | |
| LoRAr=8, #Params=1.33M, Mem.(GB)=4.52026.05 | 67.98 | 91.6 | 84.87 | 90.2 | 95.28 | 93.89 | 87.3 | |
| FFT#Params=184M, Mem.(GB)=5.92026.05 | 67.56 | 91.46 | 82.88 | 90.69 | 94.13 | 93.37 | 86.68 | |
| DoRAr=8, #Params=1.41M, Mem.(GB)=5.82026.05 | 67.06 | 91.6 | 87.19 | 90.49 | 95.23 | 94.09 | 87.61 | |
| OFTv2b=32, #Params=1.29M, Mem.(GB)=4.52026.05 | 66.79 | 91.22 | 84.03 | 89.61 | 93.72 | 92.64 | 86.34 | |
| PiSSAr=8, #Params=1.33M, Mem.(GB)=4.52026.05 | 66.5 | 91.4 | 83.77 | 89.9 | 93.17 | 92.72 | 86.24 | |
| GOFTv2#Params=0.08M, Mem.(GB)=18.52026.05 | 65.45 | — | — | — | — | — | — | |
| LoRA-XSr=136, #Params=1.33M, Mem.(GB)=4.22026.05 | 64.67 | 91.48 | 84.17 | 91.27 | 93.85 | 93.14 | 86.43 |