Natural Language Understanding on GLUE (All 8 Tasks)
88.69RTE AccuracyPiSSA
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PiSSAParams=1.33M, Backbone=DeBERTa-v3-base2026.05 | 88.69 | 92.33 | 73.12 | 91.5 | 96.22 | 94.43 | 90.37 | 92 | 89.83 | |
| AdaLoRAParams=1.27M, Backbone=DeBERTa-v3-base2026.05 | 88.09 | 92.23 | 71.45 | 90.69 | 96.1 | 94.55 | 90.76 | 91.84 | 89.46 | |
| DoRAParams=1.27M, Backbone=DeBERTa-v3-base2026.05 | 86.04 | 92.07 | 70.85 | 90.93 | 95.79 | 94.1 | 90.29 | 91.79 | 88.98 | |
| LoRA-GParams=1.33M, Backbone=DeBERTa-v3-base, Initialization=Gaussian2026.05 | 85.2 | 91.99 | 69.82 | 89.95 | 94.95 | 93.87 | 90.65 | 91.6 | 88.5 | |
| PAdapterParams=1.18M, Backbone=DeBERTa-v3-base2026.05 | 85.2 | 92.04 | 68.77 | 89.46 | 95.61 | 94.29 | 90.33 | 91.54 | 88.41 | |
| LoRA-KParams=1.33M, Backbone=DeBERTa-v3-base, Initialization=Kaiming2026.05 | 84.84 | 92.03 | 70.69 | 90.28 | 95.64 | 93.84 | 89.96 | 91.68 | 88.62 | |
| HAdapterParams=1.22M, Backbone=DeBERTa-v3-base2026.05 | 84.48 | 91.91 | 68.64 | 89.95 | 95.53 | 94.11 | 90.13 | 91.48 | 88.28 | |
| Spectrally Restricted FinetuningParams=0.36M, Backbone=DeBERTa-v3-base2026.05 | 84.12 | 89.22 | 68.13 | 88.48 | 95.53 | 92.06 | 87.45 | 88.85 | 86.73 | |
| Full FTParams=184M, Backbone=DeBERTa-v3-base2026.05 | 83.75 | 92.4 | 69.19 | 89.46 | 95.63 | 94.03 | 89.9 | 91.6 | 88.25 | |
| BitFitParams=0.1M, Backbone=DeBERTa-v3-base2026.05 | 78.7 | 88.41 | 66.96 | 87.75 | 94.84 | 92.24 | 89.37 | 91.35 | 86.2 |