Natural Language Understanding on GLUE (SST-2, MRPC, CoLA, QNLI, RTE, STS-B Subset)
97.3SST-2 AccuracyMoC
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MoCModel=DeBERTa-Large2026.06 | 97.3 | 92.7 | 77 | 96.2 | 93.9 | 92.9 | 91.7 | |
| MoCModel=RoBERTa-Large2026.06 | 97.2 | 92.6 | 71.4 | 95.2 | 89.9 | 92.5 | 89.8 | |
| MoCModel=DeBERTa-Base2026.06 | 96.9 | 91 | 71.8 | 94.5 | 87.8 | 91.9 | 89 | |
| LoRA*Model=DeBERTa-Large2026.06 | 96.9 | 91.8 | 75 | 95.9 | 91 | 92.7 | 90.5 | |
| Control*Model=RoBERTa-Large2026.06 | 96.7 | 90.9 | 69.9 | 95.2 | 86.3 | 92.5 | 88.6 | |
| ControlModel=DeBERTa-Large2026.06 | 96.7 | 92.7 | 76.4 | 95.9 | 90 | 92.3 | 90.6 | |
| AdapterPBackbone=RoBERTa-large, Params (%)=0.222026.05 | 96.6 | 89.7 | 67.8 | 94.8 | 80.1 | 91.9 | 86.8 | |
| DoRA*Model=RoBERTa-Large2026.06 | 96.6 | 90.9 | 68.8 | 95.2 | 84.7 | 92.5 | 88.1 | |
| DoRA*Model=DeBERTa-Large2026.06 | 96.6 | 92.1 | 76.9 | 95.8 | 91.1 | 92.6 | 90.9 | |
| MoLExModel=DeBERTa-Large2026.06 | 96.6 | 91.9 | 75.9 | 95.9 | 91.6 | 92.7 | 90.8 | |
| FullBackbone=RoBERTa-large, Params (%)=100.02026.05 | 96.4 | 90.9 | 68 | 94.7 | 86.6 | 92.4 | 88.2 | |
| S2FTBackbone=RoBERTa-large, Params (%)=0.012026.05 | 96.4 | 90.7 | 67.6 | 94.6 | 88.1 | 92.3 | 88.3 | |
| MoLEx*Model=RoBERTa-Large2026.06 | 96.4 | 91.4 | 68.2 | 94.8 | 87.1 | 92 | 88.3 | |
| ControlModel=DeBERTa-Base2026.06 | 96.3 | 89.8 | 71.3 | 94.4 | 84.8 | 91.5 | 88 | |
| LoRABackbone=RoBERTa-large, Params (%)=0.222026.05 | 96.2 | 90.2 | 68.2 | 94.8 | 85.2 | 92.3 | 87.8 | |
| LoRA*Model=RoBERTa-Large2026.06 | 96.2 | 90.2 | 68.2 | 94.8 | 85.2 | 92.3 | 87.8 | |
| LoRA*Model=DeBERTa-Base2026.06 | 96.2 | 89.4 | 70.4 | 94.3 | 83.4 | 91 | 87.5 | |
| AdapterPBackbone=RoBERTa-large, Params (%)=0.862026.05 | 96.1 | 90.2 | 68.3 | 94.8 | 83.8 | 92.1 | 87.6 | |
| LR-LoRA-4Backbone=RoBERTa-large, Params=1.8M†2026.06 | 96.1 | 90.8 | 69.5 | 94.9 | 86.2 | 92.1 | 88.3 | |
| FourierFTBackbone=RoBERTa-large, Params (%)=0.012026.05 | 96 | 90.9 | 67.1 | 94.4 | 87.4 | 91.9 | 88 | |
| DoRA*Model=DeBERTa-Base2026.06 | 96 | 90.2 | 71.2 | 94.3 | 84 | 91 | 87.8 | |
| MoLExModel=DeBERTa-Base2026.06 | 96 | 90.1 | 71 | 94.3 | 85.8 | 91.4 | 88.1 | |
| PiSSABackbone=RoBERTa-large, Params (%)=0.222026.05 | 95.8 | 91.5 | 68.1 | 94.4 | 87.9 | 92 | 88.2 | |
| VeRA-256Backbone=RoBERTa-large, Params=0.26M2026.06 | 95.8 | 89.3 | 65.3 | 94.1 | 81.6 | 91.8 | 86.3 | |
| SineLoRA-4Backbone=RoBERTa-large, Params=1.8M2026.06 | 95.6 | 88.9 | 66.1 | 94.4 | 83.9 | 91.6 | 86.8 | |
| MoCModel=RoBERTa-Base2026.06 | 95.6 | 90.7 | 66.7 | 93.4 | 81.9 | 91.5 | 86.7 | |
| LoRA-4Backbone=RoBERTa-large, Params=1.8M2026.06 | 95.5 | 87.2 | 64.7 | 94.5 | 83.6 | 91.8 | 86.2 | |
| RandLoRA-100Backbone=RoBERTa-large, Params=1.8M2026.06 | 95.5 | 90.1 | 67.4 | 94.1 | 84.5 | 91.4 | 87.2 | |
| MoLEx*Model=RoBERTa-Base2026.06 | 95.4 | 89.8 | 64.8 | 93.2 | 77.3 | 91 | 85.3 | |
| S2FTBackbone=RoBERTa-base, Params (%)=0.022026.05 | 95.3 | 90.4 | 64.8 | 92.8 | 79.8 | 90.7 | 85.6 | |
| Control*Model=RoBERTa-Base2026.06 | 95.3 | 89.3 | 65.3 | 93.1 | 76.9 | 90.6 | 85.1 | |
| LoRABackbone=RoBERTa-base, Params (%)=0.242026.05 | 95.1 | 89.7 | 63.4 | 93.3 | 78.4 | 91.5 | 85.2 | |
| LoRA*Model=RoBERTa-Base2026.06 | 95.1 | 89.7 | 63.4 | 93.3 | 78.4 | 91.5 | 85.2 | |
| DoRA*Model=RoBERTa-Base2026.06 | 95 | 89.7 | 64.9 | 92.9 | 79.2 | 91.3 | 85.5 | |
| FullBackbone=RoBERTa-base, Params (%)=100.02026.05 | 94.8 | 90.2 | 63.6 | 92.8 | 78.7 | 91.2 | 85.2 | |
| AdapterDBackbone=RoBERTa-base, Params (%)=0.722026.05 | 94.7 | 88.4 | 62.6 | 93 | 75.9 | 90.3 | 84.2 | |
| DyLoRABackbone=RoBERTa-large, Params (%)=0.222026.05 | 94.7 | 90.7 | 65.3 | 93.6 | 87.2 | 91.4 | 87.2 | |
| LR-LoRA-4Backbone=RoBERTa-base, Params=0.7M†2026.06 | 94.6 | 89.1 | 63.2 | 92.9 | 76.5 | 90.8 | 84.5 | |
| AdaLoRABackbone=RoBERTa-base, Params (%)=0.242026.05 | 94.5 | 88.7 | 62 | 93.1 | 81 | 90.5 | 85 | |
| SAD-LoRA-CoeffRank=42026.07 | 94.5 | 90 | 53.9 | 92.2 | 63.1 | 88.5 | — | |
| SAD-LoRARank=82026.07 | 94.5 | 90.9 | 55.8 | 92.5 | 71.8 | 89.1 | — | |
| LoRA-4Backbone=RoBERTa-base, Params=0.7M2026.06 | 94.4 | 87.3 | 58.4 | 92.7 | 71.5 | 90.5 | 82.4 | |
| DyLoRABackbone=RoBERTa-base, Params (%)=0.242026.05 | 94.3 | 89.5 | 61.1 | 92.2 | 78.7 | 91.1 | 84.5 | |
| SAD-LoRARank=42026.07 | 94.3 | 89.9 | 55.8 | 92.1 | 69.9 | 88.4 | — | |
| AdapterDBackbone=RoBERTa-base, Params (%)=0.242026.05 | 94.2 | 88.5 | 60.8 | 93.1 | 71.5 | 89.7 | 83 | |
| FourierFTBackbone=RoBERTa-base, Params (%)=0.022026.05 | 94.2 | 90 | 63.8 | 92.2 | 79.1 | 90.8 | 85 | |
| SineLoRA-4Backbone=RoBERTa-base, Params=0.7M2026.06 | 94.1 | 87.8 | 59.1 | 92.2 | 72.8 | 90.4 | 82.7 | |
| SAD-LoRA-AlignRank=42026.07 | 94.1 | 90.1 | 55.3 | 92.5 | 70 | 88.7 | — | |
| PiSSA-InitRank=82026.07 | 94 | 91.1 | 53.8 | 92.3 | 68.8 | 88.7 | — | |
| Logit-MSERank=42026.07 | 93.8 | 87.2 | 42.3 | 90.8 | 59 | 78.9 | — | |
| NN-InitRank=82026.07 | 93.8 | 90.2 | 55.4 | 92.8 | 72.1 | 89.2 | — | |
| BitFitBackbone=RoBERTa-base, Params (%)=0.082026.05 | 93.7 | 92.7 | 62 | 91.8 | 81.5 | 90.8 | 85.4 | |
| PiSSA-InitRank=42026.07 | 93.7 | 90.2 | 51.9 | 92.1 | 66.1 | 88 | — | |
| KD-LoRARank=82026.07 | 93.7 | 90.2 | 47.8 | 92 | 67.4 | 84.7 | — | |
| SAD-LoRA-CoeffRank=82026.07 | 93.7 | 90.5 | 53.7 | 92.5 | 68 | 89.1 | — | |
| SAD-LoRA-AlignRank=82026.07 | 93.6 | 90.1 | 56.2 | 92.8 | 72.2 | 89.3 | — | |
| Logit-MSERank=82026.07 | 93.5 | 89.8 | 47.3 | 91.9 | 65.2 | 84.7 | — | |
| NN-InitRank=42026.07 | 93.4 | 90.3 | 55.5 | 92.5 | 69.2 | 88.7 | — | |
| KD-LoRARank=42026.07 | 93.3 | 87.8 | 42.6 | 91.3 | 65 | 78.9 | — | |
| CorDABackbone=RoBERTa-base, Params (%)=0.242026.05 | 93.1 | 89.7 | 59.6 | 91.5 | 76.2 | 90.2 | 83.4 | |
| RandLoRA-64Backbone=RoBERTa-base, Params=0.7M2026.06 | 92.2 | 88 | 59.4 | 91.3 | 74.7 | 90.3 | 82.6 | |
| VeRA-1024Backbone=RoBERTa-base, Params=0.26M2026.06 | 91.9 | 88.4 | 59.9 | 90.5 | 74.9 | 90.4 | 82.7 |