Natural Language Understanding on GLUE (test) (SST-2, CoLA, MRPC, QQP, MNLI-mm, QNLI, STS-B, RTE)
94.9QNLILoRA
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LoRABackbone=RoBERTa-large, # Params=0.8M2026.02 | 94.9 | 90.6 | 91.6 | 96.2 | 68.2 | 92.6 | 90.9 | 87.4 | 89 | — | — | — | — | |
| AdptPBackbone=RoBERTa-large, # Params=3.0M, setup=restricted2026.02 | 94.8 | 90.2 | 91.9 | 96.1 | 68.3 | 92.1 | 90.2 | 83.8 | 88.4 | — | — | — | — | |
| AdptPBackbone=RoBERTa-large, # Params=0.8M, setup=restricted2026.02 | 94.8 | 90.5 | 91.7 | 96.6 | 67.8 | 91.9 | 89.7 | 80.1 | 87.9 | — | — | — | — | |
| LoRABackbone=RoBERTa-large, # Params=0.8M, setup=restricted2026.02 | 94.8 | 90.6 | 91.6 | 96.2 | 68.2 | 92.3 | 90.2 | 85.2 | 88.6 | — | — | — | — | |
| FTBackbone=RoBERTa-large, # Params=355.0M2026.02 | 94.7 | 90.2 | 92.2 | 96.4 | 68 | 92.4 | 90.9 | 86.6 | 88.9 | — | — | — | — | |
| AdptHBackbone=RoBERTa-large, # Params=6.0M, setup=restricted2026.02 | 94.7 | 89.9 | 92.1 | 96.2 | 66.5 | 91 | 88.7 | 83.4 | 87.8 | — | — | — | — | |
| AdptHBackbone=RoBERTa-large, # Params=0.8M, setup=restricted2026.02 | 94.7 | 90.3 | 91.5 | 96.3 | 66.3 | 91.5 | 87.7 | 72.9 | 86.4 | — | — | — | — | |
| CRAFTBackbone=RoBERTa-large, # Params=0.04M2026.02 | 94.7 | 90.2 | 89.7 | 96.2 | 67.7 | 91.4 | 90.2 | 84.2 | 88 | — | — | — | — | |
| LoRABackbone=RoBERTa-base, # Params=0.3M2026.02 | 93.3 | 87.5 | 90.8 | 95.1 | 63.4 | 91.5 | 89.7 | 86.6 | 87.2 | — | — | — | — | |
| AdptDBackbone=RoBERTa-base, # Params=0.3M2026.02 | 93.1 | 87.1 | 90.2 | 94.2 | 60.8 | 89.7 | 88.5 | 71.5 | 84.4 | — | — | — | — | |
| AdptDBackbone=RoBERTa-base, # Params=0.9M2026.02 | 93 | 87.3 | 90.6 | 94.7 | 62.6 | 90.3 | 88.4 | 75.9 | 85.4 | — | — | — | — | |
| FTBackbone=RoBERTa-base, # Params=125.0M2026.02 | 92.8 | 87.6 | 91.9 | 94.8 | 63.6 | 91.2 | 90.2 | 78.7 | 86.4 | — | — | — | — | |
| CRAFTBackbone=RoBERTa-base, # Params=0.04M2026.02 | 92 | 85.7 | 89 | 95.1 | 59 | 90.4 | 89.2 | 75.8 | 84.5 | — | — | — | — | |
| BitFitBackbone=RoBERTa-base, # Params=0.1M2026.02 | 91.8 | 84.7 | 84 | 93.7 | 62 | 90.8 | 92.7 | 81.5 | 85.2 | — | — | — | — | |
| RINGBackbone=RoBERTa-base, Evaluation Protocol=Fine-tuning, Fine-tuning Technique=LoRA (r=8, alpha=8)2026.01 | 91.5 | 98.6 | 88.8 | 92.4 | 58.5 | 88.2 | 88.2 | 76.5 | — | — | — | — | — | |
| SA-MergingMerging Category=Data-free, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 91.5 | — | 84.6 | 93.2 | 69.8 | 85.5 | 84 | 83 | 82.5 | — | — | — | 68 | |
| IndividualMerging Category=Non-merging, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 90.9 | — | 84 | 92.9 | 69.1 | 87.4 | 85.5 | 84.4 | 84.6 | — | — | — | 82.7 | |
| R-KalmanBackbone=RoBERTa-base, Evaluation Protocol=Fine-tuning, Fine-tuning Technique=LoRA (r=8, alpha=8)2026.01 | 90.8 | 87.5 | 86.1 | 90.9 | 59.5 | 89.8 | 88.2 | 74.6 | — | — | — | — | — | |
| NGDBackbone=RoBERTa-base, Evaluation Protocol=Fine-tuning, Fine-tuning Technique=LoRA (r=8, alpha=8)2026.01 | 90.6 | 97.2 | 88.6 | 91.2 | 56.2 | 81.1 | 86.5 | 71 | — | — | — | — | — | |
| RENGBackbone=RoBERTa-base, Evaluation Protocol=Fine-tuning, Fine-tuning Technique=LoRA (r=8, alpha=8)2026.01 | 90.5 | 97.6 | 90.2 | 92.6 | 62.2 | 89.2 | 88.1 | 75.2 | — | — | — | — | — | |
| AdamWBackbone=RoBERTa-base, Evaluation Protocol=Fine-tuning, Fine-tuning Technique=LoRA (r=8, alpha=8)2026.01 | 90 | 85.5 | 87.2 | 90.4 | 62 | 88.8 | 87.8 | 71.8 | — | — | — | — | — | |
| AdaMerging++Merging Category=Test-time / data-assisted, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 90 | — | 83.6 | 91.6 | 69.1 | 74.1 | 78.4 | 79.1 | 78.3 | — | — | — | 60.3 | |
| PCBMerging Category=Data-free, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 90 | — | 83.7 | 91.8 | 68 | 79.5 | 80 | 79 | 79 | — | — | — | 60 | |
| TIESMerging Category=Data-free, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 89.8 | — | 83.7 | 91.6 | 68.3 | 71.2 | 79.4 | 79.4 | 77.5 | — | — | — | 56.3 | |
| Task Arithmetic.Merging Category=Data-free, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 89.2 | — | 83.2 | 91.2 | 67.5 | 71 | 78 | 78 | 76.7 | — | — | — | 55.5 | |
| Weight Avg.Merging Category=Data-free, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 89 | — | 83 | 91 | 67 | 70 | 77 | 77 | 76.1 | — | — | — | 55 | |
| SD-Clip2Straggler=Large, Async Mode=Server2026.02 | 87.59 | 82.5 | 84.47 | 92.78 | 81.4 | 85.6 | 84.31 | 54.51 | 81.65 | — | — | — | — | |
| WUDIMerging Category=Data-free, Base Model=Flan-T5-base, Model Adaptation=LoRA2026.05 | 87.2 | — | 83.1 | 93.2 | 68.6 | 85 | 77.7 | 75.8 | 81.2 | — | — | — | 79 | |
| Clip2Straggler=Large, Async Mode=Client2026.02 | 86.89 | 82.19 | 85.39 | 92.29 | 80.15 | 86.92 | 85.05 | 67.87 | 81.33 | — | — | — | — | |
| SD-Clip2Straggler=Large, Async Mode=Client2026.02 | 86.55 | 83.35 | 86.96 | 91.51 | 80.73 | 86.6 | 86.52 | 72.56 | 84.35 | — | — | — | — | |
| DC-Clip2Straggler=Large, Async Mode=Server2026.02 | 86.47 | 83.5 | 86.92 | 92.78 | 80.57 | 85.6 | 77.45 | 58.14 | 81.43 | — | — | — | — | |
| DC-Clip2Straggler=Large, Async Mode=Client2026.02 | 86.36 | 82.29 | 84.44 | 91.63 | 81.78 | 87.91 | 86.03 | 66.43 | 83.36 | — | — | — | — | |
| Clip2Straggler=Large, Async Mode=Server2026.02 | 85.78 | 82.58 | 86.09 | 91.97 | 80.44 | 85.63 | 83.58 | 59.95 | 82 | — | — | — | — | |
| FADASStraggler=Large, Async Mode=N/A2026.02 | 80.58 | 79.28 | 83.82 | 89.22 | 79.44 | 79.6 | 77.23 | 55.6 | 78.1 | — | — | — | — | |
| SaES-ID-CRTime/FE (s)=0.15292026.03 | 59.86 | 52.44 | — | 90.37 | — | — | — | 58.56 | — | 81.36 | 54.05 | 4.75 | — | |
| (1+1)-ES-ID-CRTime/FE (s)=0.15012026.03 | 59.8 | 49.04 | — | 89.66 | — | — | — | 56.1 | — | 81.5 | 54.37 | 8.32 | — | |
| ZOSGD-CRTime/FE (s)=0.14992026.03 | 59.48 | 46.6 | — | 89.29 | — | — | — | 54.66 | — | 79.37 | 54.04 | 3.68 | — | |
| ZOSGDTime/FE (s)=0.14982026.03 | 59 | 46.42 | — | 86.74 | — | — | — | 53.65 | — | 77.69 | 46.16 | 0.81 | — | |
| (1+1)-ESTime/FE (s)=0.14512026.03 | 54.42 | 46.42 | — | 86.95 | — | — | — | 52.92 | — | 77.33 | 49.52 | 4.67 | — | |
| BBTTime/FE (s)=0.14422026.03 | 54.3 | 46.48 | — | 88.03 | — | — | — | 54.3 | — | 77.33 | 51.09 | 3.94 | — | |
| SaESTime/FE (s)=0.14922026.03 | 53.84 | 46.54 | — | 87.41 | — | — | — | 52.92 | — | 77.33 | 51.62 | 5.2 | — | |
| BDPL2026.03 | 53.1 | 42.5 | — | 87.6 | — | — | — | 53.5 | — | 78.1 | 56.4 | 4.6 | — | |
| SaES-IDTime/FE (s)=0.15032026.03 | 52.88 | 46.42 | — | 86.83 | — | — | — | 53.86 | — | 77.33 | 50.93 | 4.56 | — | |
| (1+1)-ES-IDTime/FE (s)=0.14642026.03 | 52.68 | 46.42 | — | 86.95 | — | — | — | 52.71 | — | 77.33 | 47.74 | 3.71 | — | |
| RLPrompt2026.03 | 52.6 | 42.8 | — | 88.4 | — | — | — | 51.8 | — | 68.9 | 53.7 | 5 | — | |
| ICL2026.03 | 52.06 | 47.1 | — | 69.36 | — | — | — | 54.15 | — | 48.13 | 45.73 | 0.54 | — | |
| ManualPrompt2026.03 | 48.7 | 40.8 | — | 78.1 | — | — | — | 50.18 | — | 77.69 | 29.37 | 0.79 | — |