Natural Language Understanding on GLUE and SuperGLUE (test val)
95.7SST-2SCALEARN UNIFORM
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SCALEARN UNIFORMBackbone=RoBERTa-Large, Learning Paradigm=Two-stage MTL2023.10 | 95.7 | 90.09 | 90.54 | 93.84 | 92.13 | 88.33 | 85.85 | 66.85 | 88.24 | 80.5 | 82.04 | 70.28 | 59.62 | 90.4 | 95 | 84.63 | — | |
| ADAPTERFUSIONBackbone=RoBERTa-Large, Learning Paradigm=Two-stage MTL2023.10 | 95.64 | 89.79 | 90.83 | 94.14 | 92.08 | 89.12 | 85.85 | 66.52 | 89.26 | 79.25 | 82.4 | 69.5 | 62.69 | 88.6 | 90.36 | 84.4 | — | |
| HYPERFORMERBackbone=RoBERTa-Large, Learning Paradigm=Joint MTL2023.10 | 95.57 | 89.75 | 90.05 | 94.37 | 91.8 | 86.76 | 88.69 | 62.34 | 85.16 | 79.78 | 82.16 | 51.93 | 61.86 | 89 | 84.52 | 82.25 | — | |
| LORABackbone=RoBERTa-Large, Learning Paradigm=STL2023.10 | 95.47 | 89.66 | 89.66 | 94.2 | 91.98 | 87.7 | 80.51 | 63.8 | 88.3 | 79.1 | 78.02 | 68.46 | 62.12 | 76.6 | 92.86 | 82.56 | — | |
| PROPETLBackbone=RoBERTa-Large, Learning Paradigm=STL2023.10 | 95.41 | 89.78 | 89.23 | 94.32 | 91.45 | 87.65 | 84.55 | 65.85 | 87.86 | 81.19 | 81.61 | 69.62 | 63.46 | 69 | 94.11 | 83.01 | — | |
| COMPACTER++Backbone=RoBERTa-Large, Learning Paradigm=STL2023.10 | 95.41 | 89.15 | 87.33 | 92.93 | 91.46 | 87.84 | 79.71 | 65.66 | 88.34 | 79.18 | 79.53 | 69.26 | 62.26 | 79 | 87.5 | 82.3 | — | |
| SCALEARNBackbone=RoBERTa-Large, Learning Paradigm=Two-stage MTL2023.10 | 95.36 | 89.67 | 89.7 | 93.98 | 92.29 | 88.28 | 85.78 | 67.2 | 85.43 | 80.08 | 82.43 | 70.16 | 66.73 | 91 | 93.93 | 84.8 | — | |
| FINETUNEBackbone=RoBERTa-Large, Learning Paradigm=STL2023.10 | 95.3 | 89.57 | 89.75 | 93.91 | 91.89 | 86.27 | 81.52 | 60.15 | 81.6 | 79.03 | 81.65 | 69.72 | 63.46 | 52 | 90.36 | 80.41 | — | |
| SCALEARN UNIFORM++Backbone=RoBERTa-Large, Learning Paradigm=Two-stage MTL2023.10 | 95.3 | 90.1 | 90.45 | 93.91 | 92.12 | 88.97 | 84.77 | 65.83 | 88.28 | 80.46 | 82.23 | 70.09 | 60.1 | 89.2 | 95.36 | 84.48 | — | |
| ADAPTERBackbone=RoBERTa-Large, Learning Paradigm=STL2023.10 | 95.24 | 89.62 | 89.87 | 94.13 | 91.81 | 87.82 | 81.23 | 64.07 | 88.52 | 80.73 | 82.36 | 69.16 | 63.25 | 71.9 | 92.68 | 82.83 | — | |
| ADAPTER-MBackbone=RoBERTa-Large, Learning Paradigm=Joint MTL2023.10 | 95.18 | 89.3 | 90.04 | 93.9 | 89.41 | 85.46 | 86.52 | 57.36 | 81.81 | 77.81 | 80.53 | 55.69 | 59.29 | 83.33 | 80.95 | 80.44 | — | |
| PROPETL MBackbone=RoBERTa-Large, Learning Paradigm=Joint MTL2023.10 | 95.03 | 88.93 | 88.25 | 93.69 | 90.44 | 86.11 | 86.4 | 57.86 | 83.85 | 77.86 | 78.65 | 51.2 | 45.83 | 90.33 | 87.5 | 80.13 | — | |
| (IA)3Backbone=RoBERTa-Large, Learning Paradigm=STL2023.10 | 94.95 | 88.69 | 87.79 | 91.72 | 91.39 | 86.37 | 80.79 | 64.7 | 87.47 | 77.91 | 80.97 | 68.65 | 60.58 | 74 | 90 | 81.73 | — | |
| SCALEARN++Backbone=RoBERTa-Large, Learning Paradigm=Two-stage MTL2023.10 | 94.61 | 90.13 | 90.22 | 94.49 | 92.35 | 87.7 | 86.21 | 67.23 | 87.53 | 80.14 | 82.51 | 69.4 | 62.82 | 89.8 | 94.29 | 84.63 | — | |
| DSGDType=FO, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=526.3GB2026.02 | 94.15 | — | — | — | — | — | 73.64 | — | 70.7 | 69.8 | 69.8 | 62.54 | — | — | — | 0 | — | |
| ADAPTERSOUPBackbone=RoBERTa-Large, Learning Paradigm=Two-stage MTL2023.10 | 94.04 | 57.29 | 84.01 | 73.71 | 71.16 | 71.51 | 63.54 | 42.22 | 68.09 | 31.32 | 72.48 | 62.54 | 62.98 | 54.5 | 71.88 | 65.42 | — | |
| DSGD-LoRAType=FO, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=629.1MB2026.02 | 93.92 | — | — | — | — | — | 71.84 | — | 71.1 | 68 | 67.1 | 61.6 | — | — | — | -1.68 | — | |
| DSGDType=FO, Network Topology=Ring Network, Model=OPT-1.3B, Cost=526.3GB2026.02 | 93.69 | — | — | — | — | — | 71.84 | — | 71.5 | 68.5 | 66.1 | 62.33 | — | — | — | — | 0 | |
| DSGD-LoRAType=FO, Network Topology=Ring Network, Model=OPT-1.3B, Cost=629.1MB2026.02 | 93.58 | — | — | — | — | — | 70.76 | — | 71.2 | 68.8 | 65.7 | 61.44 | — | — | — | — | -0.61 | |
| ChocoSGDType=FO, Network Topology=Ring Network, Model=OPT-1.3B, Cost=15.79GB2026.02 | 93.35 | — | — | — | — | — | 67.87 | — | 72.05 | 67.4 | 68.6 | 61.6 | — | — | — | — | -0.69 | |
| FINETUNE-MBackbone=RoBERTa-Large, Learning Paradigm=Joint MTL2023.10 | 93.2 | 88.23 | 89.81 | 92.48 | 85.41 | 79.25 | 84.12 | 51.48 | 74.48 | 75.07 | 78.99 | 52.4 | 58.01 | 77.67 | 81.55 | 77.48 | — | |
| SeedFloodType=ZO, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=400KB2026.02 | 92.89 | — | — | — | — | — | 63.9 | — | 71.9 | 61.9 | 66.7 | 57.99 | — | — | — | -5.98 | — | |
| SeedFloodType=ZO, Network Topology=Ring Network, Model=OPT-1.3B, Cost=400KB2026.02 | 92.78 | — | — | — | — | — | 63.18 | — | 71.9 | 63.7 | 67.2 | 57.99 | — | — | — | — | -4.13 | |
| ChocoSGDType=FO, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=15.79GB2026.02 | 92.09 | — | — | — | — | — | 70.76 | — | 71.6 | 68.1 | 68 | 61.76 | — | — | — | -1.85 | — | |
| DZSGDType=ZO, Network Topology=Ring Network, Model=OPT-1.3B, Cost=5.26TB2026.02 | 91.4 | — | — | — | — | — | 61.01 | — | 70.05 | 59.7 | 64.4 | 59.56 | — | — | — | — | -6.46 | |
| DZSGDType=ZO, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=5.26TB2026.02 | 91.28 | — | — | — | — | — | 66.07 | — | 71.6 | 61.4 | 65 | 59.25 | — | — | — | -6.04 | — | |
| DZSGD-LoRAType=ZO, Network Topology=Ring Network, Model=OPT-1.3B, Cost=6.29GB2026.02 | 91.17 | — | — | — | — | — | 54.51 | — | 71 | 59.95 | 64.4 | 57.99 | — | — | — | — | -8.25 | |
| DZSGD-LoRAType=ZO, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=6.29GB2026.02 | 90.14 | — | — | — | — | — | 54.87 | — | 71.2 | 56.3 | 64.7 | 58.46 | — | — | — | -10.36 | — | |
| AGZOBackbone=Qwen3-4b, Optimization=Zeroth-Order fine-tuning, Steps=20,0002026.01 | 89.2 | — | — | — | — | — | 84.8 | — | — | 85.3 | 82 | 67.8 | — | — | 87.5 | — | — | |
| ICLBackbone=Qwen3-4b, Protocol=In-context learning2026.01 | 88.7 | — | — | — | — | — | 83.5 | — | — | 84.9 | 82.7 | 61.5 | — | — | 82.1 | — | — | |
| Choco-LoRAType=FO, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=18.8MB2026.02 | 87.66 | — | — | — | — | — | 59.93 | — | 72.9 | 55.4 | 63.1 | 57.84 | — | — | — | -10.03 | — | |
| MEZOBackbone=Qwen3-4b, Optimization=Zeroth-Order fine-tuning, Steps=20,0002026.01 | 87.5 | — | — | — | — | — | 83.7 | — | — | 85 | 82.3 | 66.6 | — | — | 85.7 | — | — | |
| LOZOBackbone=Qwen3-4b, Optimization=Zeroth-Order fine-tuning, Steps=20,0002026.01 | 86.6 | — | — | — | — | — | 80.1 | — | — | 85.2 | 82.2 | 65.9 | — | — | 85.7 | — | — | |
| Choco-LoRAType=FO, Network Topology=Ring Network, Model=OPT-1.3B, Cost=18.8MB2026.02 | 83.95 | — | — | — | — | — | 59.21 | — | 72.7 | 55.5 | 63.1 | 59.88 | — | — | — | — | -8.96 | |
| ZeroBackbone=Qwen3-4b, Protocol=Zero-shot prompting2026.01 | 64.9 | — | — | — | — | — | 80.5 | — | — | 76.5 | 79 | 59.5 | — | — | 37.5 | — | — | |
| ZeroShotType=-, Network Topology=Ring Network, Model=OPT-1.3B, Cost=02026.02 | 53.56 | — | — | — | — | — | 53.43 | — | 70.5 | 45.4 | 45.5 | 56.9 | — | — | — | — | -23.91 | |
| ZeroShotType=-, Network Topology=Meshgrid Network, Model=OPT-1.3B, Cost=02026.02 | 53.56 | — | — | — | — | — | 53.43 | — | 70.5 | 45.4 | 45.5 | 56.9 | — | — | — | -24.93 | — |