Text Classification on GLUE (val)
92.4MRPC AccuracyGreedy soup
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Greedy soupBase Model=T5-base2022.03 | 92.4 | 79.1 | 60.2 | 94.7 | — | — | — | — | — | |
| BERTModel Size=large, +MaskSub=false2023.06 | 92 | 69.7 | 63.3 | 93.8 | 86.8 | 88.2 | 92.3 | 89.3 | 84.4 | |
| Best individual modelBase Model=T5-base2022.03 | 91.8 | 78.3 | 58.8 | 94.6 | — | — | — | — | — | |
| BERT + MaskSubModel Size=large, +MaskSub=true2023.06 | 91.5 | 69.3 | 65.2 | 94 | 87.1 | 89 | 92.7 | 88.6 | 84.7 | |
| MoE-LoRAModel=RoBERTalarge, Param x10^3=811, Rank=82025.06 | 89.9 | 82.6 | 61.1 | — | — | — | — | — | 77.8 | |
| LoRAModel=RoBERTalarge, Param x10^3=786, Rank=82025.06 | 89.3 | 83 | 68 | — | — | — | — | — | 80 | |
| BERT + MaskSubModel Size=base, +MaskSub=true2023.06 | 89.2 | 63.2 | 58.3 | 91.9 | 84.5 | 87.7 | 91.3 | 86.8 | 81.6 | |
| LoRAModel=RoBERTalarge, Param (×10^3)=786, Rank=82025.06 | 89 | 80.8 | 65 | — | — | — | 92.9 | — | 81.7 | |
| MoE-LoRAModel=RoBERTalarge, Param (×10^3)=814, Rank=82025.06 | 89 | 83 | 63 | — | — | — | 90.2 | — | 81 | |
| MetaTT-(4+1)DModel=RoBERTalarge, Param x10^3=18.2, Rank=82025.06 | 89 | 84.4 | 64 | — | — | — | — | — | 79.2 | |
| MTL-LoRAModel=RoBERTalarge, Param x10^3=789, Rank=42025.06 | 88.7 | 82.1 | 58.8 | — | — | — | — | — | 76.5 | |
| MTL-LoRAModel=RoBERTalarge, Param (×10^3)=789, Rank=42025.06 | 88.6 | 82 | 60 | — | — | — | 90.2 | — | 80.4 | |
| MoE-LoRAModel=RoBERTabase, Param x10^3=307, Rank=82025.06 | 88.6 | 82.8 | 60.1 | — | — | — | — | — | 77.2 | |
| BERTModel Size=base, +MaskSub=false2023.06 | 88.5 | 62.8 | 54.7 | 91.6 | 84.1 | 87.5 | 91 | 87 | 80.9 | |
| MetaTT-4DModel=RoBERTalarge, Param x10^3=18.0, Rank=82025.06 | 88.4 | 81.1 | 59.5 | — | — | — | — | — | 76.3 | |
| Best individual modelBase Model=BERT-base uncased2022.03 | 88.3 | 61 | 59.1 | 92.5 | — | — | — | — | — | |
| Greedy soupBase Model=BERT-base uncased2022.03 | 88.3 | 61.7 | 59.1 | 93 | — | — | — | — | — | |
| MetaTT-4DModel=RoBERTalarge, Param (×10^3)=18.0, Rank=82025.06 | 88.2 | 82.4 | 59 | — | — | — | 90.3 | — | 80 | |
| MTL-LoRAModel=RoBERTabase, Param x10^3=296, Rank=42025.06 | 87.8 | 71.6 | 53 | — | — | — | — | — | 70.8 | |
| MetaTT-(4+1)DModel=RoBERTalarge, Param (×10^3)=18.3, Rank=82025.06 | 87.6 | 83.4 | 63 | — | — | — | 91.1 | — | 81.2 | |
| MoE-LoRAModel=RoBERTabase, Param (×10^3)=309, Rank=82025.06 | 87.4 | 75.6 | 54.5 | — | — | — | 88.2 | — | 76.4 | |
| MetaTT-(4+1)DModel=RoBERTabase, Param (×10^3)=13.5, Rank=82025.06 | 87.3 | 70 | 54 | — | — | — | 85.8 | — | 74.5 | |
| LoRAModel=RoBERTabase, Param (×10^3)=295, Rank=82025.06 | 87.1 | 75 | 58.6 | — | — | — | 88 | — | 77.3 | |
| MTL-LoRAModel=RoBERTabase, Param (×10^3)=296, Rank=42025.06 | 87 | 76.4 | 51.5 | — | — | — | 87.6 | — | 75.6 | |
| LoRAModel=RoBERTabase, Param x10^3=295, Rank=82025.06 | 86.5 | 77.6 | 60.7 | — | — | — | — | — | 74.9 | |
| MetaTT-(4+1)DModel=RoBERTabase, Param x10^3=13.4, Rank=82025.06 | 86 | 71.5 | 54 | — | — | — | — | — | 70.5 | |
| MetaTT-4DModel=RoBERTabase, Param x10^3=13.2, Rank=82025.06 | 85.9 | 72 | 53.2 | — | — | — | — | — | 70.3 | |
| MetaTT-4DModel=RoBERTabase, Param (×10^3)=13.2, Rank=82025.06 | 84.8 | 75 | 53.8 | — | — | — | 85.9 | — | 74.8 |