Natural Language Understanding on SuperGLUE RoBERTa-large (test)
89.21ReCoRDADAPTERFUSION
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ADAPTERFUSIONLearning Strategy=Two-stage MTL, Backbone=RoBERTa-large2023.10 | 89.21 | 80.52 | 82.21 | 69.09 | 63.46 | 81.2 | 95.71 | 86.06 | 80.93 | |
| SCALEARNUNIFORMLearning Strategy=Two-stage MTL, Backbone=RoBERTa-large2023.10 | 88.85 | 80.42 | 81.85 | 69.91 | 61.54 | 82 | 90 | 84.04 | 79.83 | |
| SCALEARNUNIFORM++Learning Strategy=Two-stage MTL, Backbone=RoBERTa-large2023.10 | 88.85 | 80.7 | 82.13 | 70.19 | 62.98 | 83.6 | 91.07 | 84.84 | 80.54 | |
| ADAPTERLearning Strategy=STL, Backbone=RoBERTa-large2023.10 | 88.52 | 80.73 | 82.36 | 69.16 | 63.25 | 71.9 | 92.68 | 81.23 | 78.73 | |
| COMPACTER++Learning Strategy=STL, Backbone=RoBERTa-large2023.10 | 88.34 | 79.18 | 79.53 | 69.26 | 62.26 | 79 | 87.5 | 79.71 | 78.1 | |
| LORALearning Strategy=STL, Backbone=RoBERTa-large2023.10 | 88.3 | 79.1 | 78.02 | 68.46 | 62.12 | 76.6 | 92.86 | 80.51 | 78.58 | |
| SCALEARN++Learning Strategy=Two-stage MTL, Backbone=RoBERTa-large2023.10 | 88.28 | 80.76 | 83.08 | 69.59 | 62.98 | 87.8 | 91.07 | 85.7 | 81.16 | |
| PROPETLLearning Strategy=STL, Backbone=RoBERTa-large2023.10 | 87.86 | 81.19 | 81.61 | 69.62 | 63.46 | 69 | 94.11 | 84.55 | 78.92 | |
| SCALEARNLearning Strategy=Two-stage MTL, Backbone=RoBERTa-large2023.10 | 87.85 | 78.4 | 80.29 | 68.56 | 62.98 | 85.4 | 92.86 | 84.91 | 80.16 | |
| (IA)3Learning Strategy=STL, Backbone=RoBERTa-large2023.10 | 87.47 | 77.91 | 80.97 | 68.65 | 60.58 | 77 | 90 | 80.79 | 77.93 | |
| ADAPTER-MLearning Strategy=Joint MTL, Backbone=RoBERTa-large2023.10 | 86.76 | 75.15 | 77.18 | 51.57 | 53.21 | 67.67 | 80.95 | 77.38 | 71.23 | |
| PROPETL-MLearning Strategy=Joint MTL, Backbone=RoBERTa-large2023.10 | 84.83 | 79.6 | 82.02 | 55.33 | 59.62 | 86.67 | 88.1 | 85.56 | 77.71 | |
| HYPERFORMERLearning Strategy=Joint MTL, Backbone=RoBERTa-large2023.10 | 84.38 | 79.68 | 81.87 | 53.81 | 63.46 | 82.33 | 83.93 | 86.88 | 77.04 | |
| HYPERFORMER++Learning Strategy=Joint MTL, Backbone=RoBERTa-large2023.10 | 83.66 | 80.21 | 71.5 | 49.14 | 62.98 | 54 | 67.86 | 66.97 | 67.29 | |
| FINETUNE-MLearning Strategy=Joint MTL, Backbone=RoBERTa-large2023.10 | 83.57 | 78.08 | 81.7 | 53.03 | 49.36 | 86.67 | 82.14 | 74.8 | 73.81 | |
| FINETUNELearning Strategy=STL, Backbone=RoBERTa-large2023.10 | 81.6 | 79.03 | 81.65 | 69.72 | 63.46 | 52 | 90.36 | 81.52 | 74.92 | |
| ADAPTERSOUPLearning Strategy=Two-stage MTL, Backbone=RoBERTa-large2023.10 | 70.33 | 38.42 | 73.2 | 62.23 | 63.46 | 54.5 | 68.75 | 61.37 | 61.53 |