Natural Language Understanding on GLUE (Full 8-Task Suite)
60.2CoLA ScoreIndividual
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| IndividualBackbone=RoBERTa2024.12 | 60.2 | 94 | 89.2 | 90.6 | 91.4 | 87.2 | 92.7 | 79.1 | 85.6 | |
| SyMergeBackbone=RoBERTa2024.12 | 60 | 93.8 | 89.2 | 90.4 | 85.2 | 84 | 89.2 | 79.1 | 83.9 | |
| ProbSurgery w/TABackbone=RoBERTa, Base merging strategy=Task Arithmetic2024.12 | 54.7 | 93.6 | 89.7 | 89.4 | 85.2 | 77.6 | 85.3 | 76.9 | 81.6 | |
| Surgery w/TABackbone=RoBERTa, Base merging strategy=Task Arithmetic2024.12 | 46.8 | 93.6 | 89.7 | 88.7 | 85.1 | 78 | 85.3 | 76.5 | 80.5 | |
| EMR-MergingBackbone=RoBERTa2024.12 | 40 | 93.4 | 86.3 | 82.8 | 89.7 | 85.5 | 89.6 | 74.4 | 80.2 | |
| TSV-MBackbone=RoBERTa2024.12 | 28.7 | 88.4 | 83.1 | 76.8 | 85.8 | 59.9 | 75.2 | 47.7 | 68.2 | |
| ISO-MergingBackbone=RoBERTa2024.12 | 27.3 | 86.2 | 79.2 | 73.7 | 85.9 | 71.7 | 81.1 | 70.4 | 71.9 | |
| LiNeSBackbone=RoBERTa2024.12 | 26.1 | 86.4 | 78.9 | 72.9 | 83.3 | 56 | 75.9 | 59.9 | 67.4 | |
| Ties-MergingBackbone=RoBERTa2024.12 | 20.5 | 84.4 | 81.1 | 58.2 | 85.7 | 64.7 | 74.8 | 43 | 64 | |
| Task ArithmeticBackbone=RoBERTa2024.12 | 18.8 | 85.9 | 79.9 | 74 | 83.8 | 59.1 | 69.7 | 62.1 | 66.7 | |
| MagMaxBackbone=RoBERTa2024.12 | 17.3 | 76 | 70.8 | 71.3 | 85.8 | 70.4 | 59.5 | 45.1 | 62 | |
| Weight AveragingBackbone=RoBERTa2024.12 | 14 | 64.1 | 69.4 | 31.8 | 75.4 | 42.2 | 58.7 | 55.2 | 51.3 |