Multi-task Evaluation on GLUE 8 tasks
75CoLA ScoreIndividual
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| IndividualBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 75 | 83.4 | 87.5 | 91.5 | 85.4 | 85.9 | 93.6 | 88.7 | 86.4 | |
| AdaMergingBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 70.9 | 90.7 | 78.7 | 67.4 | 81.1 | 80.4 | 89.4 | 82 | 80.1 | |
| DARE + TIESBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 70.7 | 91.7 | 78.2 | 80.3 | 83.6 | 56.8 | 90.2 | 80.1 | 79 | |
| WEMoEBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 70.7 | 91.5 | 78.2 | 72.5 | 82.4 | 76.8 | 90 | 78.7 | 80.1 | |
| Task ArithmeticBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 70.5 | 92.3 | 78.4 | 77.8 | 83.6 | 57.8 | 90.2 | 80.5 | 78.9 | |
| TIES-MergingBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 70.3 | 91.7 | 78.9 | 78.3 | 83.5 | 65 | 90.2 | 81.6 | 79.9 | |
| DAREBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 69.2 | 91.7 | 79.7 | 74.1 | 83.9 | 63.1 | 89.8 | 81.2 | 79.1 | |
| Weight AverageBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 69.1 | 91.7 | 79.4 | 73.2 | 83.9 | 62.6 | 89.8 | 81.2 | 78.9 | |
| SyMergeBackbone=FLAN-T5 Base, Task Format=Text-generation, Batch Size=4, Iterations=1000, Evaluation Protocol=FusionBench2024.12 | 69.1 | 93.4 | 79.7 | 82.6 | 83.4 | 82.9 | 88.1 | 80.9 | 82.5 |