Natural Language Understanding on GLUE (test) (CoLA, MNLI, MRPC, QNLI, QQP, RTE, SST2, STSB)
69.7CoLA ScoreWeight Averaging
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Weight AveragingBackbone=Flan-T5-base, Fine-tuning strategy=LoRA2023.12 | 69.7 | 78.2 | 59.7 | 78.9 | 90.1 | 83.8 | 80.5 | 91.2 | 72 | |
| Pre-trainedBackbone=Flan-T5-base, Fine-tuning strategy=LoRA2023.12 | 69.1 | 75.7 | 56.5 | 76.2 | 88.4 | 82.1 | 80.1 | 91.2 | 62.2 | |
| IndividualBackbone=Flan-T5-base, Fine-tuning strategy=LoRA2023.12 | 69.1 | 84.6 | 82.7 | 85.5 | 90.9 | 84 | 84.4 | 92.9 | 87.4 | |
| Concrete TABackbone=Flan-T5-base, Fine-tuning strategy=LoRA, Type=Task Arithmetic (TA)-Based2023.12 | 69.1 | 78 | 58.1 | 78.4 | 89.9 | 83.5 | 79.4 | 91.6 | 73.4 | |
| LW AMBackbone=Flan-T5-base, Fine-tuning strategy=LoRA, Type=Layer-wise AdaMerging (LW AM)-Based2023.12 | 69.1 | 78.3 | 60.3 | 78.4 | 90 | 83.6 | 79.1 | 91.6 | 74.1 | |
| LW Concrete AMBackbone=Flan-T5-base, Fine-tuning strategy=LoRA, Type=Layer-wise AdaMerging (LW AM)-Based2023.12 | 69 | 78.5 | 59.4 | 80.1 | 89.9 | 82.9 | 79.1 | 91.7 | 75.4 | |
| Task ArithmeticBackbone=Flan-T5-base, Fine-tuning strategy=LoRA, Type=Task Arithmetic (TA)-Based2023.12 | 68.8 | 77.4 | 55.2 | 78.7 | 89.8 | 83.7 | 79.1 | 91.5 | 72.4 | |
| Ties-MergingBackbone=Flan-T5-base, Fine-tuning strategy=LoRA, Type=Task Arithmetic (TA)-Based2023.12 | 68.3 | 77.5 | 56.3 | 79.4 | 89.8 | 83.7 | 79.4 | 91.6 | 71.2 |