Multi-task Language Understanding on MergeBench Llama-3.2-3B (test)
44.1Average ScoreSWUDI
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SWUDIBackbone=Llama-3.2-3B2026.06 | 44.1 | 58.45 | 37.8 | 57.67 | 24.58 | 45.02 | 47.01 | 37.55 | 44.69 | |
| TIES w/ DAREBackbone=Llama-3.2-3B2026.06 | 43.56 | 52.99 | 35.37 | 57.41 | 25.51 | 47.42 | 47.55 | 37.04 | 45.19 | |
| SWUDI-ABackbone=Llama-3.2-3B2026.06 | 43.46 | 58.15 | 36.59 | 56.35 | 22.18 | 45.42 | 46.85 | 37.3 | 44.83 | |
| OptMergeBackbone=Llama-3.2-3B2026.06 | 43.37 | 53.53 | 34.76 | 58.99 | 25.51 | 45.23 | 46.99 | 37.3 | 44.66 | |
| TSV MergingBackbone=Llama-3.2-3B2026.06 | 43.24 | 55.72 | 36.59 | 56.88 | 20.15 | 46.28 | 48.17 | 37.13 | 45.01 | |
| WUDI MergingBackbone=Llama-3.2-3B2026.06 | 42.2 | 52.54 | 37.2 | 57.14 | 17.93 | 46.56 | 44.86 | 37.21 | 44.17 | |
| TA w/ DAREBackbone=Llama-3.2-3B2026.06 | 42.01 | 46.7 | 33.54 | 59.26 | 18.67 | 47.8 | 48.12 | 36.44 | 45.56 | |
| Task ArithmeticBackbone=Llama-3.2-3B2026.06 | 40.92 | 44.73 | 33.54 | 59.79 | 14.42 | 47.38 | 46.34 | 36.27 | 44.88 | |
| Iso-CBackbone=Llama-3.2-3B2026.06 | 40.12 | 48.22 | 35.37 | 55.56 | 8.13 | 44.83 | 47.49 | 37.04 | 44.29 | |
| Weight AverageBackbone=Llama-3.2-3B2026.06 | 39.49 | 42.76 | 31.71 | 59.52 | 9.24 | 46.02 | 46.35 | 35.76 | 44.53 | |
| TIES MergingBackbone=Llama-3.2-3B2026.06 | 39.11 | 42.61 | 30.49 | 57.14 | 7.58 | 44.91 | 48.25 | 37.13 | 44.75 |