Question Answering on MMLU, TruthfulQA, and BBQ
69.3MMLU AccuracyZero-shot
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Zero-shot#Params=x12026.05 | 69.3 | 51.27 | 80.69 | 67.09 | |
| DiDi-Merging-L#Params=x2.02026.05 | 68.97 | 58.24 | 93.32 | 73.51 | |
| DiDi-Merging-M#Params=x1.42026.05 | 68.85 | 57.32 | 92.44 | 72.87 | |
| FREE-Merging#Params=x2.042026.05 | 68.83 | 57.39 | 92.14 | 72.78 | |
| IndividualBackbone=Qwen-14B2025.12 | 68.36 | 54.35 | 93.53 | 72.08 | |
| Fine-tuned#Params=x32026.05 | 68.35 | 54.34 | 93.53 | 72.07 | |
| DTS-DBackbone=Qwen-14B, AMR=3.57, r=0.32025.12 | 68.32 | 54.11 | 92.99 | 71.81 | |
| Twin-Merging#Params=x2.132026.05 | 68.32 | 55.76 | 90.98 | 71.68 | |
| DTS-TBackbone=Qwen-14B, AMR=3.57, r=0.32025.12 | 68.3 | 54.12 | 92.97 | 71.8 | |
| DTS-T*Backbone=Qwen-14B, AMR=0.92, r=adaptive2025.12 | 68.2 | 53.99 | 92.9 | 71.7 | |
| DTS-D*Backbone=Qwen-14B, AMR=0.92, r=adaptive2025.12 | 68.18 | 53.99 | 92.91 | 71.7 | |
| FREE-MergingBackbone=Qwen-14B, AMR=10.002025.12 | 68.13 | 53.91 | 92.54 | 71.52 | |
| Twin-MergingBackbone=Qwen-14B, AMR=100.02025.12 | 68.07 | 52.38 | 90.73 | 70.39 | |
| T-SwitchBackbone=Qwen-14B, AMR=6.252025.12 | 68.05 | 53.72 | 92.5 | 71.42 | |
| SVDBackbone=Qwen-14B, AMR=5.392025.12 | 67.99 | 52.45 | 91.71 | 70.72 | |
| EMR-MergingBackbone=Qwen-14B, AMR=15.622025.12 | 67.94 | 52.5 | 91.02 | 70.49 | |
| EMR-Merging#Params=x42026.05 | 67.82 | 55.01 | 90.13 | 70.98 | |
| Ties-MergingBackbone=Qwen-14B, AMR=0.002025.12 | 67.28 | 50.02 | 84.1 | 67.13 | |
| DAREBackbone=Qwen-14B, AMR=0.002025.12 | 67.23 | 51.31 | 83.74 | 67.43 | |
| Weight-AveragingBackbone=Qwen-14B, AMR=0.002025.12 | 67.11 | 50.02 | 82.32 | 66.48 | |
| Task-ArithmeticBackbone=Qwen-14B, AMR=0.002025.12 | 66.63 | 53.38 | 78.24 | 66.08 |