Language Model Merging Evaluation on Qwen3-8B (test)
73.2MMLUSingle Best
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Single BestModel=Qwen3-8B, Experts=Ten Tulu-finetuned experts2026.05 | 73.2 | 43.2 | 80.2 | 64.4 | 61.4 | 54.8 | 63 | 40 | 60 | |
| Task ArithmeticModel=Qwen3-8B, Experts=Ten Tulu-finetuned experts2026.05 | 72.1 | 43.2 | 80.9 | 58 | 35.5 | 41.3 | 56.6 | 23.2 | 51.3 | |
| EvoGMModel=Qwen3-8B, Experts=Ten Tulu-finetuned experts2026.05 | 71.8 | 42.9 | 82.4 | 58.6 | 65.5 | 60.7 | 61.4 | 39.4 | 60.3 | |
| Model SwarmModel=Qwen3-8B, Experts=Ten Tulu-finetuned experts2026.05 | 71.2 | 42 | 78.9 | 66.9 | 58.8 | 55.1 | 63.7 | 38.1 | 59.3 | |
| MTLModel=Qwen3-8B, Experts=Ten Tulu-finetuned experts2026.05 | 70.2 | 39.9 | 71.2 | 66.7 | 42.8 | 36.3 | 55.8 | 29.6 | 51.6 | |
| TIES MergingModel=Qwen3-8B, Experts=Ten Tulu-finetuned experts2026.05 | 67.3 | 36.9 | 80.5 | 55.8 | 21.4 | 45.7 | 57.1 | 34.9 | 50 | |
| BaseModel=Qwen3-8B, Experts=Ten Tulu-finetuned experts2026.05 | 65.5 | 34.9 | 79.2 | 58.9 | 12.5 | 53.4 | 56.1 | 30.8 | 48.9 |