Language Understanding on MMLU-Pro (test) by Model Size
23.6MMLU-Pro (test) AccuracyModel Swarm
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Model SwarmBackbone=Qwen2.5-1.5B2026.05 | 23.6 | — | — | — | — | — | — | |
| TABackbone=Qwen2.5-1.5B2026.05 | 23.4 | — | — | — | — | — | — | |
| DAREBackbone=Qwen2.5-1.5B2026.05 | 23.3 | — | — | — | — | — | — | |
| Single BestBackbone=Qwen2.5-1.5B2026.05 | 23.2 | — | — | — | — | — | — | |
| PSO-MergingBackbone=Qwen2.5-1.5B2026.05 | 23.2 | — | — | — | — | — | — | |
| Model SoupBackbone=Qwen2.5-1.5B2026.05 | 22.5 | — | — | — | — | — | — | |
| EvoGMBackbone=Qwen2.5-1.5B2026.05 | 22.4 | — | — | — | — | — | — | |
| TIESBackbone=Qwen2.5-1.5B2026.05 | 22.1 | — | — | — | — | — | — | |
| CMABackbone=Qwen2.5-1.5B2026.05 | 21.7 | — | — | — | — | — | — | |
| BaseBackbone=Qwen2.5-1.5B2026.05 | 20.7 | — | — | — | — | — | — | |
| MTLBackbone=Qwen2.5-1.5B2026.05 | 20.7 | — | — | — | — | — | — | |
| AIMMerging2026.05 | — | 26.5 | 41.8 | 58.1 | 62.9 | 66.4 | 51.1 | |
| EWC2026.05 | — | 24.9 | 40 | 55.4 | 60.4 | 65.3 | 49.2 | |
| FOREVER2026.05 | — | 24.7 | 38.5 | 54.7 | 59.6 | 66.5 | 48.8 | |
| GCWM2026.05 | — | 27.1 | 43.5 | 59.4 | 63.7 | 67.8 | 52.3 | |
| L&S2026.05 | — | 26.2 | 41.1 | 57.3 | 62.4 | 65.6 | 50.5 | |
| MTL2026.05 | — | 27.5 | 44.4 | 61 | 65.3 | 68.6 | 53.4 | |
| OPCM2026.05 | — | 26.8 | 41.7 | 58.2 | 61.9 | 66.6 | 51 | |
| Seq. SFT2026.05 | — | 24.4 | 36.8 | 51.6 | 55.2 | 60.4 | 45.7 |