Hardened Language Understanding on MMLU-Pro (test)
23.4Accuracy (MMLU-Pro Test)Task Arithmetic
Evaluation results
| Method | Links | |
|---|---|---|
| Task ArithmeticBase model=Qwen2.5-1.5B2026.05 | 23.4 | |
| DAREBase model=Qwen2.5-1.5B2026.05 | 23.3 | |
| Single BestBase model=Qwen2.5-1.5B2026.05 | 23.2 | |
| Model SwarmBase model=Qwen2.5-1.5B2026.05 | 22.9 | |
| Model SoupBase model=Qwen2.5-1.5B2026.05 | 22.5 | |
| EvoGMBase model=Qwen2.5-1.5B2026.05 | 22.5 | |
| TIESBase model=Qwen2.5-1.5B2026.05 | 22.1 | |
| CMABase model=Qwen2.5-1.5B2026.05 | 21.7 | |
| BaseBase model=Qwen2.5-1.5B2026.05 | 20.7 | |
| MTLBase model=Qwen2.5-1.5B2026.05 | 20.7 | |
| PSO-MergingBase model=Qwen2.5-1.5B2026.05 | 19.7 |