Multitask Language Understanding on C-MMLU (Accuracy)
51.2Accuracy (C-MMLU)Meta-Llama-3-8B
Evaluation results
| Method | Links | |
|---|---|---|
| Meta-Llama-3-8BType=Candidate Model2026.02 | 51.2 | |
| RouterDCType=Base Router2026.02 | 51 | |
| RouterDC + RACER-G (w/ Agg.)Base Router=RouterDC, Score Definition=router score-gap, Aggregation=w/ Agg.2026.02 | 50.7 | |
| RouterDC + RACER-P (w/ Agg.)Base Router=RouterDC, Score Definition=inverse probability, Aggregation=w/ Agg.2026.02 | 50.6 | |
| Chinese-Mistral-7B-v0.1Type=Candidate Model2026.02 | 49.1 | |
| KNNR + RACER-P (w/ Agg.)Base Router=KNNR, Score Definition=inverse probability, Aggregation=w/ Agg.2026.02 | 49 | |
| KNNR + RACER-G (w/ Agg.)Base Router=KNNR, Score Definition=router score-gap, Aggregation=w/ Agg.2026.02 | 48.8 | |
| MLPR + RACER-P (w/ Agg.)Base Router=MLPR, Score Definition=inverse probability, Aggregation=w/ Agg.2026.02 | 48.4 | |
| MLPR + RACER-G (w/ Agg.)Base Router=MLPR, Score Definition=router score-gap, Aggregation=w/ Agg.2026.02 | 48.3 | |
| KNNRType=Base Router2026.02 | 48.1 | |
| Mistral-7B-v0.1Type=Candidate Model2026.02 | 44.6 | |
| MetaMath-Mistral-7BType=Candidate Model2026.02 | 44.4 | |
| MLPRType=Base Router2026.02 | 44.4 | |
| Dolphin-2.6-Mistral-7bType=Candidate Model2026.02 | 44.2 | |
| Dolphin-2.9-Llama3-8bType=Candidate Model2026.02 | 43.9 | |
| Zephyr-7b-betaType=Candidate Model2026.02 | 43.5 |