Multi-task Language Understanding on MMLU (MMLU Performance (MM))
65MMLU Performance (MM)Baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineBackbone=Llama-3.1-8B-Instruct2026.06 | 65 | |
| NoiseBackbone=Llama-3.1-8B-Instruct2026.06 | 64.6 | |
| MeanBackbone=Llama-3.1-8B-Instruct2026.06 | 63.9 | |
| EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 63.6 | |
| RMUBackbone=Llama-3.1-8B-Instruct2026.06 | 62.2 | |
| RMU + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 62 | |
| SNMF + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 60.4 | |
| CRISP + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 60.4 | |
| CRISPBackbone=Llama-3.1-8B-Instruct2026.06 | 59.3 | |
| SNMFBackbone=Llama-3.1-8B-Instruct2026.06 | 58.7 | |
| NoiseBackbone=Gemma-2-2B-it2026.06 | 54.8 | |
| MeanBackbone=Gemma-2-2B-it2026.06 | 54.2 | |
| BaselineBackbone=Gemma-2-2B-it2026.06 | 54.2 | |
| EMBERBackbone=Gemma-2-2B-it2026.06 | 53.7 | |
| CRISP + EMBERBackbone=Gemma-2-2B-it2026.06 | 53.2 | |
| CRISPBackbone=Gemma-2-2B-it2026.06 | 52.6 | |
| RMU + EMBERBackbone=Gemma-2-2B-it2026.06 | 51.8 | |
| SNMF + EMBERBackbone=Gemma-2-2B-it2026.06 | 51 | |
| RMUBackbone=Gemma-2-2B-it2026.06 | 50.3 | |
| SNMFBackbone=Gemma-2-2B-it2026.06 | 48.4 |