Multitask Language Understanding on MMLU (MMLU Score, Average)
53.94MMLU ScoreWanda
Evaluation results
| Method | Links | ||
|---|---|---|---|
| WandaType=P25% Q4b, Storage=7.70GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 53.94 | 69.2 | |
| WandaType=P25%, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 53.84 | 66.62 | |
| MoNEType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 49.03 | 63.97 | |
| He et al.Type=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 49.03 | 63.97 | |
| OursQType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 48.91 | 67.61 | |
| C-PruneType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 47.97 | 61.81 | |
| BaselineType=None, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 45.58 | 68.6 | |
| Attribution-guided and Coverage-Maximized Expert-wise PruningType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 44.16 | 64.92 | |
| WandaType=P50%, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 40.81 | 49.29 | |
| BaselineType=None, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 38.18 | 65.62 | |
| EAC-MoEType=P11% Q3.03b, Storage=7.19GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 37.45 | 65.3 | |
| PuzzleMoEType=M25%, Storage=25.04GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 37.2 | 62.36 | |
| PuzzleMoEType=M50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 36.9 | 62.08 | |
| He et al.Type=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 36.05 | 53.88 | |
| OursQType=P25% Q4b, Storage=6.26GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 35.7 | 63.78 | |
| WandaType=P25%, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 33.63 | 61.56 | |
| EAC-MoEType=P38% Q3.03b, Storage=4.47GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 33.33 | 62.7 | |
| WandaType=P50%, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 32.99 | 60.82 | |
| Attribution-guided and Coverage-Maximized Expert-wise PruningType=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 31.66 | 62.34 | |
| C-PruneType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=5-shot2026.06 | 29.52 | 42.39 | |
| He et al.Type=P25% Q4b, Storage=7.70GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 27.9 | 59.7 | |
| He et al.Type=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 23.47 | 44.17 | |
| MoNEType=P25%, Storage=25.04GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 23.12 | 33.94 | |
| MoNEType=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=5-shot2026.06 | 23.12 | 33.24 |