Question Answering on ARC Challenge (ARC-c and Avg Metrics)
47.61ARC-c ScoreOursQ
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OursQType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 47.61 | 67.61 | |
| BaselineType=None, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 47.53 | 65.62 | |
| BaselineType=None, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 46.93 | 68.6 | |
| WandaType=P25%, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 46.84 | 66.62 | |
| MoNEType=P25%, Storage=24.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 46.67 | 65.65 | |
| WandaType=P25% Q4b, Storage=7.70GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 46.59 | 69.2 | |
| EAC-MoEType=P11% Q3.03b, Storage=7.19GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 46.16 | 65.3 | |
| EAC-MoEType=P38% Q3.03b, Storage=4.47GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 45.05 | 62.7 | |
| MoNEType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 44.86 | 63.97 | |
| He et al.Type=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 44.86 | 63.97 | |
| WandaType=P25%, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 44.54 | 61.56 | |
| WandaType=P50%, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 44.37 | 60.82 | |
| He et al.Type=P25% Q4b, Storage=7.70GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 44 | 59.7 | |
| PuzzleMoEType=M25%, Storage=25.04GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 44 | 62.36 | |
| OursQType=P25% Q4b, Storage=6.26GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 43.77 | 63.78 | |
| PuzzleMoEType=M50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 43 | 62.08 | |
| Attribution-guided and Coverage-Maximized Expert-wise PruningType=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 42.58 | 62.34 | |
| MoE-I2Type=L53.98%, Storage=15.39GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 42.58 | 59.62 | |
| Attribution-guided and Coverage-Maximized Expert-wise PruningType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 42.49 | 64.92 | |
| C-PruneType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 41.72 | 61.81 | |
| MoNEType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 37.2 | 59.8 | |
| He et al.Type=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 33.15 | 53.88 | |
| WandaType=P50%, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 31.4 | 49.29 | |
| He et al.Type=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 28.67 | 44.17 | |
| C-PruneType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 24.4 | 42.39 | |
| MoNEType=P25%, Storage=25.04GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 22.7 | 33.94 | |
| MoNEType=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 22.53 | 33.24 |