Reading Comprehension on BoolQ (Accuracy, Avg.)
85.66Accuracy (%)FP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FPModel=Dream-Instruct, Quantization=Full Precision2026.06 | 85.66 | 71.01 | |
| FPModel=Dream-Base, Quantization=Full Precision2026.06 | 84.46 | 70.01 | |
| FAIR-CalibModel=Dream-Instruct, Quantization=W4A42026.06 | 81.77 | 66.66 | |
| FAIR-CalibModel=Dream-Base, Quantization=W4A42026.06 | 80.31 | 64.64 | |
| BaselineType=None, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 79.82 | 68.6 | |
| FlatQuantModel=Dream-Instruct, Quantization=W4A42026.06 | 79.66 | 63.98 | |
| WandaType=P25%, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 79.39 | 66.62 | |
| FlatQuantModel=Dream-Base, Quantization=W4A42026.06 | 79.14 | 62.08 | |
| WandaType=P25% Q4b, Storage=7.70GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 78.99 | 69.2 | |
| MoNEType=P25%, Storage=24.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 78.47 | 65.65 | |
| QuaRotModel=Dream-Instruct, Quantization=W4A42026.06 | 76.91 | 61.14 | |
| MoE-I2Type=L53.98%, Storage=15.39GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 76.79 | 59.62 | |
| OursQType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 75.29 | 67.61 | |
| WandaType=P50%, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 75.11 | 60.82 | |
| EAC-MoEType=P11% Q3.03b, Storage=7.19GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 75.02 | 65.3 | |
| QuaRotModel=Dream-Base, Quantization=W4A42026.06 | 74.77 | 57.49 | |
| PuzzleMoEType=M50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 74.5 | 62.08 | |
| WandaType=P25%, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 74.16 | 61.56 | |
| OursQType=P25% Q4b, Storage=6.26GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 73.55 | 63.78 | |
| MoNEType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 73.39 | 59.8 | |
| PuzzleMoEType=M25%, Storage=25.04GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 73.1 | 62.36 | |
| BaselineType=None, Storage=32.7GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 72.57 | 65.62 | |
| EAC-MoEType=P38% Q3.03b, Storage=4.47GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 72.51 | 62.7 | |
| Attribution-guided and Coverage-Maximized Expert-wise PruningType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 71.77 | 64.92 | |
| C-PruneType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 70.98 | 61.81 | |
| Attribution-guided and Coverage-Maximized Expert-wise PruningType=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 68.78 | 62.34 | |
| MoNEType=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 68.65 | 63.97 | |
| He et al.Type=P25% Q4b, Storage=6.00GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 68.65 | 63.97 | |
| He et al.Type=P25% Q4b, Storage=7.70GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 66 | 59.7 | |
| WandaType=P50%, Storage=31.41GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 63.18 | 49.29 | |
| RTNModel=Dream-Instruct, Quantization=W4A42026.06 | 59.94 | 45.76 | |
| RTNModel=Dream-Base, Quantization=W4A42026.06 | 59.63 | 42 | |
| C-PruneType=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 55.87 | 42.39 | |
| He et al.Type=P50%, Storage=16.57GB, Model Family=DeepSeek-V2-Lite, Evaluation Protocol=Zero-shot2026.06 | 50.45 | 53.88 | |
| He et al.Type=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 38.65 | 44.17 | |
| MoNEType=P25%, Storage=25.04GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 37.83 | 33.94 | |
| MoNEType=P50%, Storage=17.34GB, Model Family=Deepseek-MoE-16B, Evaluation Protocol=Zero-shot2026.06 | 37.83 | 33.24 |