Multiple Choice Question Answering on ARC Challenge (Normalized Accuracy)
56.4Normalized AccuracyOriginal
Evaluation Results
| Method | Links | |
|---|---|---|
| OriginalModel=Qwen3-30B-A3B, Reduction ratio=0%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 56.4 | |
| FrequencyModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 55.9 | |
| M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 55.2 | |
| REAPModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 55 | |
| REAPModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 54.9 | |
| ConMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 54.6 | |
| FrequencyModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 52 | |
| ConMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 51.4 | |
| M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 50.7 | |
| OriginalModel=OLMoE-1B-7B-0125, Reduction ratio=0%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 49.2 | |
| ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 48.8 | |
| REAPModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 47 | |
| HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 46.5 | |
| M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 46.5 | |
| OriginalModel=deepseek-moe-16b-base, Reduction ratio=0%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 45.9 | |
| FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 45.2 | |
| HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 45.1 | |
| REAPModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 44.7 | |
| ConMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 44.7 | |
| FrequencyModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 43.2 | |
| HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 41.8 | |
| M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 41.1 | |
| REAPModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 40.9 | |
| ConMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 39.8 | |
| FrequencyModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 39.2 | |
| HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 37.1 | |
| M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 36.5 | |
| HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 36.4 | |
| FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 34.3 | |
| REAPModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 34.3 | |
| M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 33.5 | |
| ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 32.7 | |
| HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.05 | 30.7 |