List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on ARC Challenge (Normalized Accuracy)

56.4Normalized AccuracyOriginal
29.67236.61143.5550.489May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
OriginalModel=Qwen3-30B-A3B, Reduction ratio=0%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0556.4
FrequencyModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0555.9
M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0555.2
REAPModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0555
REAPModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0554.9
ConMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0554.6
FrequencyModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0552
ConMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0551.4
M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0550.7
OriginalModel=OLMoE-1B-7B-0125, Reduction ratio=0%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0549.2
ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0548.8
REAPModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0547
HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0546.5
M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0546.5
OriginalModel=deepseek-moe-16b-base, Reduction ratio=0%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0545.9
FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0545.2
HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0545.1
REAPModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0544.7
ConMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0544.7
FrequencyModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0543.2
HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0541.8
M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0541.1
REAPModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0540.9
ConMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0539.8
FrequencyModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0539.2
HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0537.1
M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0536.5
HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0536.4
FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0534.3
REAPModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0534.3
M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0533.5
ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=None, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0532.7
HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging, Evaluation protocol=One-shot, no post-compression fine-tuning2026.0530.7