List your company
ResearchFormerly WizWand

Multiple-choice Question Answering on BoolQ

0.887MC AccuracyOriginal
0.406520.531260.6560.78074Feb 9, 2026Mar 3, 2026Mar 26, 2026Apr 18, 2026May 11, 2026Jun 3, 2026Jun 26, 2026
Updated 4mo ago

Evaluation results

MethodLinks
OriginalModel=Qwen3-30B-A3B, Reduction ratio=0%2026.050.887
FrequencyModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning2026.050.885
ConMoEModel=Qwen3-30B-A3B, Reduction ratio=25%2026.050.885
M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging2026.050.884
REAPModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning2026.050.881
FrequencyModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning2026.050.874
REAPModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning2026.050.861
HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging2026.050.859
ConMoEModel=Qwen3-30B-A3B, Reduction ratio=50%2026.050.851
Spherical SteeringZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.020.8294
CAAZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.020.8242
M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging2026.050.818
ITIZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.020.8153
SADI-HEADZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.020.802
BaselineZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.020.8012
HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging2026.050.755
ConMoEModel=deepseek-moe-16b-base, Reduction ratio=25%2026.050.745
OriginalModel=deepseek-moe-16b-base, Reduction ratio=0%2026.050.74
M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging2026.050.738
HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging2026.050.734
AR (autoregressive)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.050.719
OriginalModel=OLMoE-1B-7B-0125, Reduction ratio=0%2026.050.704
FrequencyModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning2026.050.699
ConMoEModel=deepseek-moe-16b-base, Reduction ratio=50%2026.050.697
REAPModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning2026.050.692
Bidirectional dLLMShots=0-sh, Candidate Type=short-target2026.060.684
HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging2026.050.678
R2LM-PI (plug-in)Shots=0-sh, Candidate Type=short-target2026.060.673
R2LMShots=0-sh, Candidate Type=short-target2026.060.67
HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging2026.050.659
Causal dLLMShots=0-sh, Candidate Type=short-target2026.060.653
FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning2026.050.643
AR (autoregressive, ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.050.636
FrequencyModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning2026.050.635
REAPModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning2026.050.631
M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging2026.050.631
MDLM (masked diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.050.628
M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging2026.050.619
ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%2026.050.618
SMDM-1BZero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1B2026.050.615
Duo (uniform diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.050.596
REAPModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning2026.050.589
ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%2026.050.579
HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging2026.050.578
FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning2026.050.573
Eso-LM (interpolating)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.050.534
ChanceZero-shot=true, Likelihood-based evaluation=true2026.050.504
REAPModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning2026.050.499
M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging2026.050.429
CFM (ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.050.425