List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on PIQA

80.5AccuracyOriginal
50.44458.24766.0573.853Nov 8, 2025Dec 16, 2025Jan 23, 2026Mar 3, 2026Apr 10, 2026May 18, 2026Jun 26, 2026
Updated 4mo ago

Evaluation results

MethodLinks
OriginalModel=Qwen3-30B-A3B, Reduction ratio=0%2026.0580.5
M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%2026.0580.4
FrequencyModel=Qwen3-30B-A3B, Reduction ratio=25%2026.0580.4
ConMoEModel=Qwen3-30B-A3B, Reduction ratio=25%2026.0580.4
REAPModel=deepseek-moe-16b-base, Reduction ratio=25%2026.0579.8
REAPModel=Qwen3-30B-A3B, Reduction ratio=25%2026.0579.7
OriginalModel=deepseek-moe-16b-base, Reduction ratio=0%2026.0579.7
ConMoEModel=deepseek-moe-16b-base, Reduction ratio=25%2026.0579.6
OriginalModel=OLMoE-1B-7B-0125, Reduction ratio=0%2026.0579.6
FrequencyModel=Qwen3-30B-A3B, Reduction ratio=50%2026.0579.4
FrequencyModel=deepseek-moe-16b-base, Reduction ratio=25%2026.0579.1
REAPModel=Qwen3-30B-A3B, Reduction ratio=50%2026.0579
HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%2026.0578.5
M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%2026.0578.2
AR (autoregressive)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.0578.1
ConMoEModel=deepseek-moe-16b-base, Reduction ratio=50%2026.0578.1
ConMoEModel=Qwen3-30B-A3B, Reduction ratio=50%2026.0578
REAPModel=deepseek-moe-16b-base, Reduction ratio=50%2026.0578
REAPModel=OLMoE-1B-7B-0125, Reduction ratio=25%2026.0577.9
ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%2026.0577.8
FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=25%2026.0577.5
FrequencyModel=deepseek-moe-16b-base, Reduction ratio=50%2026.0577.3
M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%2026.0577.2
AR (autoregressive, ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.0576.9
HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%2026.0576.2
HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%2026.0575.8
M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%2026.0575.1
JTP (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.1174.92
HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%2026.0574.6
LLaDa-8B-BaseZero-shot=true, Likelihood-based evaluation=true, Number of Parameters=8B2026.0574.4
MTP (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.1174.32
NextLat (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.1173.61
GPTParameters=1.3B, Training tokens=100B2025.1173.45
JTP (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.1173.34
NoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1173.3
Selective RoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1173.1
NextLat (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.1173.07
MTP (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.1172.8
RoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1172.4
REAPModel=OLMoE-1B-7B-0125, Reduction ratio=50%2026.0571
M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%2026.0570.4
FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=50%2026.0570.4
RoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1169.6
RoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B2025.1169.5
Selective RoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1169.1
Selective RoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B, Variant=stable_selective_rope2025.1169.1
Selective RoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1168.7
NoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B2025.1168.6
RoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1168.4
ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%2026.0568.3
NoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1167.8
R2LMShots=0-sh, Candidate Type=long-target2026.0667.2
HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%2026.0567.1
NoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1167
R2LM-PI (plug-in)Shots=0-sh, Candidate Type=long-target2026.0664.6
Causal dLLMShots=0-sh, Candidate Type=long-target2026.0663.8
HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%2026.0563
Duo (uniform diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.0562.7
MDLM (masked diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.0562.2
M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%2026.0562
Bidirectional dLLMShots=0-sh, Candidate Type=long-target2026.0661.5
SMDM-1BZero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1B2026.0560.3
MDLMevaluation protocol=likelihood-based answer scoring2026.0558.27
CFM (ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.0557.2
Eso-LM (interpolating)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.0555.6
CoFReevaluation protocol=likelihood-based answer scoring2026.0555.33
ChanceZero-shot=true, Likelihood-based evaluation=true2026.0551.6