List your company
ResearchFormerly WizWand

Science Question Answering on ARC Easy (Accuracy)

86.4AccuracyFull
26.18441.81757.4573.083Apr 17, 2025Jun 24, 2025Sep 1, 2025Nov 9, 2025Jan 16, 2026Mar 26, 2026Jun 3, 2026
Updated 4mo ago

Evaluation results

MethodLinks
FullModel=Qwen1.5MoE-A2.7B, E↓=100%, A↓=100%2026.0686.4
TENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=61.38%2026.0685.1
ENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=60%2026.0684.6
Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.0684.5
FullModel=DeepSeek-V2-Lite, E↓=100%, A↓=100%2026.0684.1
GEMMA-7BState=Dense2025.0681.65
Teacher (GPT-OSS-20B)Evaluation Mode=completion mode2026.0580.9
EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.0680.1
FrequencyModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.0680
TENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=63.76%2026.0679.1
ENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=30%2026.0678.2
TENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=34.51%2026.0678.2
RandomModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.0677.8
ENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=60%2026.0677.8
EASY-EPModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.0676.7
LLAMA-2-7BState=Dense2025.0676.35
FrequencyModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.0675.3
DEEPSEEK-7BState=Dense2025.0675.25
VICUNA-1.3-7BState=Dense2025.0674.28
Gating ScoreModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.0674
CLIMBSize=950M2025.0473.57
TENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=36.38%2026.0673.3
SmolLMSize=360M2025.0470.08
MASKLLMBackbone=LLAMA-2-7B2025.0669.57
MASKLLMBackbone=DEEPSEEK-7B2025.0668.1
CLIMBSize=350M2025.0467.38
MaskProBackbone=DEEPSEEK-7B2025.0666.74
SPARSEGPTBackbone=DEEPSEEK-7B2025.0666.37
MaskProBackbone=LLAMA-2-7B2025.0666.12
AMD-OLMoSize=1.2B2025.0465.95
GBLMBackbone=DEEPSEEK-7B2025.0665.82
MASKLLMBackbone=VICUNA-1.3-7B2025.0665.57
WANDABackbone=DEEPSEEK-7B2025.0665.49
Llama-3.2Size=1.2B2025.0465.49
PRUNER-ZEROBackbone=DEEPSEEK-7B2025.0664.94
Qwen2.5Size=490M2025.0464.6
SPARSEGPTBackbone=LLAMA-2-7B2025.0664.52
MaskProBackbone=VICUNA-1.3-7B2025.0664.23
GBLMBackbone=LLAMA-2-7B2025.0663.17
SPARSEGPTBackbone=VICUNA-1.3-7B2025.0662.92
GBLMBackbone=VICUNA-1.3-7B2025.0662.84
WANDABackbone=LLAMA-2-7B2025.0662.79
PRUNER-ZEROBackbone=VICUNA-1.3-7B2025.0662.33
MAGNITUDEBackbone=LLAMA-2-7B2025.0661.87
WANDABackbone=VICUNA-1.3-7B2025.0661.7
PRUNER-ZEROBackbone=LLAMA-2-7B2025.0661.32
X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.0461.1
X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.0460.9
X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.0460.5
TinyLlamaSize=1.1B2025.0460.31
ENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=30%2026.0659.6
X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.0459.5
X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.0459.1
X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.0458.4
X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.0458.1
X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.0457.7
X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+1.17B, Evaluation Protocol=Zero/Few-shot2026.0456.7
X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+1.87B, Evaluation Protocol=Zero/Few-shot2026.0456.3
Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.0656.3
MoRTBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.0455.6
EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.0655.3
MoRTBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.0455.1
MAGNITUDEBackbone=DEEPSEEK-7B2025.0654.92
MAGNITUDEBackbone=VICUNA-1.3-7B2025.0654.59
X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.0454
EngramBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.0453.5
EngramBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.0453.5
BaselineBackbone Scale=1.15B Scale, Cfg=N/A, # Params=1.15B, Evaluation Protocol=Zero/Few-shot2026.0453.2
X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.0453
FrequencyModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.0652.6
RetokenBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.0452.3
EngramBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.0451.6
RetokenBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.0451.2
RetokenBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.0450.7
MoRTBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.0449.6
MoRTBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.0449.3
EngramBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.0449.1
RetokenBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.0448.8
EngramBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.0448.6
MoRTBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.0447.9
MoRTBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.0447.9
RetokenBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.0447.9
RetokenBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.0446.1
EngramBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.0443.5
BaselineBackbone Scale=0.73B Scale, Cfg=N/A, # Params=0.73B, Evaluation Protocol=Zero/Few-shot2026.0441.4
EASY-EPModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.0638.4
DO-ACPScoring=DO-ACP, K=42026.0536.7
CPScoring=CP, K=42026.0536.4
ACPScoring=ACP, K=42026.0535.6
RandomModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.0634.3
SFScoring=SF, K=42026.0534
ACPScoring=ACP, K=82026.0533.6
DO-ACPScoring=DO-ACP, K=82026.0533.5
SFScoring=SF, K=82026.0533.3
MaskProBackbone=GEMMA-7B2025.0632.92
WANDABackbone=GEMMA-7B2025.0632.66
GBLMBackbone=GEMMA-7B2025.0632.38
CPScoring=CP, K=82026.0532.2
SPARSEGPTBackbone=GEMMA-7B2025.0630.64
Random FFN + teacher attn2026.0528.5