List your company
ResearchFormerly WizWand

Science Question Answering on ARC-E

97.53AccuracyQwen3-4B
73.3579.627585.90592.1825Oct 15, 2021Jul 21, 2022Apr 27, 2023Feb 1, 2024Nov 6, 2024Aug 13, 2025May 20, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Qwen3-4BParams=4B2025.1297.53
Qwen2.5-3BParams=3B2025.1292.59
Qwen3-1.7BParams=1.7B2025.1291.89
Qwen2.5-1.5BParams=1.5B2025.1290.48
Qwen-14Brole=Teacher model, shot=5-shot2024.0789.24
SmolLM3-3BParams=3B2025.1288.54
Qwen-1.5 14B (Teacher)Model Scale=14B2024.0787.58
LLAMA-4-SCOUTPARAMS=109B, Variant=ORIGINAL2025.1187.3
LLAMA-4-SCOUTPARAMS=109B, Variant=FCSD2025.1187
In-Squeezereduction=128 ... -> 1, strategy=Min steps2026.0286
Cont-Squeezereduction=128 -> 1, training_steps=+700 steps2026.0285.89
FP16Model=LLAMA3-70B, Evaluation Protocol=Zero-shot, Bit-width=Full-precision2024.0385.86
Cont-Squeezereduction=128 -> 1, training_steps=+200 steps2026.0285.5
Direct Fine-tuningrank=1, training_steps=+700 steps2026.0285.39
In-Squeezereduction=128 ... -> 1, strategy=Standard2026.0285.35
Direct Fine-tuningrank=1, training_steps=+0 steps2026.0285.2
Direct Fine-tuningrank=1, training_steps=+200 steps2026.0285.07
QWEN-3-30B MOEPARAMS=30B, Variant=ORIGINAL2025.1184.8
Qwen3-0.6BParams=0.6B2025.1284.48
GenKnowSubBase Model=Phi-2, Setting=En, Evaluation Protocol=zero-shot2025.0584.38
GenKnowSubBase Model=Phi-2, Setting=Avg, Evaluation Protocol=zero-shot2025.0584.03
Full-precisionModel=Mixtral 8x22B, Avg. bits/exp.=16 (FP), Memory (GB)=281.2, Evaluation protocol=zero-shot2026.0484.01
llama-3.2-3BParams=3B2025.1283.95
BF16Model=Mixtral-8x7B, Precision=BF16, Evaluation Protocol=Zero-shot2026.0583.71
Qwen2-1.5BParams=1.5B2025.1283.6
Full-precisionModel=Mixtral 8x7B, Avg. bits/exp.=16 (FP), Memory (GB)=96.8, Evaluation protocol=zero-shot2026.0483.5
ArrowBase Model=Phi-2, Evaluation Protocol=zero-shot2025.0583.38
QWEN-3-30B MOEPARAMS=30B, Variant=FCSD2025.1183.3
PCMind-2.1-Kaiyuan-2BParams=2B2025.1282.89
SmolLM2-1.7BParams=1.7B2025.1282.72
gemma2-2BParams=2B2025.1282.54
YuLan-Mini-2.4BParams=2.4B2025.1282.54
Phi-2Base Model=Phi-2, Evaluation Protocol=zero-shot2025.0582.11
DenseBackbone=LLaMA-1, Model Parameters=65B, Sparsity=0%, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1181.4
SARQC-GBS(Saliency)Model=Mixtral-8x7B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.0580.6
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.0480.47
DenseBackbone=LLaMA-1, Model Parameters=30B, Sparsity=0%, Evaluation Protocol=zero-shot2025.1180.3
LLAMA-4-SCOUTPARAMS=109B, Variant=SFT2025.1180.3
DEEPSEEK-V2-LITEPARAMS=16B, Variant=ORIGINAL2025.1180.2
AutoPruneBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1180.06
UniformModel=Mixtral 8x7B, Avg. bits/exp.=3, Memory (GB)=19.3, Evaluation protocol=zero-shot2026.0480.05
BF16Model=Qwen3-MoE-30B, Precision=BF16, Evaluation Protocol=Zero-shot2026.0580.05
Mistral 7BModality=Pretrained2023.1080
AutoPruneBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1179.89
WandaBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1179.75
SparseGPTBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1179.35
WandaBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1179.29
Jet-Nemotron-4BSearch=N/R, Train=400B2026.0579.25
SARQC-GBS(Saliency)Model=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.0579.21
SARQC-GBS(Identity)Model=Mixtral-8x7B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.0579
FP16Model=LLaMA2-13B, Precision=FP16, Evaluation Protocol=Zero-shot2026.0578.96
SparseGPTBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.1178.96
GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=162021.1278.9
DEEPSEEK-V2-LITEPARAMS=16B, Variant=FCSD2025.1178.9
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.0478.87
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.0478.62
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.0478.41
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.0478.41
Coherence Boosting (GPT-3 175B)alpha=-0.442021.1078.32
SARQC-GBS(Identity)Model=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.0578.32
GPTQModel=Mixtral-8x7B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.0577.99
FP16Model=LLAMA3-8B, Evaluation Protocol=Zero-shot, Bit-width=Full-precision2024.0377.82
GradOTSetting=Plug-in, Backbone=LLaMA-13B2025.0777.8
Llama3 8B2026.0277.7
DenseBackbone=LLaMA-1, Model Parameters=13B, Sparsity=0%, Evaluation Protocol=zero-shot2025.1177.4
G-LionShots=3, Optimizer=Global Lion, Model=LLaMA 7B, Instruction Finetuning=true2024.0377.11
G-AdamWShots=3, Optimizer=Global AdamW, Model=LLaMA 7B, Instruction Finetuning=true2024.0377.06
DASH-3Budget=17.75, Search=12.3M / run, Train=700M2026.0576.98
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.0476.94
Distributed Lion-MaVoShots=3, Optimizer=D-Lion (MaVo), Model=LLaMA 7B, Instruction Finetuning=true2024.0376.86
DASH-3Budget=9.0, Search=12.3M / run, Train=700M2026.0576.85
OLMo-2-0425-1BParams=1B2025.1276.72
UniformModel=Mixtral 8x22B, Avg. bits/exp.=3, Memory (GB)=57.5, Evaluation protocol=zero-shot2026.0476.68
DASH-3Budget=4.875, Search=12.3M / run, Train=700M2026.0576.68
LLaMA 7B (Baseline)Shots=0, Instruction Finetuning=false, Model=LLaMA 7B2024.0376.64
+ DDKrole=Student model, shot=5-shot2024.0776.64
GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.1276.6
OT†Setting=Plug-in, Backbone=LLaMA-13B2025.0776.5
Llama-2#Param=6.9B, Training Tokens=2T, Shots=zero-shot2024.1076.4
MagnitudeBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.1176.4
HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=17.0, Evaluation protocol=zero-shot2026.0476.38
Distributed Lion-AvgShots=3, Optimizer=D-Lion (Avg), Model=LLaMA 7B, Instruction Finetuning=true2024.0376.35
GPT-3 175Balpha=02021.1076.22
Nemotron-4 15B + MaskLLMBackbone=Nemotron-4 15B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.0975.93
PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.0475.51
FP16Model=LLaMA2-7B, Precision=FP16, Evaluation Protocol=Zero-shot2026.0575.46
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.375, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.0475.38
DenseBackbone=LLaMA-1, Model Parameters=7B, Sparsity=0%, Evaluation Protocol=zero-shot2025.1175.34
LLaMA 2 13BModality=Pretrained2023.1075.2
+ MiniLLMrole=Student model, shot=5-shot2024.0775.02
Qwen-1.5 1.8B + DDKDistillation=DDK2024.0775.01
+ TEDrole=Student model, shot=5-shot2024.0774.94
BF16Model=DeepSeek-MoE-16B, Precision=BF16, Evaluation Protocol=Zero-shot2026.0574.87
Jet-Nemotron-2BSearch=200B, Train=400B2026.0574.79
Qwen-1.5 1.8B + TEDDistillation=TED2024.0774.74
Qwen-1.5 1.8B + MiniLLMDistillation=MiniLLM2024.0774.52
Full ModelSetting=Zero-shot, Backbone=LLaMA-13B2025.0774.5
Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.0474.45
DEEPSEEK-V2-LITEPARAMS=16B, Variant=SFT2025.1174.3
GPTQModel=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.0574.28