Science Question Answering on ARC-E
97.53AccuracyQwen3-4B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-4BParams=4B2025.12 | 97.53 | |
| Qwen2.5-3BParams=3B2025.12 | 92.59 | |
| Qwen3-1.7BParams=1.7B2025.12 | 91.89 | |
| Qwen2.5-1.5BParams=1.5B2025.12 | 90.48 | |
| Qwen-14Brole=Teacher model, shot=5-shot2024.07 | 89.24 | |
| SmolLM3-3BParams=3B2025.12 | 88.54 | |
| Qwen-1.5 14B (Teacher)Model Scale=14B2024.07 | 87.58 | |
| LLAMA-4-SCOUTPARAMS=109B, Variant=ORIGINAL2025.11 | 87.3 | |
| LLAMA-4-SCOUTPARAMS=109B, Variant=FCSD2025.11 | 87 | |
| In-Squeezereduction=128 ... -> 1, strategy=Min steps2026.02 | 86 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+700 steps2026.02 | 85.89 | |
| FP16Model=LLAMA3-70B, Evaluation Protocol=Zero-shot, Bit-width=Full-precision2024.03 | 85.86 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+200 steps2026.02 | 85.5 | |
| Direct Fine-tuningrank=1, training_steps=+700 steps2026.02 | 85.39 | |
| In-Squeezereduction=128 ... -> 1, strategy=Standard2026.02 | 85.35 | |
| Direct Fine-tuningrank=1, training_steps=+0 steps2026.02 | 85.2 | |
| Direct Fine-tuningrank=1, training_steps=+200 steps2026.02 | 85.07 | |
| QWEN-3-30B MOEPARAMS=30B, Variant=ORIGINAL2025.11 | 84.8 | |
| Qwen3-0.6BParams=0.6B2025.12 | 84.48 | |
| GenKnowSubBase Model=Phi-2, Setting=En, Evaluation Protocol=zero-shot2025.05 | 84.38 | |
| GenKnowSubBase Model=Phi-2, Setting=Avg, Evaluation Protocol=zero-shot2025.05 | 84.03 | |
| Full-precisionModel=Mixtral 8x22B, Avg. bits/exp.=16 (FP), Memory (GB)=281.2, Evaluation protocol=zero-shot2026.04 | 84.01 | |
| llama-3.2-3BParams=3B2025.12 | 83.95 | |
| BF16Model=Mixtral-8x7B, Precision=BF16, Evaluation Protocol=Zero-shot2026.05 | 83.71 | |
| Qwen2-1.5BParams=1.5B2025.12 | 83.6 | |
| Full-precisionModel=Mixtral 8x7B, Avg. bits/exp.=16 (FP), Memory (GB)=96.8, Evaluation protocol=zero-shot2026.04 | 83.5 | |
| ArrowBase Model=Phi-2, Evaluation Protocol=zero-shot2025.05 | 83.38 | |
| QWEN-3-30B MOEPARAMS=30B, Variant=FCSD2025.11 | 83.3 | |
| PCMind-2.1-Kaiyuan-2BParams=2B2025.12 | 82.89 | |
| SmolLM2-1.7BParams=1.7B2025.12 | 82.72 | |
| gemma2-2BParams=2B2025.12 | 82.54 | |
| YuLan-Mini-2.4BParams=2.4B2025.12 | 82.54 | |
| Phi-2Base Model=Phi-2, Evaluation Protocol=zero-shot2025.05 | 82.11 | |
| DenseBackbone=LLaMA-1, Model Parameters=65B, Sparsity=0%, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 81.4 | |
| SARQC-GBS(Saliency)Model=Mixtral-8x7B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 80.6 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.04 | 80.47 | |
| DenseBackbone=LLaMA-1, Model Parameters=30B, Sparsity=0%, Evaluation Protocol=zero-shot2025.11 | 80.3 | |
| LLAMA-4-SCOUTPARAMS=109B, Variant=SFT2025.11 | 80.3 | |
| DEEPSEEK-V2-LITEPARAMS=16B, Variant=ORIGINAL2025.11 | 80.2 | |
| AutoPruneBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 80.06 | |
| UniformModel=Mixtral 8x7B, Avg. bits/exp.=3, Memory (GB)=19.3, Evaluation protocol=zero-shot2026.04 | 80.05 | |
| BF16Model=Qwen3-MoE-30B, Precision=BF16, Evaluation Protocol=Zero-shot2026.05 | 80.05 | |
| Mistral 7BModality=Pretrained2023.10 | 80 | |
| AutoPruneBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 79.89 | |
| WandaBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 79.75 | |
| SparseGPTBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 79.35 | |
| WandaBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 79.29 | |
| Jet-Nemotron-4BSearch=N/R, Train=400B2026.05 | 79.25 | |
| SARQC-GBS(Saliency)Model=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 79.21 | |
| SARQC-GBS(Identity)Model=Mixtral-8x7B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 79 | |
| FP16Model=LLaMA2-13B, Precision=FP16, Evaluation Protocol=Zero-shot2026.05 | 78.96 | |
| SparseGPTBackbone=LLaMA-1, Model Parameters=30B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot2025.11 | 78.96 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=162021.12 | 78.9 | |
| DEEPSEEK-V2-LITEPARAMS=16B, Variant=FCSD2025.11 | 78.9 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.75, Memory (GB)=17.7, Evaluation protocol=zero-shot2026.04 | 78.87 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.04 | 78.62 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.04 | 78.41 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.625, Memory (GB)=16.9, Evaluation protocol=zero-shot2026.04 | 78.41 | |
| Coherence Boosting (GPT-3 175B)alpha=-0.442021.10 | 78.32 | |
| SARQC-GBS(Identity)Model=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 78.32 | |
| GPTQModel=Mixtral-8x7B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 77.99 | |
| FP16Model=LLAMA3-8B, Evaluation Protocol=Zero-shot, Bit-width=Full-precision2024.03 | 77.82 | |
| GradOTSetting=Plug-in, Backbone=LLaMA-13B2025.07 | 77.8 | |
| Llama3 8B2026.02 | 77.7 | |
| DenseBackbone=LLaMA-1, Model Parameters=13B, Sparsity=0%, Evaluation Protocol=zero-shot2025.11 | 77.4 | |
| G-LionShots=3, Optimizer=Global Lion, Model=LLaMA 7B, Instruction Finetuning=true2024.03 | 77.11 | |
| G-AdamWShots=3, Optimizer=Global AdamW, Model=LLaMA 7B, Instruction Finetuning=true2024.03 | 77.06 | |
| DASH-3Budget=17.75, Search=12.3M / run, Train=700M2026.05 | 76.98 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=16.1, Evaluation protocol=zero-shot2026.04 | 76.94 | |
| Distributed Lion-MaVoShots=3, Optimizer=D-Lion (MaVo), Model=LLaMA 7B, Instruction Finetuning=true2024.03 | 76.86 | |
| DASH-3Budget=9.0, Search=12.3M / run, Train=700M2026.05 | 76.85 | |
| OLMo-2-0425-1BParams=1B2025.12 | 76.72 | |
| UniformModel=Mixtral 8x22B, Avg. bits/exp.=3, Memory (GB)=57.5, Evaluation protocol=zero-shot2026.04 | 76.68 | |
| DASH-3Budget=4.875, Search=12.3M / run, Train=700M2026.05 | 76.68 | |
| LLaMA 7B (Baseline)Shots=0, Instruction Finetuning=false, Model=LLaMA 7B2024.03 | 76.64 | |
| + DDKrole=Student model, shot=5-shot2024.07 | 76.64 | |
| GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.12 | 76.6 | |
| OT†Setting=Plug-in, Backbone=LLaMA-13B2025.07 | 76.5 | |
| Llama-2#Param=6.9B, Training Tokens=2T, Shots=zero-shot2024.10 | 76.4 | |
| MagnitudeBackbone=LLaMA-1, Model Parameters=65B, Sparsity=50% unstructured, Evaluation Protocol=zero-shot, Quantization=8-bit2025.11 | 76.4 | |
| HessianModel=Mixtral 8x7B, Avg. bits/exp.=2.5, Memory (GB)=17.0, Evaluation protocol=zero-shot2026.04 | 76.38 | |
| Distributed Lion-AvgShots=3, Optimizer=D-Lion (Avg), Model=LLaMA 7B, Instruction Finetuning=true2024.03 | 76.35 | |
| GPT-3 175Balpha=02021.10 | 76.22 | |
| Nemotron-4 15B + MaskLLMBackbone=Nemotron-4 15B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.09 | 75.93 | |
| PMQModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.04 | 75.51 | |
| FP16Model=LLaMA2-7B, Precision=FP16, Evaluation Protocol=Zero-shot2026.05 | 75.46 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.375, Memory (GB)=15.3, Evaluation protocol=zero-shot2026.04 | 75.38 | |
| DenseBackbone=LLaMA-1, Model Parameters=7B, Sparsity=0%, Evaluation Protocol=zero-shot2025.11 | 75.34 | |
| LLaMA 2 13BModality=Pretrained2023.10 | 75.2 | |
| + MiniLLMrole=Student model, shot=5-shot2024.07 | 75.02 | |
| Qwen-1.5 1.8B + DDKDistillation=DDK2024.07 | 75.01 | |
| + TEDrole=Student model, shot=5-shot2024.07 | 74.94 | |
| BF16Model=DeepSeek-MoE-16B, Precision=BF16, Evaluation Protocol=Zero-shot2026.05 | 74.87 | |
| Jet-Nemotron-2BSearch=200B, Train=400B2026.05 | 74.79 | |
| Qwen-1.5 1.8B + TEDDistillation=TED2024.07 | 74.74 | |
| Qwen-1.5 1.8B + MiniLLMDistillation=MiniLLM2024.07 | 74.52 | |
| Full ModelSetting=Zero-shot, Backbone=LLaMA-13B2025.07 | 74.5 | |
| Router norm + Max varModel=Mixtral 8x7B, Avg. bits/exp.=2.125, Memory (GB)=13.8, Evaluation protocol=zero-shot2026.04 | 74.45 | |
| DEEPSEEK-V2-LITEPARAMS=16B, Variant=SFT2025.11 | 74.3 | |
| GPTQModel=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 74.28 |