Science Question Answering on ARC Easy (Accuracy)
86.4AccuracyFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullModel=Qwen1.5MoE-A2.7B, E↓=100%, A↓=100%2026.06 | 86.4 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=61.38%2026.06 | 85.1 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=60%2026.06 | 84.6 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 84.5 | |
| FullModel=DeepSeek-V2-Lite, E↓=100%, A↓=100%2026.06 | 84.1 | |
| GEMMA-7BState=Dense2025.06 | 81.65 | |
| Teacher (GPT-OSS-20B)Evaluation Mode=completion mode2026.05 | 80.9 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 80.1 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 80 | |
| TENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=63.76%2026.06 | 79.1 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=30%2026.06 | 78.2 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=34.51%2026.06 | 78.2 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 77.8 | |
| ENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=60%2026.06 | 77.8 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 76.7 | |
| LLAMA-2-7BState=Dense2025.06 | 76.35 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 75.3 | |
| DEEPSEEK-7BState=Dense2025.06 | 75.25 | |
| VICUNA-1.3-7BState=Dense2025.06 | 74.28 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 74 | |
| CLIMBSize=950M2025.04 | 73.57 | |
| TENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=36.38%2026.06 | 73.3 | |
| SmolLMSize=360M2025.04 | 70.08 | |
| MASKLLMBackbone=LLAMA-2-7B2025.06 | 69.57 | |
| MASKLLMBackbone=DEEPSEEK-7B2025.06 | 68.1 | |
| CLIMBSize=350M2025.04 | 67.38 | |
| MaskProBackbone=DEEPSEEK-7B2025.06 | 66.74 | |
| SPARSEGPTBackbone=DEEPSEEK-7B2025.06 | 66.37 | |
| MaskProBackbone=LLAMA-2-7B2025.06 | 66.12 | |
| AMD-OLMoSize=1.2B2025.04 | 65.95 | |
| GBLMBackbone=DEEPSEEK-7B2025.06 | 65.82 | |
| MASKLLMBackbone=VICUNA-1.3-7B2025.06 | 65.57 | |
| WANDABackbone=DEEPSEEK-7B2025.06 | 65.49 | |
| Llama-3.2Size=1.2B2025.04 | 65.49 | |
| PRUNER-ZEROBackbone=DEEPSEEK-7B2025.06 | 64.94 | |
| Qwen2.5Size=490M2025.04 | 64.6 | |
| SPARSEGPTBackbone=LLAMA-2-7B2025.06 | 64.52 | |
| MaskProBackbone=VICUNA-1.3-7B2025.06 | 64.23 | |
| GBLMBackbone=LLAMA-2-7B2025.06 | 63.17 | |
| SPARSEGPTBackbone=VICUNA-1.3-7B2025.06 | 62.92 | |
| GBLMBackbone=VICUNA-1.3-7B2025.06 | 62.84 | |
| WANDABackbone=LLAMA-2-7B2025.06 | 62.79 | |
| PRUNER-ZEROBackbone=VICUNA-1.3-7B2025.06 | 62.33 | |
| MAGNITUDEBackbone=LLAMA-2-7B2025.06 | 61.87 | |
| WANDABackbone=VICUNA-1.3-7B2025.06 | 61.7 | |
| PRUNER-ZEROBackbone=LLAMA-2-7B2025.06 | 61.32 | |
| X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 61.1 | |
| X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 60.9 | |
| X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 60.5 | |
| TinyLlamaSize=1.1B2025.04 | 60.31 | |
| ENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=30%2026.06 | 59.6 | |
| X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 59.5 | |
| X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 59.1 | |
| X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 58.4 | |
| X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 58.1 | |
| X-GRAM-100%Backbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 57.7 | |
| X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+1.17B, Evaluation Protocol=Zero/Few-shot2026.04 | 56.7 | |
| X-GRAM-50%Backbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+1.87B, Evaluation Protocol=Zero/Few-shot2026.04 | 56.3 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 56.3 | |
| MoRTBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 55.6 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 55.3 | |
| MoRTBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 55.1 | |
| MAGNITUDEBackbone=DEEPSEEK-7B2025.06 | 54.92 | |
| MAGNITUDEBackbone=VICUNA-1.3-7B2025.06 | 54.59 | |
| X-GRAM-100%Backbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 54 | |
| EngramBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 53.5 | |
| EngramBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 53.5 | |
| BaselineBackbone Scale=1.15B Scale, Cfg=N/A, # Params=1.15B, Evaluation Protocol=Zero/Few-shot2026.04 | 53.2 | |
| X-GRAM-50%Backbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 53 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 52.6 | |
| RetokenBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 52.3 | |
| EngramBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 51.6 | |
| RetokenBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 51.2 | |
| RetokenBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 50.7 | |
| MoRTBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 49.6 | |
| MoRTBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 49.3 | |
| EngramBackbone Scale=1.15B Scale, Cfg=1x, # Params=1.15B+3.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 49.1 | |
| RetokenBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 48.8 | |
| EngramBackbone Scale=1.15B Scale, Cfg=2x, # Params=1.15B+7.47B, Evaluation Protocol=Zero/Few-shot2026.04 | 48.6 | |
| MoRTBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 47.9 | |
| MoRTBackbone Scale=0.73B Scale, Cfg=2x, # Params=0.73B+4.67B, Evaluation Protocol=Zero/Few-shot2026.04 | 47.9 | |
| RetokenBackbone Scale=1.15B Scale, Cfg=4x, # Params=1.15B+14.94B, Evaluation Protocol=Zero/Few-shot2026.04 | 47.9 | |
| RetokenBackbone Scale=0.73B Scale, Cfg=1x, # Params=0.73B+2.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 46.1 | |
| EngramBackbone Scale=0.73B Scale, Cfg=4x, # Params=0.73B+9.33B, Evaluation Protocol=Zero/Few-shot2026.04 | 43.5 | |
| BaselineBackbone Scale=0.73B Scale, Cfg=N/A, # Params=0.73B, Evaluation Protocol=Zero/Few-shot2026.04 | 41.4 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 38.4 | |
| DO-ACPScoring=DO-ACP, K=42026.05 | 36.7 | |
| CPScoring=CP, K=42026.05 | 36.4 | |
| ACPScoring=ACP, K=42026.05 | 35.6 | |
| RandomModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 34.3 | |
| SFScoring=SF, K=42026.05 | 34 | |
| ACPScoring=ACP, K=82026.05 | 33.6 | |
| DO-ACPScoring=DO-ACP, K=82026.05 | 33.5 | |
| SFScoring=SF, K=82026.05 | 33.3 | |
| MaskProBackbone=GEMMA-7B2025.06 | 32.92 | |
| WANDABackbone=GEMMA-7B2025.06 | 32.66 | |
| GBLMBackbone=GEMMA-7B2025.06 | 32.38 | |
| CPScoring=CP, K=82026.05 | 32.2 | |
| SPARSEGPTBackbone=GEMMA-7B2025.06 | 30.64 | |
| Random FFN + teacher attn2026.05 | 28.5 |