Science Question Answering on ARC Challenge (Character-level Accuracy)
76.1Character-level AccuracyFull
Evaluation Results
| Method | Links | |
|---|---|---|
| FullModel=Qwen1.5MoE-A2.7B, E↓=100%, A↓=100%2026.06 | 76.1 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=61.38%2026.06 | 75.1 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=60%2026.06 | 74.9 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 73 | |
| FullModel=DeepSeek-V2-Lite, E↓=100%, A↓=100%2026.06 | 70.3 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 68.3 | |
| TENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=34.51%2026.06 | 66.9 | |
| TENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=63.76%2026.06 | 66.8 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 66.7 | |
| ENPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=30%2026.06 | 66 | |
| ENPModel=DeepSeek-V2-Lite, E↓=60%, A↓=60%2026.06 | 65.7 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=60%, A↓=100%2026.06 | 64.5 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 62.7 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 61.8 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 57.5 | |
| TENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=36.38%2026.06 | 57.1 | |
| LFM2 1.2BParameter Count=1.2B, Evaluation Protocol=25-shot2026.03 | 52.2 | |
| MobileLLM-Flash 1.4BParameter Count=1.4B, Evaluation Protocol=25-shot2026.03 | 50.56 | |
| LFM2 700MParameter Count=700M, Evaluation Protocol=25-shot2026.03 | 49.4 | |
| ENPModel=DeepSeek-V2-Lite, E↓=30%, A↓=30%2026.06 | 48.4 | |
| EASY-EPModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 44.8 | |
| LFM2 350MParameter Count=350M, Evaluation Protocol=25-shot2026.03 | 44.54 | |
| FrequencyModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 44.1 | |
| Gating ScoreModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 43.2 | |
| MobileLLM-Flash 650MParameter Count=650M, Evaluation Protocol=25-shot2026.03 | 42.58 | |
| Nemotron-Flash 1BParameter Count=1B, Evaluation Protocol=25-shot2026.03 | 41.47 | |
| MobileLLM-Flash 350MParameter Count=350M, Evaluation Protocol=25-shot2026.03 | 38.89 | |
| Qwen3 0.6BParameter Count=0.6B, Evaluation Protocol=25-shot2026.03 | 38.57 | |
| Gemma3 1BParameter Count=1B, Evaluation Protocol=25-shot2026.03 | 38.4 | |
| Llama3.2 1BParameter Count=1B, Evaluation Protocol=25-shot2026.03 | 38.28 | |
| RandomModel=DeepSeek-V2-Lite, E↓=60%, A↓=100%2026.06 | 31.7 | |
| EASY-EPModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 29.6 | |
| Gemma3 270MParameter Count=270M, Evaluation Protocol=25-shot2026.03 | 29 | |
| Gating ScoreModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 27.8 | |
| RandomModel=Qwen1.5MoE-A2.7B, E↓=30%, A↓=100%2026.06 | 25.7 | |
| RandomModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 24.2 | |
| FrequencyModel=DeepSeek-V2-Lite, E↓=30%, A↓=100%2026.06 | 24 |