Science Question Answering on ARC-Challenge, ARC-Easy, and OpenBookQA
57.9ARC-C AccuracyMobileMoE-L
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MobileMoE-LNact/Ntotal=922M/5.3B, Model Type=Instruct-tuned2026.05 | 57.9 | 81.9 | 43.2 | |
| OLMoE-1B-7BNact/Ntotal=1.3B/6.9B, Model Type=Instruct-tuned2026.05 | 57.6 | 75.9 | 46.8 | |
| SmolLM2 1.7BNact/Ntotal=1.7B, Model Type=Instruct-tuned2026.05 | 53.4 | 62.9 | 45.8 | |
| Qwen3.5 2BNact/Ntotal=1.9B, Model Type=Instruct-tuned2026.05 | 53.2 | 66 | 35.2 | |
| MobileMoE-MNact/Ntotal=528M/2.8B, Model Type=Instruct-tuned2026.05 | 52.5 | 79.9 | 38.2 | |
| OLMo 2 1BNact/Ntotal=1.5B, Model Type=Instruct-tuned2026.05 | 45.1 | 69.8 | 42.2 | |
| MobileMoE-SNact/Ntotal=272M/1.3B, Model Type=Instruct-tuned2026.05 | 43.1 | 73.4 | 32.4 | |
| Llama 3.2 1BNact/Ntotal=1.2B, Model Type=Instruct-tuned2026.05 | 42.1 | 63.7 | 37.8 | |
| Qwen3.5 0.8BNact/Ntotal=749M, Model Type=Instruct-tuned2026.05 | 41.8 | 61.4 | 30.8 | |
| Gemma 3 1BNact/Ntotal=1.0B, Model Type=Instruct-tuned2026.05 | 40.1 | 63.1 | 38.4 | |
| SmolLM2 360MNact/Ntotal=362M, Model Type=Instruct-tuned2026.05 | 38.8 | 49.1 | 36.2 | |
| MobileLLM-ProNact/Ntotal=1.1B, Model Type=Instruct-tuned2026.05 | 35.8 | 52.7 | 29.6 | |
| Gemma 3 270MNact/Ntotal=270M, Model Type=Instruct-tuned2026.05 | 27.7 | 50.5 | 35 |