Commonsense Reasoning on HellaSwag, PIQA, SIQA, and WinoGrande
78.8HellaSwag AccuracyOLMoE-1B-7B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OLMoE-1B-7BNact/Ntotal=1.3B/6.9B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 78.8 | 79.7 | 50.8 | 68.7 | |
| MobileMoE-LNact/Ntotal=922M/5.3B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 73 | 78.9 | 53.4 | 66.1 | |
| SmolLM2 1.7BNact/Ntotal=1.7B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 71.7 | 76.2 | 44.6 | 68.4 | |
| OLMo 2 1BNact/Ntotal=1.5B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 67.3 | 75.2 | 46.1 | 63.5 | |
| MobileMoE-MNact/Ntotal=528M/2.8B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 66.6 | 77.6 | 49 | 63 | |
| Qwen3.5 2BNact/Ntotal=1.9B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 62.2 | 72.8 | 41 | 63 | |
| Llama 3.2 1BNact/Ntotal=1.2B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 61.7 | 74.8 | 43.1 | 61.5 | |
| Gemma 3 1BNact/Ntotal=1.0B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 57.8 | 72.3 | 42 | 59 | |
| SmolLM2 360MNact/Ntotal=362M, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 56.9 | 71.6 | 40.6 | 57.4 | |
| MobileMoE-SNact/Ntotal=272M/1.3B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 56.5 | 74.9 | 42.2 | 58.4 | |
| MobileLLM-ProNact/Ntotal=1.1B, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 52.2 | 73.2 | 42.7 | 51.7 | |
| Qwen3.5 0.8BNact/Ntotal=749M, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 49.7 | 69.4 | 38.8 | 57.6 | |
| Gemma 3 270MNact/Ntotal=270M, Model Type=Instruct-tuned, Evaluation Framework=lm-eval, Few-shot=0-shot2026.05 | 39.4 | 67.1 | 39.6 | 53 |